From larry at hastings.org  Fri Mar  1 00:02:58 2013
From: larry at hastings.org (Larry Hastings)
Date: Thu, 28 Feb 2013 15:02:58 -0800
Subject: [Python-Dev] Announcing PEP 436: The Argument Clinic DSL
In-Reply-To: <kgjr4i$qkn$1@ger.gmane.org>
References: <512BFDAB.5050103@hastings.org> <kgimr1$q5k$1@ger.gmane.org>
	<512D0342.2060603@hastings.org> <kgjr4i$qkn$1@ger.gmane.org>
Message-ID: <512FE222.8000401@hastings.org>

On 02/26/2013 06:30 PM, Terry Reedy wrote:
> On 2/26/2013 1:47 PM, Larry Hastings wrote:
>> I think positional-only functions should be discouraged, but we don't
>
> If I were writing something like Clinic, I would be tempted to not 
> have that option. But I was actually thinking about something in the 
> positional-only writeup that mentioned the possibility of adding 
> something to the positional-only option.

Clinic needs to be usable for every builtin in order to be a credible 
solution.  And the positional-only approach to parsing is sufficiently 
different from the positional-and-keywords approach that I couldn't 
sweep the conceptual difference under the rug--I had to explicitly 
support weird stuff like optional positional arguments on the /left/.  
So I really don't have a choice.  All we can do is say "please don't use 
this for new code".

> As I understand it, C module files are structured something like the 
> following, which is 'unusual' for a python file.
>
> def meth1_impl(...
>
> def meth2_impl(...
>
> class C:
>   meth1 = meth1_impl
>   meth2 = meth2_impl

At the moment Clinic is agnostic about where in Python the callable 
lives.  The "module name" in the DSL is really just used in 
documentation and to construct the name of the static functions.  So if 
you specify a class name as your "module name" it'll work fine and look 
correct.  However, maybe we need to think about this a little more when 
it comes time to add Signature metadata.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130228/2c15504a/attachment.html>

From doug.hellmann at gmail.com  Fri Mar  1 00:39:21 2013
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Thu, 28 Feb 2013 18:39:21 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
Message-ID: <31B54506-7B08-4E77-B966-A6171BBC3DA6@gmail.com>


On Feb 27, 2013, at 11:51 AM, Michael Foord wrote:

> Hello all,
> 
> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
> 
> The agenda of topics for discussion so far includes the following:
> 
> * A report on pypy status - Maciej and Armin
> * Jython and IronPython status reports - Dino / Frank 
> * Packaging (Doug Hellmann and Monty Taylor at least)

Since the time I suggested we add packaging to the agenda, Nick has set up a separate summit meeting for Friday evening. I don't know if it makes sense to leave this on the agenda for Wednesday or not.

Nick, what do you think?

Doug

> * Cleaning up interpreter initialisation (both in hopes of finding areas
>  to rationalise and hence speed things up, as well as making things
>  more embedding friendly). Nick Coghlan
> * Adding new async capabilities to the standard library (Guido)
> * cffi and the standard library - Maciej
> * flufl.enum and the standard library - Barry Warsaw
> * The argument clinic - Larry Hastings
> 
> If you have other items you'd like to discuss please let me know and I can add them to the agenda.
> 
> All the best,
> 
> Michael Foord
> 
> --
> http://www.voidspace.org.uk/
> 
> 
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing 
> http://www.sqlite.org/different.html
> 
> 
> 
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/doug.hellmann%40gmail.com


From stefan_ml at behnel.de  Fri Mar  1 10:41:00 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Fri, 01 Mar 2013 10:41:00 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
Message-ID: <kgpt39$nc5$1@ger.gmane.org>

Michael Foord, 27.02.2013 17:51:
> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
> 
> The agenda of topics for discussion so far includes the following:
> 
> * A report on pypy status - Maciej and Armin
> * Jython and IronPython status reports - Dino / Frank 
> * Packaging (Doug Hellmann and Monty Taylor at least)
> * Cleaning up interpreter initialisation (both in hopes of finding areas
>   to rationalise and hence speed things up, as well as making things
>   more embedding friendly). Nick Coghlan
> * Adding new async capabilities to the standard library (Guido)
> * cffi and the standard library - Maciej
> * flufl.enum and the standard library - Barry Warsaw
> * The argument clinic - Larry Hastings

Hi,

as in the years before, none of the Cython developers is attending the
PyCon-US, so we won't appear that the language summit either. It's a bit
sad that it always takes place at that venue, but I guess there'll just
always be people that can't come to one meeting or the other, so PyCon-US
would just catch the majority. I think it would still be interesting for
many of the attendants to get a status report about Cython, as there seems
to be some confusion and incomplete knowledge about what Cython actually
is, what we have achieved and where we are heading. But maybe the confusion
is large enough to require more than just a little status report to clear
it up.

It's also true that many of the topics above aren't really interesting for
us, because we just inherit them with CPython, e.g. stdlib changes.
Packaging is only relevant as far as it impacts the distribution of binary
extensions, and the main changes appear to be outside of that area (which
doesn't mean it's not truly wonderful that they are happening, Python
packaging has seen a lot of great improvements during the last years and
I'm very happy to see it getting even better).

Interpreter initialisation would be interesting and Cython could
potentially help in some spots here by making code easier to maintain and
optimise, for example. We've had this discussion for the importlib
bootstrapping and I'm sure there's more that could be done. It's sad to see
so much C-level work go into areas that really don't need to be that low-level.

I'm not so happy with the argument clinic, but that's certainly also
because I'm biased. I've written the argument unpacking code for Cython
some years ago, so it's not surprising that I'm quite happy with that and
fail to see the need for a totally new DSL *and* a totally new
implementation, especially with its mapping to the slowish ParseTuple*()
C-API functions. I've also not seen a good argument why the existing Py3
function signatures can't do what the proposed DSL tries to achieve. They'd
at least make it clear that the intention is to make things more
Python-like, and would at the same time provide the documentation.

The topics that would be interesting for us sound more like they'd benefit
from a "CPython runtime summit". I really think that it would be beneficial
for the CPython developers to learn how we solved problems that they have
on their lists or might at least run into in a couple of years, and for us
to see if we can't come up with cleaner solutions for problems that CPython
currently makes hard to do outside of the core. For example, making
C-implemented code Python compatible is actually not trivial and has cost
us a lot of investment. Nowadays, CPython is actually further away from
that in some areas than Cython, and I don't think it needs to stay that
way. It could certainly help both Cython and CPython if CPython gained some
of the capabilities for itself that we had to implement ourselves in clean
or hacky ways, but always outside of the core. There isn't really a reason
why C-implemented parts of CPython must behave all that different from
Python implemented parts, why modules must have a different API than other
objects, why builtins can't accept keyword arguments, ... These things just
get in the way once their camouflage as features is blown up.

Another topic is C-level calling between extensions that only see each
other through the CPython core. Python call semantics are nice, but also
extremely slow compared to C calls. Capsules are simple but slow and
static. Implementing a dynamic C calling interface between extensions with
a safe way to pass C signatures around and validate them (or adapt to them)
on the other side would be much easier with CPython support built into its
C function type than trying to do this outside of the core. And there's a
huge area of applications for such a feature, especially with the
increasing number of tools that do dynamic code generation on the CPython
platform. Eventually, this might also become interesting for non-CPythons,
as it might provide a way to interface efficiently with CPython extensions.

So, have fun at the language summit everyone, I'm looking forward to
skipping through the writeup. And I'd really like to see a CPython summit
happen at some point. There's so much interesting stuff going on in that
area that it's worth getting some people together to move these things forward.

Stefan



From solipsis at pitrou.net  Fri Mar  1 11:33:24 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 1 Mar 2013 11:33:24 +0100
Subject: [Python-Dev] High volumes and off topic discussions
References: <CAK5idxQcV=xFefGhoGFBzQoo6YxQTD9TdGmiN6qcp8MrGcWUyA@mail.gmail.com>
	<20130228150326.2e7b0944@pitrou.net>
	<CAK5idxR5G3qJYKwnE==kw6SatdnaNUMh2_+KBYD+qjUvXGLFZQ@mail.gmail.com>
Message-ID: <20130301113324.67ce5107@pitrou.net>

Le Thu, 28 Feb 2013 19:17:39 +0200,
Maciej Fijalkowski <fijall at gmail.com> a ?crit :
> On Thu, Feb 28, 2013 at 4:03 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > Le Thu, 28 Feb 2013 13:36:10 +0200,
> > Maciej Fijalkowski <fijall at gmail.com> a ?crit :
> >> Hi
> >>
> >> I know this is a hard topic, but python-dev is already incredibly
> >> high-volume and dragging discussion off-topic is making following
> >> important stuff (while ignoring unimportant stuff) very hard.
> >>
> >> For example in a recent topic "cffi in stdlib" I find a mail that
> >> says "we have to find a sufficiently silly species of snake". It's
> >> even funny, but it definitely makes it very hard to follow for
> >> those of us who don't read python-dev 24/7. Would it be reasonable
> >> for python-dev to generally try to stay on topic (for example if
> >> the thread is called "silly species of snakes", I absolutely don't
> >> mind people posting there whatever they feel like as long as I'm
> >> not expected to read every single message).
> >
> > I'm afraid you're trying to devise derogatory distinctions regarding
> > drifting discussions.
> >
> > Seriously, yes, I approve of changing the subject line, although I
> > forgot to do it this time.
> > For the record, you can also read the list through a NNTP gateway
> > using Gmane, it can make things easier.
> 
> How does that help with knowing what mails to read what mails not to
> read?

It doesn't, but at least it won't flood your personal inbox ;-)

Regards

Antoine.



From ezio.melotti at gmail.com  Fri Mar  1 14:22:57 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Fri, 1 Mar 2013 15:22:57 +0200
Subject: [Python-Dev] [Python-checkins] cpython (3.3): Don't deadlock on
	a reentrant call.
In-Reply-To: <3ZHTky0hvLzSyt@mail.python.org>
References: <3ZHTky0hvLzSyt@mail.python.org>
Message-ID: <CACBhJdHmhQi1Q8KOaPQcrmOkuW5mqU5adUu2+YgEBtk177ZR2A@mail.gmail.com>

Hi,

On Fri, Mar 1, 2013 at 2:02 PM, raymond.hettinger
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/1920422626a5
> changeset:   82437:1920422626a5
> branch:      3.3
> parent:      82435:43ac02b7e322
> user:        Raymond Hettinger <python at rcn.com>
> date:        Fri Mar 01 03:47:57 2013 -0800
> summary:
>   Don't deadlock on a reentrant call.

this seems to have broken builds without threads.
After this commit I get a compile error:
$ ./configure --without-threads --with-pydebug && make -j2
[...]
./python -E -S -m sysconfig --generate-posix-vars
Could not find platform dependent libraries <exec_prefix>
Consider setting $PYTHONHOME to <prefix>[:<exec_prefix>]
Could not import runpy module
Exception ignored in: 'garbage collection'
Traceback (most recent call last):
  File "/home/wolf/dev/py/py3k/Lib/runpy.py", line 16, in <module>
    import imp
  File "/home/wolf/dev/py/py3k/Lib/imp.py", line 23, in <module>
    import tokenize
  File "/home/wolf/dev/py/py3k/Lib/tokenize.py", line 28, in <module>
    import re
  File "/home/wolf/dev/py/py3k/Lib/re.py", line 124, in <module>
    import functools
  File "/home/wolf/dev/py/py3k/Lib/functools.py", line 22, in <module>
    from dummy_threading import RLock
  File "/home/wolf/dev/py/py3k/Lib/dummy_threading.py", line 45, in <module>
    import threading
  File "/home/wolf/dev/py/py3k/Lib/threading.py", line 6, in <module>
    from time import sleep as _sleep
ImportError: No module named 'time'
Fatal Python error: unexpected exception during garbage collection

Current thread 0x00000000:
make: *** [pybuilddir.txt] Aborted (core dumped)

See also:
http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.x/builds/4006
http://buildbot.python.org/all/builders/AMD64%20Fedora%20without%20threads%203.3/builds/516

(Also having tests for this change would be nice.)

Best Regards,
Ezio Melotti

From barry at python.org  Fri Mar  1 15:32:23 2013
From: barry at python.org (Barry Warsaw)
Date: Fri, 1 Mar 2013 09:32:23 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
Message-ID: <20130301093223.743a04c8@anarchist.wooz.org>

On Feb 28, 2013, at 08:44 AM, fwierzbicki at gmail.com wrote:

>Sorry I meant "is_jython" as a sort of shorthand for a case by case
>check. It would be cool if we had a nice set of checks somewhere like
>"is_refcounted", etc. Would the sys.implementation area be a good
>place for such things?

Yep.  Unless it proves too unwieldy I suppose.

>On the other hand in some ways Jython is sort of like Python on a
>weird virtual OS that lets the real OS bleed through some. This may
>still need to be checked in that way (there's are still checks of <if
>os.name == 'nt'> right?)

Yeah, but that all ooooold code ;)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130301/642b0e58/attachment.pgp>

From brett at python.org  Fri Mar  1 15:57:59 2013
From: brett at python.org (Brett Cannon)
Date: Fri, 1 Mar 2013 09:57:59 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <kgpt39$nc5$1@ger.gmane.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgpt39$nc5$1@ger.gmane.org>
Message-ID: <CAP1=2W4yz_ZBY7tG=kzttLNTk53qavV52xoLT3RMhA0J7FPUpA@mail.gmail.com>

On Fri, Mar 1, 2013 at 4:41 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Michael Foord, 27.02.2013 17:51:
> > PyCon, and the Python Language Summit, is nearly upon us. We have a good
> number of people confirmed to attend. If you are intending to come to the
> language summit but haven't let me know please do so.
> >
> > The agenda of topics for discussion so far includes the following:
> >
> > * A report on pypy status - Maciej and Armin
> > * Jython and IronPython status reports - Dino / Frank
> > * Packaging (Doug Hellmann and Monty Taylor at least)
> > * Cleaning up interpreter initialisation (both in hopes of finding areas
> >   to rationalise and hence speed things up, as well as making things
> >   more embedding friendly). Nick Coghlan
> > * Adding new async capabilities to the standard library (Guido)
> > * cffi and the standard library - Maciej
> > * flufl.enum and the standard library - Barry Warsaw
> > * The argument clinic - Larry Hastings
>
> Hi,
>
> as in the years before, none of the Cython developers is attending the
> PyCon-US, so we won't appear that the language summit either. It's a bit
> sad that it always takes place at that venue, but I guess there'll just
> always be people that can't come to one meeting or the other, so PyCon-US
> would just catch the majority. I think it would still be interesting for
> many of the attendants to get a status report about Cython, as there seems
> to be some confusion and incomplete knowledge about what Cython actually
> is, what we have achieved and where we are heading. But maybe the confusion
> is large enough to require more than just a little status report to clear
> it up.
>

There are actually two language summits each year: PyCon US and EuroPython.
But you are right that the US one is the biggest as it's the easiest to get
the most core devs in a single room.

Hopefully you can make PyCon US (it's a great conference). And if it's a US
issue, it will be in Canada in 2014 and 2015.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130301/33ebf21a/attachment.html>

From stefan.bucur at gmail.com  Fri Mar  1 16:24:42 2013
From: stefan.bucur at gmail.com (Stefan Bucur)
Date: Fri, 1 Mar 2013 16:24:42 +0100
Subject: [Python-Dev] Disabling string interning for null and single-char
	causes segfaults
Message-ID: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>

Hi,

I'm working on an automated bug finding tool that I'm trying to apply on
the Python interpreter code (version 2.7.3). Because of early prototype
limitations, I needed to disable string interning in stringobject.c. More
precisely, I modified the PyString_FromStringAndSize and
PyString_FromString to no longer check for the null and single-char cases,
and create instead a new string every time (I can send the patch if needed).

However, after applying this modification, when running "make test" I get a
segfault in the test___all__ test case.

Before digging deeper into the issue, I wanted to ask here if there are any
implicit assumptions about string identity and interning throughout the
interpreter implementation. For instance, are two single-char strings
having the same content supposed to be identical objects?

I'm assuming that it's either this, or some refcount bug in the interpreter
that manifests only when certain strings are no longer interned and thus
have a higher chance to get low refcount values.

Thanks a lot,
Stefan Bucur
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130301/c86e3b52/attachment.html>

From demianbrecht at gmail.com  Fri Mar  1 16:43:06 2013
From: demianbrecht at gmail.com (Demian Brecht)
Date: Fri, 1 Mar 2013 07:43:06 -0800
Subject: [Python-Dev] FileCookieJars
Message-ID: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>

Cross-posting from python-ideas due to no response there. Perhaps it's
due to a general lack of usage/caring for cookiejar, but figured
/someone/'s got to have an opinion about my proposal ;)

Note that I've moved my discussion from bug 16942 to 16901
(http://bugs.python.org/issue16901) as they're duplicates and 16901 is
more succinct. I've also posted a patch in 16901 implementing my
proposal.

TL;DR: CookieJar > FileCookieJar > *CookieJar are architecturally
broken and this is an attempt to rectify that (and fix a couple bugs
along the way).

---------------

Context: http://bugs.python.org/issue16942 (my patch, changing
FileCookieJar to be an abc, defining the interfaces for
*FileCookieJar).

This pertains to Terry's question about whether or not it makes sense
that an abstract base class extends a concrete class. After putting in
a little thought, he's right. It doesn't make sense.

After further thought, I'm relatively confident that the hierarchy as
it stands should be changed. Currently what's implemented in the
stdlib looks like this:

CookieJar
|
FileCookieJar
| |
| MozillaCookieJar
LWPCookieJar

What I'm proposing is that the structure is broken to be the following:

FileCookieJarProcessor             CookieJar
| |
| MozillaCookieJarProcessor
LWPCookieJarProcessor

The intention here is to have processors that operate /on/ a cookiejar
object via composition rather than inheritance. This aligns with how
urllib.request.HTTPCookieProcessor works (which has the added bonus of
cross-module consistency).

The only attributes that concrete FileCookieJarProcessor classes touch
(at least, in the stdlib) are _cookies and _cookies_lock. I have mixed
feelings about whether these should continue to be noted as
"non-public" with the _ prefix or not as keeping the _ would break
convention of operating on non-public fields, but am unsure of the
ramifications of changing them to public.

Making this change then allows for FileCookieJar(Processor) to be an
abstract base class without inheriting from CookieJar which doesn't
make a whole lot of sense from an architecture standpoint.

I have yet to see what impact these changes have to the cookiejar
extensions at http://wwwsearch.sf.net but plan on doing so if this
approach seems sound.

This will obviously break backwards compatibility, so I'm not entirely
sure what best practice is around that: leave well enough alone even
though it might not make sense, keep the old implementations around
and deprecate them to be eventually replaced by the processors, or
other ideas?


--
Demian Brecht
http://demianbrecht.github.com

From status at bugs.python.org  Fri Mar  1 18:07:29 2013
From: status at bugs.python.org (Python tracker)
Date: Fri,  1 Mar 2013 18:07:29 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20130301170729.38A1356A38@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2013-02-22 - 2013-03-01)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3877 (+11)
  closed 25227 (+41)
  total  29104 (+52)

Open issues with patches: 1695 


Issues opened (42)
==================

#16121: shlex.shlex.error_leader() reports incorrect line number
http://bugs.python.org/issue16121  reopened by r.david.murray

#17223: Initializing array.array with unicode type code and buffer seg
http://bugs.python.org/issue17223  reopened by sbt

#17278: SIGSEGV in _heapqmodule.c
http://bugs.python.org/issue17278  opened by maxxedev

#17279: Document which named built-in classes can be subclassed
http://bugs.python.org/issue17279  opened by terry.reedy

#17281: Broken links at pypi
http://bugs.python.org/issue17281  opened by lyda

#17282: document the defaultTest parameter to unittest.main()
http://bugs.python.org/issue17282  opened by chris.jerdonek

#17284: create mercurial section in devguide's committing.rst
http://bugs.python.org/issue17284  opened by chris.jerdonek

#17285: subprocess.check_output incorrectly state that output is alway
http://bugs.python.org/issue17285  opened by Baptiste.Lepilleur

#17286: Make subprocess handling text output with universal_newlines m
http://bugs.python.org/issue17286  opened by Baptiste.Lepilleur

#17288: cannot jump from a return after setting f_lineno
http://bugs.python.org/issue17288  opened by xdegaye

#17289: readline.set_completer_delims() doesn't play well with others
http://bugs.python.org/issue17289  opened by bfroehle

#17293: uuid.getnode() MAC address on AIX
http://bugs.python.org/issue17293  opened by aivarsk

#17294: compile-flag for single-execution to return value instead of p
http://bugs.python.org/issue17294  opened by Albert.Zeyer

#17295: __slots__ on PyVarObject subclass
http://bugs.python.org/issue17295  opened by ronaldoussoren

#17298: Twisted test failure triggered by change in 2.7 branch
http://bugs.python.org/issue17298  opened by glyph

#17299: Test cPickle with real files
http://bugs.python.org/issue17299  opened by serhiy.storchaka

#17301: An in-place version of many bytearray methods is needed
http://bugs.python.org/issue17301  opened by gregory.p.smith

#17302: HTTP/2.0 - Implementations/Testing efforts
http://bugs.python.org/issue17302  opened by karlcow

#17305: IDNA2008 encoding missing
http://bugs.python.org/issue17305  opened by marten

#17306: Improve the way abstract base classes are shown in help()
http://bugs.python.org/issue17306  opened by rhettinger

#17307: HTTP PUT request Example
http://bugs.python.org/issue17307  opened by orsenthil

#17308: Dialog.py crashes when putty Window resized
http://bugs.python.org/issue17308  opened by harshaap

#17309: __bytes__ doesn't work in subclass of int
http://bugs.python.org/issue17309  opened by pkoning

#17310: Ctypes callbacks shows problem on Windows Python (64bit)
http://bugs.python.org/issue17310  opened by Matt.Clarke

#17311: use distutils terminology in "PyPI package display" section
http://bugs.python.org/issue17311  opened by chris.jerdonek

#17312: test_aifc doesn't clean up after itself
http://bugs.python.org/issue17312  opened by chris.jerdonek

#17314: Stop using imp.find_module() in multiprocessing
http://bugs.python.org/issue17314  opened by brett.cannon

#17315: test_posixpath doesn't clean up after itself
http://bugs.python.org/issue17315  opened by chris.jerdonek

#17316: Add Django 1.5 to benchmarks
http://bugs.python.org/issue17316  opened by brett.cannon

#17317: Benchmark driver should calculate actual benchmark count in -h
http://bugs.python.org/issue17317  opened by brett.cannon

#17318: xml.sax and xml.dom fetch DTDs by default
http://bugs.python.org/issue17318  opened by rsandwick3

#17319: http.server.BaseHTTPRequestHandler send_response_only doesn't 
http://bugs.python.org/issue17319  opened by karlcow

#17320: os.path.abspath in window7, return error
http://bugs.python.org/issue17320  opened by xiaowei.py

#17322: urllib.request add_header() currently allows trailing spaces (
http://bugs.python.org/issue17322  opened by karlcow

#17323: Disable [X refs, Y blocks] ouput in debug builds
http://bugs.python.org/issue17323  opened by ezio.melotti

#17324: SimpleHTTPServer serves files even if the URL has a trailing s
http://bugs.python.org/issue17324  opened by larry

#17325: improve organization of the PyPI distutils docs
http://bugs.python.org/issue17325  opened by chris.jerdonek

#17326: Windows build docs still referring to VS 2008 in 3.3
http://bugs.python.org/issue17326  opened by cito

#17327: Add PyDict_GetItemSetDefault() as C-API for dict.setdefault()
http://bugs.python.org/issue17327  opened by scoder

#17328: Fix reference leak in dict_setdefault() in case of resize fail
http://bugs.python.org/issue17328  opened by scoder

#17329: Document unittest.SkipTest
http://bugs.python.org/issue17329  opened by brett.cannon

#17296: Cannot unpickle classes derived from 'Exception'
http://bugs.python.org/issue17296  opened by Andreas.Hausmann



Most recent 15 issues with no replies (15)
==========================================

#17329: Document unittest.SkipTest
http://bugs.python.org/issue17329

#17328: Fix reference leak in dict_setdefault() in case of resize fail
http://bugs.python.org/issue17328

#17327: Add PyDict_GetItemSetDefault() as C-API for dict.setdefault()
http://bugs.python.org/issue17327

#17326: Windows build docs still referring to VS 2008 in 3.3
http://bugs.python.org/issue17326

#17325: improve organization of the PyPI distutils docs
http://bugs.python.org/issue17325

#17320: os.path.abspath in window7, return error
http://bugs.python.org/issue17320

#17319: http.server.BaseHTTPRequestHandler send_response_only doesn't 
http://bugs.python.org/issue17319

#17317: Benchmark driver should calculate actual benchmark count in -h
http://bugs.python.org/issue17317

#17316: Add Django 1.5 to benchmarks
http://bugs.python.org/issue17316

#17315: test_posixpath doesn't clean up after itself
http://bugs.python.org/issue17315

#17312: test_aifc doesn't clean up after itself
http://bugs.python.org/issue17312

#17310: Ctypes callbacks shows problem on Windows Python (64bit)
http://bugs.python.org/issue17310

#17309: __bytes__ doesn't work in subclass of int
http://bugs.python.org/issue17309

#17308: Dialog.py crashes when putty Window resized
http://bugs.python.org/issue17308

#17306: Improve the way abstract base classes are shown in help()
http://bugs.python.org/issue17306



Most recent 15 issues waiting for review (15)
=============================================

#17328: Fix reference leak in dict_setdefault() in case of resize fail
http://bugs.python.org/issue17328

#17327: Add PyDict_GetItemSetDefault() as C-API for dict.setdefault()
http://bugs.python.org/issue17327

#17325: improve organization of the PyPI distutils docs
http://bugs.python.org/issue17325

#17314: Stop using imp.find_module() in multiprocessing
http://bugs.python.org/issue17314

#17307: HTTP PUT request Example
http://bugs.python.org/issue17307

#17296: Cannot unpickle classes derived from 'Exception'
http://bugs.python.org/issue17296

#17293: uuid.getnode() MAC address on AIX
http://bugs.python.org/issue17293

#17288: cannot jump from a return after setting f_lineno
http://bugs.python.org/issue17288

#17284: create mercurial section in devguide's committing.rst
http://bugs.python.org/issue17284

#17278: SIGSEGV in _heapqmodule.c
http://bugs.python.org/issue17278

#17277: incorrect line numbers in backtrace after removing a trace fun
http://bugs.python.org/issue17277

#17272: request.full_url: unexpected results on assignment
http://bugs.python.org/issue17272

#17269: getaddrinfo segfaults on OS X when provided with invalid argum
http://bugs.python.org/issue17269

#17267: datetime.time support for '+' and 'now'
http://bugs.python.org/issue17267

#17264: Update Building C and C++ Extensions with distutils documentat
http://bugs.python.org/issue17264



Top 10 most discussed issues (10)
=================================

#17263: crash when tp_dealloc allows other threads
http://bugs.python.org/issue17263  25 msgs

#14468: Update cloning guidelines in devguide
http://bugs.python.org/issue14468  18 msgs

#16930: mention limitations and/or alternatives to hg graft
http://bugs.python.org/issue16930  13 msgs

#17223: Initializing array.array with unicode type code and buffer seg
http://bugs.python.org/issue17223  13 msgs

#12641: Remove -mno-cygwin from distutils
http://bugs.python.org/issue12641  11 msgs

#10572: Move test sub-packages to Lib/test
http://bugs.python.org/issue10572  10 msgs

#16121: shlex.shlex.error_leader() reports incorrect line number
http://bugs.python.org/issue16121  10 msgs

#17267: datetime.time support for '+' and 'now'
http://bugs.python.org/issue17267  10 msgs

#15083: Rewrite ElementTree tests in a cleaner and safer way
http://bugs.python.org/issue15083   9 msgs

#15305: Test harness unnecessarily disambiguating twice
http://bugs.python.org/issue15305   8 msgs



Issues closed (37)
==================

#5033: setup.py crashes if sqlite version contains 'beta'
http://bugs.python.org/issue5033  closed by petri.lehtinen

#8936: webbrowser regression on windows
http://bugs.python.org/issue8936  closed by terry.reedy

#10886: Unhelpful backtrace for multiprocessing.Queue
http://bugs.python.org/issue10886  closed by neologix

#12749: lib re cannot match non-BMP ranges (all versions, all builds)
http://bugs.python.org/issue12749  closed by ezio.melotti

#13684: httplib tunnel infinite loop
http://bugs.python.org/issue13684  closed by pitrou

#14720: sqlite3 microseconds
http://bugs.python.org/issue14720  closed by petri.lehtinen

#15132: Let unittest.TestProgram()'s defaultTest argument be a list
http://bugs.python.org/issue15132  closed by petri.lehtinen

#16403: update distutils docs to say that maintainer replaces author
http://bugs.python.org/issue16403  closed by petri.lehtinen

#16406: move the "Uploading Packages" section to distutils/packageinde
http://bugs.python.org/issue16406  closed by chris.jerdonek

#16695: Clarify fnmatch & glob docs about the handling of leading "."s
http://bugs.python.org/issue16695  closed by petri.lehtinen

#16935: unittest should understand SkipTest at import time during test
http://bugs.python.org/issue16935  closed by ezio.melotti

#17079: Fix test discovery for test_ctypes.py
http://bugs.python.org/issue17079  closed by ezio.melotti

#17082: Fix test discovery for test_dbm*.py
http://bugs.python.org/issue17082  closed by ezio.melotti

#17130: Add runcall() function to profile.py and cProfile.py
http://bugs.python.org/issue17130  closed by eric.araujo

#17137: Malfunctioning compiled code in Python 3.3 x64
http://bugs.python.org/issue17137  closed by haypo

#17197: c/profile refactoring
http://bugs.python.org/issue17197  closed by giampaolo.rodola

#17217: Fix test discovery for test_format.py on Windows
http://bugs.python.org/issue17217  closed by ezio.melotti

#17220: Little enhancements of _bootstrap.py
http://bugs.python.org/issue17220  closed by serhiy.storchaka

#17249: reap threads in test_capi
http://bugs.python.org/issue17249  closed by ezio.melotti

#17275: io.BufferedWriter shows wrong type in argument error message
http://bugs.python.org/issue17275  closed by r.david.murray

#17280: path.basename and ntpath.basename functions returns an incorre
http://bugs.python.org/issue17280  closed by eric.smith

#17283: Lib/test/__main__.py should share code with regrtest.py
http://bugs.python.org/issue17283  closed by chris.jerdonek

#17287: slice inconsistency
http://bugs.python.org/issue17287  closed by r.david.murray

#17290: pythonw - loading cursor bug when launching scripts
http://bugs.python.org/issue17290  closed by python-dev

#17291: Login-data raising EOFError
http://bugs.python.org/issue17291  closed by gregory.p.smith

#17292: Autonumbering in string.Formatter doesn't work
http://bugs.python.org/issue17292  closed by eric.smith

#17297: Issue with return in recursive functions
http://bugs.python.org/issue17297  closed by r.david.murray

#17300: ??rash when deleting deeply recursive iterator wrappers
http://bugs.python.org/issue17300  closed by serhiy.storchaka

#17303: Fix test discovery for test_future*
http://bugs.python.org/issue17303  closed by ezio.melotti

#17304: Fix test discovery for test_hash.py
http://bugs.python.org/issue17304  closed by ezio.melotti

#17313: test_logging doesn't clean up after itself
http://bugs.python.org/issue17313  closed by python-dev

#17321: Better way to pass objects between imp.find_module() and imp.l
http://bugs.python.org/issue17321  closed by brett.cannon

#675976: mhlib does not obey MHCONTEXT env var
http://bugs.python.org/issue675976  closed by r.david.murray

#1709112: test_1686475 of test_os & pagefile.sys
http://bugs.python.org/issue1709112  closed by ezio.melotti

#694374: Recursive regular expressions
http://bugs.python.org/issue694374  closed by r.david.murray

#846817: control-c is being sent to child thread rather than main
http://bugs.python.org/issue846817  closed by r.david.murray

#783528: Inconsistent results with super and __getattribute__
http://bugs.python.org/issue783528  closed by r.david.murray

From fwierzbicki at gmail.com  Fri Mar  1 18:49:28 2013
From: fwierzbicki at gmail.com (fwierzbicki at gmail.com)
Date: Fri, 1 Mar 2013 09:49:28 -0800
Subject: [Python-Dev] Merging Jython code into standard Lib [was Re:
 Python Language Summit at PyCon: Agenda]
In-Reply-To: <CAP1=2W6t0pYnC4OxX9AYzsXCCnPEogqsbAwhe_jGDdSPNLesug@mail.gmail.com>
References: <CAOTb1wcey0+2=py2sFbvk+fLN57ppQpG0qB6Pw9OgoXBOpBPBw@mail.gmail.com>
	<CADrh4zL_SM94izq057TFzvBmJAJb9cUJLYdpFep+rRDprYfFFw@mail.gmail.com>
	<20130228210023.089665ef@pitrou.net>
	<CADrh4zJh+1Ntz3W9Z+nZKNa6KKc4Gdpa0_ovA5xjEiBKiAcxQQ@mail.gmail.com>
	<CAP1=2W6t0pYnC4OxX9AYzsXCCnPEogqsbAwhe_jGDdSPNLesug@mail.gmail.com>
Message-ID: <CADrh4zKabgFfX1K3Kqkg6vuTejGaaGyiW2zJKzkyKBMfpjKy_w@mail.gmail.com>

On Thu, Feb 28, 2013 at 12:35 PM, Brett Cannon <brett at python.org> wrote:
>
> On Thu, Feb 28, 2013 at 3:17 PM, fwierzbicki at gmail.com
> <fwierzbicki at gmail.com> wrote:
>>
>> It would be nice in this particular case if there was a zlib.py that
>> imported _zlib -- then it would be easy to shim in Jython's version,
>> whether it is written in a .py file or in Java.
>
>
> That should be fine as that is what we already do for accelerator modules
> anyway. If you want to work towards having an equivalent of CPython's
> Modules/ directory so you can ditch your custom Lib/ modules by treating
> your specific code as accelerators I think we can move towards that
> solution.
Sounds great! I'm betting that implementing PEP 420 on Jython will
make mixed Python/Java code easier to deal with, so _zlib.py might
just end up living next to our Java code. So deleting Jython's Lib/
may still be an option.

-Frank

From solipsis at pitrou.net  Fri Mar  1 19:38:03 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 1 Mar 2013 19:38:03 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
Message-ID: <20130301193803.4156607c@pitrou.net>

On Fri, 1 Mar 2013 09:32:23 -0500
Barry Warsaw <barry at python.org> wrote:
> 
> >On the other hand in some ways Jython is sort of like Python on a
> >weird virtual OS that lets the real OS bleed through some. This may
> >still need to be checked in that way (there's are still checks of <if
> >os.name == 'nt'> right?)
> 
> Yeah, but that all ooooold code ;)

Hmm, what do you mean? `os.name == 'nt'` is still the proper way to
test that we're running on a Windows system (more accurately, over the
Windows API).

Regards

Antoine.



From brett at python.org  Sat Mar  2 03:31:02 2013
From: brett at python.org (Brett Cannon)
Date: Fri, 1 Mar 2013 21:31:02 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
Message-ID: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>

As of right now, importlib keeps a cache of what is in a directory for its
file finder instances. It uses mtime on the directory to try and detect
when it has changed to know when to refresh the cache. But thanks to mtime
granularities of up to a second, it is only a heuristic that isn't totally
reliable, especially across filesystems on different OSs.

This is why importlib.invalidate_caches() came into being. If you look in
our test suite you will see it peppered around where a module is created on
the fly to make sure that mtime granularity isn't a problem. But it
somewhat negates the point of the mtime heuristic when you have to make
this function call regardless to avoid potential race conditions.

http://bugs.python.org/issue17330 originally suggested trying to add
another heuristic to determine when to invalidate the cache. But even with
the suggestion it's still iffy and in no way foolproof.

So the current idea is to just drop the invalidation heuristic and go
full-blown reliance on calls to importlib.invalidate_caches() as necessary.
This makes code more filesystem-agnostic and protects people from
hard-to-detect errors when importlib only occasionally doesn't detect new
modules (I know it drove me nuts for a while when the buildbots kept
failing sporadically and only on certain OSs).

I would have just made the change but Antoine wanted it brought up here
first to make sure that no one was heavily relying on the current setup. So
if you have a good, legitimate reason to keep the reliance on mtime for
cache invalidation please speak up. But since the common case will never
care about any of this (how many people generate modules on the fly to
being with?) and to be totally portable you need to call
importlib.invalidate_caches() anyway, it's going to take a lot to convince
me to keep it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130301/de64d39a/attachment.html>

From greg at krypto.org  Sat Mar  2 08:40:37 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Fri, 1 Mar 2013 23:40:37 -0800
Subject: [Python-Dev] cffi in stdlib
In-Reply-To: <CAF-Rda-=yAcKGaQrefZX8YgK6hg=PQHR2TYi4xghg2W3tAfJBw@mail.gmail.com>
References: <CAK5idxQA+9zbBGhG0Thxzg9aBEko9UBtFTGK9_uJJ1vkyrGXwg@mail.gmail.com>
	<CAF-Rda8krMYuaV-5VDezW5ZiL_O-jEOB1Db9txAQF1XSpDNzpA@mail.gmail.com>
	<CAK5idxRefe+0S45d6es6uQj8JX_5z0YGdrFWxXQT7hqoE7YthQ@mail.gmail.com>
	<CAF-Rda-=yAcKGaQrefZX8YgK6hg=PQHR2TYi4xghg2W3tAfJBw@mail.gmail.com>
Message-ID: <CAGE7PNJuV5eeLfdnxpdBj_g=U_OP3jRf9ipft6qT_dK3SSztWA@mail.gmail.com>

On Wed, Feb 27, 2013 at 7:57 AM, Eli Bendersky <eliben at gmail.com> wrote:

>
>  > I read the cffi docs once again and went through some of the examples. I
>> > want to divide this to two topics.
>> >
>> > One is what you call the "ABI" level. IMHO, it's hands down superior to
>> > ctypes. Your readdir demo demonstrates this very nicely. I would
>> definitely
>> > want to see this in the stdlib as an alternative way to interface to C
>> > shared objects & DLLs.
>> >
>> > Two is what you call the "API" level, which is where my opinion becomes
>> > mixed. Some things just don't feel right to me:
>> >
>> > 1. Tying in a C compiler into the flow of a program. I'm not sure
>> whether we
>> > have precedents for it in the stdlib. Does this work on Windows where
>> > libraries and DLLs are usually built with MSVC?
>> >
>>
>> Yes. Precedent in the stdlib is really the C API. All the same rules
>> apply (including build and ship a dll).
>>
>
> So would you say that the main use of the API level is provide an
> alternative for writing C API code to interface to C libraries. IOW, it's
> in competition with Swig?
>

I'd hardly call it competition. The primary language I interface with is
C++ and cffi appears not see that giant elephant in the room (it'd need to
use clang for parsing if it were going to do that)...

The goal is good, but being C only isn't super exciting to me.

Would there be a goal of using cffi to replace C extension module code in
the standard library with cffi based versions instead of hand written
CPython C API code?
If not, why not?  (and what does that say about its limitations or
practical it is?)

-0.5 from me. My inclination is not to add this to the standard library.
But even if it were to be added, it sounds like others are coming up with
questions and reasons why it isn't yet ready (always the first important
step to seriously considering inclusion).

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130301/c87a7f35/attachment.html>

From greg at krypto.org  Sat Mar  2 09:01:25 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Sat, 2 Mar 2013 00:01:25 -0800
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADiSq7f7APRJJ+Wjgq66BC2rSmyhHLn3+tdEEPbOv53znxqeBQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADiSq7f7APRJJ+Wjgq66BC2rSmyhHLn3+tdEEPbOv53znxqeBQ@mail.gmail.com>
Message-ID: <CAGE7PNJyo8d4SRODydmpGnnFP6+Ot-13WpXz3UUMiR3U4T06cQ@mail.gmail.com>

On Thu, Feb 28, 2013 at 1:15 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Thu, Feb 28, 2013 at 1:37 PM, Barry Warsaw <barry at python.org> wrote:
> > On Feb 27, 2013, at 11:33 AM, fwierzbicki at gmail.com wrote:
> >>The easy part for Jython is pushing some of our "if is_jython:" stuff
> >>into the appropriate spots in CPython's Lib/.
> >
> > I wonder if there isn't a better way to do this than sprinkling
> is_jython,
> > is_pypy, is_ironpython, is_thenextbigthing all over the code base.  I
> have no
> > bright ideas here, but it seems like a feature matrix would be a better
> way to
> > go than something that assumes a particular Python implementation has a
> > particular feature set (which may change in the future).
>
> Yes, avoiding that kind of thing is a key motivation for
> sys.implementation. Any proposal for "is_jython" blocks should instead
> be reformulated as a proposal for new sys.implementation attributes.
>

I kind of wish there were an assert-like magic "if __debug__:" type of
mechanism behind this so that blocks of code destined solely for a single
interpreter won't be seen in the code objects or .pyc's of non-target
interpreters.

That idea obviously isn't fleshed out but i figure i'd better plant the
seed...

It'd mean smaller code objects and less bloat from constants (docstrings
for one implementation vs another, etc) being in memory. Taken further,
this could even be extended beyond implementations to platforms as we have
some standard library code with alternate definitions within one file for
windows vs posix, etc.

Antoine's point about code like that being untestable by most CPython
developers is valid.  I'd want --with-pydebug builds to disable any parsing
-> code object exclusions to at least make sure its syntax doesn't rot but
that still doesn't _test_ it unless we get someone maintains reliable
buildbots for every implementation using this common stdlib.

-gps
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/e3a75e67/attachment.html>

From arigo at tunes.org  Sat Mar  2 09:48:42 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 2 Mar 2013 09:48:42 +0100
Subject: [Python-Dev] cffi in stdlib
In-Reply-To: <CAGE7PNJuV5eeLfdnxpdBj_g=U_OP3jRf9ipft6qT_dK3SSztWA@mail.gmail.com>
References: <CAK5idxQA+9zbBGhG0Thxzg9aBEko9UBtFTGK9_uJJ1vkyrGXwg@mail.gmail.com>
	<CAF-Rda8krMYuaV-5VDezW5ZiL_O-jEOB1Db9txAQF1XSpDNzpA@mail.gmail.com>
	<CAK5idxRefe+0S45d6es6uQj8JX_5z0YGdrFWxXQT7hqoE7YthQ@mail.gmail.com>
	<CAF-Rda-=yAcKGaQrefZX8YgK6hg=PQHR2TYi4xghg2W3tAfJBw@mail.gmail.com>
	<CAGE7PNJuV5eeLfdnxpdBj_g=U_OP3jRf9ipft6qT_dK3SSztWA@mail.gmail.com>
Message-ID: <CAMSv6X1ery68fwC-HXpbeoa3QS7LK42x6CKcyXhT-TDpaRg+nQ@mail.gmail.com>

Hi Gregory,

On Sat, Mar 2, 2013 at 8:40 AM, Gregory P. Smith <greg at krypto.org> wrote:
>
> On Wed, Feb 27, 2013 at 7:57 AM, Eli Bendersky <eliben at gmail.com> wrote:
>> So would you say that the main use of the API level is provide an
>> alternative for writing C API code to interface to C libraries. IOW, it's in
>> competition with Swig?
>
> I'd hardly call it competition. The primary language I interface with is C++
> and cffi appears not see that giant elephant in the room

I don't think it's in competition with Swig, which does C++.  There
are certain workloads in which C++ is the elephant in the room; we
don't address such workloads.  If you want some more motivation, the
initial goal was to access the large number of standard Linux/Posix
libraries that are C (or have a C interface), but are too hard to
access for ctypes (macros, partially-documented structure types,
#define for constants, etc.).  For this goal, it works great.

> (it'd need to use clang for parsing if it were going to do that)...

I fear parsing is merely the tip of the iceberg when we talk about
interfacing with C++.


A bient?t,

Armin.

From stefan_ml at behnel.de  Sat Mar  2 10:10:24 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 02 Mar 2013 10:10:24 +0100
Subject: [Python-Dev] cffi in stdlib
In-Reply-To: <CAK5idxQA+9zbBGhG0Thxzg9aBEko9UBtFTGK9_uJJ1vkyrGXwg@mail.gmail.com>
References: <CAK5idxQA+9zbBGhG0Thxzg9aBEko9UBtFTGK9_uJJ1vkyrGXwg@mail.gmail.com>
Message-ID: <kgsfls$l1v$1@ger.gmane.org>

Hi,

looks like no-one's taken over the role of the Advocatus Diaboli yet. =)

Maciej Fijalkowski, 26.02.2013 16:13:
> I would like to discuss on the language summit a potential inclusion
> of cffi[1] into stdlib. This is a project Armin Rigo has been working
> for a while, with some input from other developers. It seems that the
> main reason why people would prefer ctypes over cffi these days is
> "because it's included in stdlib", which is not generally the reason I
> would like to hear. Our calls to not use C extensions and to use an
> FFI instead has seen very limited success with ctypes and quite a lot
> more since cffi got released. The API is fairly stable right now with
> minor changes going in and it'll definitely stablize until Python 3.4
> release.

You say that "the API is fairly stable". What about the implementation?
Will users want to install a new version next to the stdlib one in a couple
of months, just because there was a bug in the parser in Python 3.4 that
you still need to support because there's code that depends on it, or
because there is this new feature that is required to make it work with
library X, or ... ? What's the upgrade path in that case? How will you
support this? What long-term guarantees do you give to users of the stdlib
package?

Or, in other words, will the normal fallback import for cffi look like this:

    try: import stdlib_cffi
    except ImportError: import external_cffi

or will the majority of users end up prefering this order:

    try: import external_cffi
    except ImportError: import stdlib_cffi


> * Work either at the level of the ABI (Application Binary Interface)
> or the API (Application Programming Interface). Usually, C libraries
> have a specified C API but often not an ABI (e.g. they may document a
> ?struct? as having at least these fields, but maybe more). (ctypes
> works at the ABI level, whereas Cython and native C extensions work at
> the API level.)

Ok, so there are cases where you need a C compiler installed in order to
support the API. Which means that it will be a very complicated thing for
users to get working under Windows, for example, which then means that
users are actually best off not using the API-support feature if they want
portable code. Wouldn't it be simpler to target Windows with a binary than
with dynamically compiled C code? Is there a way to translate an API
description into a static ABI description for a known platform ahead of
time, or do I have to implement this myself in a separate ABI code path by
figuring out a suitable ABI description myself?

In which cases would users choose to use the C API support? And, is this
dependency explicit or can I accidentally run into the dependency on a C
compiler for my code without noticing?


> * We try to be complete. For now some C99 constructs are not
> supported, but all C89 should be, including macros (and including
> macro ?abuses?, which you can manually wrap in saner-looking C
> functions).

Ok, so the current status actually is that it's *not* complete, and that
future versions will have to catch up in terms of C compatibility. So, why
do you think it's a good time to get it into the stdlib *now*?


> * We attempt to support both PyPy and CPython, with a reasonable path
> for other Python implementations like IronPython and Jython.

You mentioned that it's fast under PyPy and slow under CPython, though.
What would be the reason to use it under CPython then? Some of the projects
that are using it (you named a couple) also have equivalent (or maybe more
or less so) native implementations for CPython already. Do you have any
benchmarks available that compare those to their cffi versions under
CPython? Is the slowdown within any reasonable bounds?

Others have already mentioned the lack of C++ support. It's ok to say that
you deliberately only want to support C, but it's also true that that's a
substantial restriction.

Stefan



From arigo at tunes.org  Sat Mar  2 12:13:50 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 2 Mar 2013 12:13:50 +0100
Subject: [Python-Dev] cffi in stdlib
In-Reply-To: <kgsfls$l1v$1@ger.gmane.org>
References: <CAK5idxQA+9zbBGhG0Thxzg9aBEko9UBtFTGK9_uJJ1vkyrGXwg@mail.gmail.com>
	<kgsfls$l1v$1@ger.gmane.org>
Message-ID: <CAMSv6X33nY25JNONt-ZMtVVFo4FwUQObhUvW0mZ9a0qbvJvEkQ@mail.gmail.com>

Hi Stefan,

On Sat, Mar 2, 2013 at 10:10 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> You say that "the API is fairly stable". What about the implementation?
> Will users want to install a new version next to the stdlib one in a couple
> of months,

I think that the implementation is fairly stable as well.  The only
place I can foresee some potential changes is in details like the
location of temporary files, for example, which needs to be discussed
(probably with people from python-dev too) as some point.

> just because there was a bug in the parser in Python 3.4 that
> you still need to support because there's code that depends on it, or
> because there is this new feature that is required to make it work with
> library X, or ... ? What's the upgrade path in that case? How will you
> support this? What long-term guarantees do you give to users of the stdlib
> package?

I think these are general questions for any package that ends up in
the stdlib.  In the case of CFFI, it is now approaching a stability
point.  This is also because we are going to integrate it with the
stdlib of PyPy soon.

Bugs in the parser have not been found so far, but if there is any, we
will treat it like we treat any other bug in the stdlib.  For that
matter, there is actually no obvious solution for the user either: he
generally has to wait for the next micro release to have the bug
fixed.

> Or, in other words, will the normal fallback import for cffi look like this:
>
>     try: import stdlib_cffi
>     except ImportError: import external_cffi
>
> or will the majority of users end up prefering this order:
>
>     try: import external_cffi
>     except ImportError: import stdlib_cffi

I would rather drop the external CFFI entirely, or keep it only to
provide backports to older Python versions.  I personally see no
objection to call the stdlib one "cffi" too (but any other name is
fine as well).

> ... Wouldn't it be simpler to target Windows with a binary than
> with dynamically compiled C code? Is there a way to translate an API
> description into a static ABI description for a known platform ahead of
> time, or do I have to implement this myself in a separate ABI code path by
> figuring out a suitable ABI description myself?

No, I believe that you missed this point: when you make "binary"
distributions of a package with setup.py, it precompiles a library for
CFFI too.  So yes, you need a C compiler on machines where you develop
the program, but not on machines where you install it.  It's the same
needs as when writing custom C extension modules by hand.

> In which cases would users choose to use the C API support? And, is this
> dependency explicit or can I accidentally run into the dependency on a C
> compiler for my code without noticing?

A C compiler is clearly required: this is if and only if you call the
function verify(), and pass it arguments that are not the same ones as
the previous time (i.e. it's not in the cache).

>> * We try to be complete. For now some C99 constructs are not
>> supported, but all C89 should be, including macros (and including
>> macro ?abuses?, which you can manually wrap in saner-looking C
>> functions).
>
> Ok, so the current status actually is that it's *not* complete, and that
> future versions will have to catch up in terms of C compatibility. So, why
> do you think it's a good time to get it into the stdlib *now*?

To be honest I don't have a precise list of C99 constructs missing.  I
used to know of a few of them, but these were eventually supported.
It is unlikely to get completed, or if it is, a fairly slow process
should be fine --- just like a substantial portion of the stdlib,
which gets occasional updates from one Python version to the next.

> You mentioned that it's fast under PyPy and slow under CPython, though.
> What would be the reason to use it under CPython then?

The reason is just ease of use.  I pretend that it takes less effort
(and little C knowledge), and is less prone to bugs and leaks, to
write a perfectly working prototype of a module to access a random C
library.  I do not pretend that you'll get the top-most performance.
For a lot of cases performance doesn't matter; and when it does, on
CPython, you can really write a C extension module by hand (as long as
you make sure to keep around the CFFI version for use by PyPy).  This
is how I see it, anyway.  The fact that we are busy rewriting existing
native well-tested CPython extensions with CFFI --- this is really
only of use for PyPy.

> Others have already mentioned the lack of C++ support. It's ok to say that
> you deliberately only want to support C, but it's also true that that's a
> substantial restriction.

I agree that it's a restriction, or rather a possible extension that
is not done.  I don't have plans to do it myself.  Please also keep in
mind that we pitch CFFI as a better ctypes, not as the ultimate tool
to access any foreign language.


A bient?t,

Armin.

From mkawalec at lavabit.com  Sat Mar  2 12:48:05 2013
From: mkawalec at lavabit.com (Michal Kawalec)
Date: Sat, 02 Mar 2013 12:48:05 +0100
Subject: [Python-Dev] Possible bug in socket.py: connection reset by peer
Message-ID: <5131E6F5.4090709@lavabit.com>

Hello,
I am experiencing an odd infrequent bug in Python 2.7.3 with GIL
enabled. For some files pushed over TCP socket I get 'connection reset
by peer' and clients only receive a randomly long part of the file.

This situation occurs only in ~0.1% of cases but if it happens for a
given file it keeps on always occuring for that file. The server host is
2-cored Linux 3.8.1 on a VMware VM.

The problem is mitigated by adding time.sleep(0.001) just before the
portion of data is being pushed though the socket. It also seems to be
known for a long time [1].

So my question is - is it something that I can expect to be fixed in the
future Python releases?


Michal


[1]
http://stackoverflow.com/questions/441374/why-am-i-seeing-connection-reset-by-peer-error

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 555 bytes
Desc: OpenPGP digital signature
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/78d9f80d/attachment.pgp>

From solipsis at pitrou.net  Sat Mar  2 13:25:19 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 2 Mar 2013 13:25:19 +0100
Subject: [Python-Dev] Possible bug in socket.py: connection reset by peer
References: <5131E6F5.4090709@lavabit.com>
Message-ID: <20130302132519.6e142983@pitrou.net>

On Sat, 02 Mar 2013 12:48:05 +0100
Michal Kawalec <mkawalec at lavabit.com> wrote:
> I am experiencing an odd infrequent bug in Python 2.7.3 with GIL
> enabled. For some files pushed over TCP socket I get 'connection reset
> by peer' and clients only receive a randomly long part of the file.

Why do you think it is a bug in Python?
Start by doing a Wireshark capture of your traffic and find out what
really happens.

Regards

Antoine.



From ncoghlan at gmail.com  Sat Mar  2 16:08:40 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 01:08:40 +1000
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
Message-ID: <CADiSq7fvccRpSWGPWwcY-5GFYzwt9FvO8fGXOK6zdDkfZQMA7Q@mail.gmail.com>

On Sat, Mar 2, 2013 at 1:24 AM, Stefan Bucur <stefan.bucur at gmail.com> wrote:
> Hi,
>
> I'm working on an automated bug finding tool that I'm trying to apply on the
> Python interpreter code (version 2.7.3). Because of early prototype
> limitations, I needed to disable string interning in stringobject.c. More
> precisely, I modified the PyString_FromStringAndSize and PyString_FromString
> to no longer check for the null and single-char cases, and create instead a
> new string every time (I can send the patch if needed).
>
> However, after applying this modification, when running "make test" I get a
> segfault in the test___all__ test case.
>
> Before digging deeper into the issue, I wanted to ask here if there are any
> implicit assumptions about string identity and interning throughout the
> interpreter implementation. For instance, are two single-char strings having
> the same content supposed to be identical objects?
>
> I'm assuming that it's either this, or some refcount bug in the interpreter
> that manifests only when certain strings are no longer interned and thus
> have a higher chance to get low refcount values.

In theory, interning is supposed to be a pure optimisation, but it
wouldn't surprise me if there are cases that assume the described
strings are always interned (especially the null string case). Our
test suite would never detect such bugs, as we never disable the
interning.

Whether or not we're interested in fixing such bugs would depend on
the size of the patches needed to address them. From our point of
view, such bugs are purely theoretical (as the assumption is always
valid in an unpatched CPython build), so if the problem is too hard to
diagnose or fix, we're more likely to declare that interning of at
least those kinds of string values is required for correctness when
creating modified versions of CPython.

I'm not sure what kind of analyser you are writing, but if it relates
to the CPython C API, you may be interested in
https://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Mar  2 16:17:35 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 01:17:35 +1000
Subject: [Python-Dev] Merging Jython code into standard Lib [was Re:
 Python Language Summit at PyCon: Agenda]
In-Reply-To: <CAP1=2W6t0pYnC4OxX9AYzsXCCnPEogqsbAwhe_jGDdSPNLesug@mail.gmail.com>
References: <CAOTb1wcey0+2=py2sFbvk+fLN57ppQpG0qB6Pw9OgoXBOpBPBw@mail.gmail.com>
	<CADrh4zL_SM94izq057TFzvBmJAJb9cUJLYdpFep+rRDprYfFFw@mail.gmail.com>
	<20130228210023.089665ef@pitrou.net>
	<CADrh4zJh+1Ntz3W9Z+nZKNa6KKc4Gdpa0_ovA5xjEiBKiAcxQQ@mail.gmail.com>
	<CAP1=2W6t0pYnC4OxX9AYzsXCCnPEogqsbAwhe_jGDdSPNLesug@mail.gmail.com>
Message-ID: <CADiSq7d8bqkonQSwYkyS08PfQxvNu6iX4coDmLvJJsN06haYLQ@mail.gmail.com>

On Fri, Mar 1, 2013 at 6:35 AM, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Thu, Feb 28, 2013 at 3:17 PM, fwierzbicki at gmail.com
> <fwierzbicki at gmail.com> wrote:
>>
>> On Thu, Feb 28, 2013 at 12:00 PM, Antoine Pitrou <solipsis at pitrou.net>
>> wrote:
>> > IMHO, we should remove the plat-* directories, they are completely
>> > unmaintained, undocumented, and serve no useful purpose.
>> Oh I didn't know that - so definitely adding to that is right out :)
>>
>> Really for cases like Jython's zlib.py (no useful code for CPython) I
>> don't have any trouble keeping them entirely in Jython. It just would
>> have been fun to delete our Lib/ :)
>>
>> It would be nice in this particular case if there was a zlib.py that
>> imported _zlib -- then it would be easy to shim in Jython's version,
>> whether it is written in a .py file or in Java.
>
>
> That should be fine as that is what we already do for accelerator modules
> anyway. If you want to work towards having an equivalent of CPython's
> Modules/ directory so you can ditch your custom Lib/ modules by treating
> your specific code as accelerators I think we can move towards that
> solution.

I'd go further and say we *should* move to that solution.

Here's an interesting thought: for pure C modules without a Python
implementation, we can migrate to this architecture even *without*
creating pure Python equivalents. All we shou;d have to do is change
the test of the pure Python version to be that the module *can't be
imported* without the accelerator, rather than the parallel tests that
we normally implement when there's a pure Python alternative to the
accelerated version. (There would likely still be some mucking about
to ensure robust pickle compatibility, since that leaks implementation
details about exact module names if you're not careful)

PyPy, Jython, IronPython would then have RPython, Java, C# versions,
while CPython has a C version, and the test suite should work
regardless. (If PyPy have equivalents in Python, they can either push
them upstream, overwrite the "import the accelerator" version).

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Sat Mar  2 16:28:57 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 2 Mar 2013 16:28:57 +0100
Subject: [Python-Dev] Merging Jython code into standard Lib [was Re:
 Python Language Summit at PyCon: Agenda]
In-Reply-To: <CADiSq7d8bqkonQSwYkyS08PfQxvNu6iX4coDmLvJJsN06haYLQ@mail.gmail.com>
References: <CAOTb1wcey0+2=py2sFbvk+fLN57ppQpG0qB6Pw9OgoXBOpBPBw@mail.gmail.com>
	<CADrh4zL_SM94izq057TFzvBmJAJb9cUJLYdpFep+rRDprYfFFw@mail.gmail.com>
	<20130228210023.089665ef@pitrou.net>
	<CADrh4zJh+1Ntz3W9Z+nZKNa6KKc4Gdpa0_ovA5xjEiBKiAcxQQ@mail.gmail.com>
	<CAP1=2W6t0pYnC4OxX9AYzsXCCnPEogqsbAwhe_jGDdSPNLesug@mail.gmail.com>
	<CADiSq7d8bqkonQSwYkyS08PfQxvNu6iX4coDmLvJJsN06haYLQ@mail.gmail.com>
Message-ID: <20130302162857.30a847e3@pitrou.net>

On Sun, 3 Mar 2013 01:17:35 +1000
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> I'd go further and say we *should* move to that solution.
> 
> Here's an interesting thought: for pure C modules without a Python
> implementation, we can migrate to this architecture even *without*
> creating pure Python equivalents. All we shou;d have to do is change
> the test of the pure Python version to be that the module *can't be
> imported* without the accelerator, rather than the parallel tests that
> we normally implement when there's a pure Python alternative to the
> accelerated version. (There would likely still be some mucking about
> to ensure robust pickle compatibility, since that leaks implementation
> details about exact module names if you're not careful)

What benefit would this have?

Current situation: each Python implementation has its own
implementation of the zlib module (as a C module for CPython, etc.).

New situation: all Python implementations share a single, mostly empty,
zlib.py file. Each Python implementation has its own implementation of
the _zlib module (as a C module for CPython, etc.) which is basically
the same as the former zlib module.

Regards

Antoine.

From solipsis at pitrou.net  Sat Mar  2 16:31:36 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 2 Mar 2013 16:31:36 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
Message-ID: <20130302163136.18b78885@pitrou.net>

On Fri, 1 Mar 2013 16:24:42 +0100
Stefan Bucur <stefan.bucur at gmail.com> wrote:
> 
> However, after applying this modification, when running "make test" I get a
> segfault in the test___all__ test case.
> 
> Before digging deeper into the issue, I wanted to ask here if there are any
> implicit assumptions about string identity and interning throughout the
> interpreter implementation. For instance, are two single-char strings
> having the same content supposed to be identical objects?

From a language POV, no, but inside a specific interpreter such as
CPython it may be a reasonable expectation.

> I'm assuming that it's either this, or some refcount bug in the interpreter
> that manifests only when certain strings are no longer interned and thus
> have a higher chance to get low refcount values.

Indeed, if it's a real bug it would be nice to get it fixed :-)

Regards

Antoine.



From ncoghlan at gmail.com  Sat Mar  2 16:36:04 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 01:36:04 +1000
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
Message-ID: <CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>

On Sat, Mar 2, 2013 at 12:31 PM, Brett Cannon <brett at python.org> wrote:
> As of right now, importlib keeps a cache of what is in a directory for its
> file finder instances. It uses mtime on the directory to try and detect when
> it has changed to know when to refresh the cache. But thanks to mtime
> granularities of up to a second, it is only a heuristic that isn't totally
> reliable, especially across filesystems on different OSs.
>
> This is why importlib.invalidate_caches() came into being. If you look in
> our test suite you will see it peppered around where a module is created on
> the fly to make sure that mtime granularity isn't a problem. But it somewhat
> negates the point of the mtime heuristic when you have to make this function
> call regardless to avoid potential race conditions.
>
> http://bugs.python.org/issue17330 originally suggested trying to add another
> heuristic to determine when to invalidate the cache. But even with the
> suggestion it's still iffy and in no way foolproof.
>
> So the current idea is to just drop the invalidation heuristic and go
> full-blown reliance on calls to importlib.invalidate_caches() as necessary.
> This makes code more filesystem-agnostic and protects people from
> hard-to-detect errors when importlib only occasionally doesn't detect new
> modules (I know it drove me nuts for a while when the buildbots kept failing
> sporadically and only on certain OSs).
>
> I would have just made the change but Antoine wanted it brought up here
> first to make sure that no one was heavily relying on the current setup. So
> if you have a good, legitimate reason to keep the reliance on mtime for
> cache invalidation please speak up. But since the common case will never
> care about any of this (how many people generate modules on the fly to being
> with?) and to be totally portable you need to call
> importlib.invalidate_caches() anyway, it's going to take a lot to convince
> me to keep it.

I think you should keep it. A long running service that periodically
scans the importers for plugins doesn't care if modules take a few
extra seconds to show up, it just wants to see them eventually.
Installers (or filesystem copy or move operations!) have no way to
inform arbitrary processes that new files have been added.

It's that case where the process that added the modules is separate
from the process scanning for them, and the communication is one way,
where the heuristic is important. Explicit invalidation only works
when they're the *same* process, or when they're closely coupled so
the adding process can tell the scanning process to invalidate the
caches (our test suite is mostly the former although there are a
couple of cases of the latter).

I have no problem with documenting invalidate_caches() as explicitly
required for correctness when writing new modules which are to be read
back by the same process, or when there is a feedback path between two
processes that may be confusing if the cache invalidation is delayed.
The implicit invalidation is only needed to pick up modules written by
*another* process.

In addition, it may be appropriate for importlib to offer a
"write_module" method that accepts (module name, target path,
contents). This would:

1. Allow in-process caches to be invalidated implicitly and
selectively when new modules are created
2. Allow importers to abstract write access in addition to read access
3. Allow the import system to complain at time of writing if the
desired module name and target path don't actually match given the
current import system state.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Mar  2 16:37:56 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 01:37:56 +1000
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
Message-ID: <CADiSq7d+iW3kFHaBSAcjRsf1O5iyfR=q_9_zfy5soLDAKZ1bDA@mail.gmail.com>

On Sun, Mar 3, 2013 at 1:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> It's that case where the process that added the modules is separate
> from the process scanning for them, and the communication is one way,
> where the heuristic is important. Explicit invalidation only works
> when they're the *same* process, or when they're closely coupled so
> the adding process can tell the scanning process to invalidate the
> caches (our test suite is mostly the former although there are a
> couple of cases of the latter).

s/are/may be/ (I don't actually remember if there are or not off the
top of my head)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Sat Mar  2 16:58:47 2013
From: brett at python.org (Brett Cannon)
Date: Sat, 2 Mar 2013 10:58:47 -0500
Subject: [Python-Dev] Merging Jython code into standard Lib [was Re:
 Python Language Summit at PyCon: Agenda]
In-Reply-To: <20130302162857.30a847e3@pitrou.net>
References: <CAOTb1wcey0+2=py2sFbvk+fLN57ppQpG0qB6Pw9OgoXBOpBPBw@mail.gmail.com>
	<CADrh4zL_SM94izq057TFzvBmJAJb9cUJLYdpFep+rRDprYfFFw@mail.gmail.com>
	<20130228210023.089665ef@pitrou.net>
	<CADrh4zJh+1Ntz3W9Z+nZKNa6KKc4Gdpa0_ovA5xjEiBKiAcxQQ@mail.gmail.com>
	<CAP1=2W6t0pYnC4OxX9AYzsXCCnPEogqsbAwhe_jGDdSPNLesug@mail.gmail.com>
	<CADiSq7d8bqkonQSwYkyS08PfQxvNu6iX4coDmLvJJsN06haYLQ@mail.gmail.com>
	<20130302162857.30a847e3@pitrou.net>
Message-ID: <CAP1=2W6kEHXmEBu7xLK2Xtj0jO7LGT2wbuxCFYTqSEVZ1yMn9w@mail.gmail.com>

On Sat, Mar 2, 2013 at 10:28 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sun, 3 Mar 2013 01:17:35 +1000
> Nick Coghlan <ncoghlan at gmail.com> wrote:
> >
> > I'd go further and say we *should* move to that solution.
> >
> > Here's an interesting thought: for pure C modules without a Python
> > implementation, we can migrate to this architecture even *without*
> > creating pure Python equivalents. All we shou;d have to do is change
> > the test of the pure Python version to be that the module *can't be
> > imported* without the accelerator, rather than the parallel tests that
> > we normally implement when there's a pure Python alternative to the
> > accelerated version. (There would likely still be some mucking about
> > to ensure robust pickle compatibility, since that leaks implementation
> > details about exact module names if you're not careful)
>
> What benefit would this have?
>
> Current situation: each Python implementation has its own
> implementation of the zlib module (as a C module for CPython, etc.).
>
> New situation: all Python implementations share a single, mostly empty,
> zlib.py file. Each Python implementation has its own implementation of
> the _zlib module (as a C module for CPython, etc.) which is basically
> the same as the former zlib module.
>

Bare minimum? They all share the same module docstring. But it could be
extended to explicitly import only the public API into zlib.py, helping to
prevent leaking interpreter-specific APIs by accident (obviously would
still be available off of _zlib if people wanted them).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/f88314dc/attachment-0001.html>

From brett at python.org  Sat Mar  2 17:16:28 2013
From: brett at python.org (Brett Cannon)
Date: Sat, 2 Mar 2013 11:16:28 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
Message-ID: <CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>

On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sat, Mar 2, 2013 at 12:31 PM, Brett Cannon <brett at python.org> wrote:
> > As of right now, importlib keeps a cache of what is in a directory for
> its
> > file finder instances. It uses mtime on the directory to try and detect
> when
> > it has changed to know when to refresh the cache. But thanks to mtime
> > granularities of up to a second, it is only a heuristic that isn't
> totally
> > reliable, especially across filesystems on different OSs.
> >
> > This is why importlib.invalidate_caches() came into being. If you look in
> > our test suite you will see it peppered around where a module is created
> on
> > the fly to make sure that mtime granularity isn't a problem. But it
> somewhat
> > negates the point of the mtime heuristic when you have to make this
> function
> > call regardless to avoid potential race conditions.
> >
> > http://bugs.python.org/issue17330 originally suggested trying to add
> another
> > heuristic to determine when to invalidate the cache. But even with the
> > suggestion it's still iffy and in no way foolproof.
> >
> > So the current idea is to just drop the invalidation heuristic and go
> > full-blown reliance on calls to importlib.invalidate_caches() as
> necessary.
> > This makes code more filesystem-agnostic and protects people from
> > hard-to-detect errors when importlib only occasionally doesn't detect new
> > modules (I know it drove me nuts for a while when the buildbots kept
> failing
> > sporadically and only on certain OSs).
> >
> > I would have just made the change but Antoine wanted it brought up here
> > first to make sure that no one was heavily relying on the current setup.
> So
> > if you have a good, legitimate reason to keep the reliance on mtime for
> > cache invalidation please speak up. But since the common case will never
> > care about any of this (how many people generate modules on the fly to
> being
> > with?) and to be totally portable you need to call
> > importlib.invalidate_caches() anyway, it's going to take a lot to
> convince
> > me to keep it.
>
> I think you should keep it. A long running service that periodically
> scans the importers for plugins doesn't care if modules take a few
> extra seconds to show up, it just wants to see them eventually.
> Installers (or filesystem copy or move operations!) have no way to
> inform arbitrary processes that new files have been added.
>

But if they are doing the scan they can also easily invalidate the caches
before performing the scan.


>
> It's that case where the process that added the modules is separate
> from the process scanning for them, and the communication is one way,
> where the heuristic is important. Explicit invalidation only works
> when they're the *same* process, or when they're closely coupled so
> the adding process can tell the scanning process to invalidate the
> caches (our test suite is mostly the former although there are a
> couple of cases of the latter).
>

That's only true if the scanning process has no idea that another process
is adding modules. If there is an expectation then it doesn't matter who
added the file as you just assume cache invalidation is necessary.


>
> I have no problem with documenting invalidate_caches() as explicitly
> required for correctness when writing new modules which are to be read
> back by the same process, or when there is a feedback path between two
> processes that may be confusing if the cache invalidation is delayed.
>

Already documented as such.


> The implicit invalidation is only needed to pick up modules written by
> *another* process.
>
> In addition, it may be appropriate for importlib to offer a
> "write_module" method that accepts (module name, target path,
> contents). This would:
>
> 1. Allow in-process caches to be invalidated implicitly and
> selectively when new modules are created
>

I don't think that's necessary. If people don't want to blindly clear all
caches for a file they can write the file, search the keys in
sys.path_importer_cache for the longest prefix for the newly created file,
and then call the invalidate_cache() method on that explicit finder.

2. Allow importers to abstract write access in addition to read access
>

That's heading down the virtual filesystem path which I don't want to go
down any farther than I have to. The API is big enough as it is and the
more entangled it gets the harder it is to change/fix, especially with the
finders having a nice, small API compared to the loaders.


> 3. Allow the import system to complain at time of writing if the
> desired module name and target path don't actually match given the
> current import system state.
>

I think that's more checking than necessary for a use case that isn't that
common.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/53088d22/attachment.html>

From ncoghlan at gmail.com  Sat Mar  2 17:41:16 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 02:41:16 +1000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <31B54506-7B08-4E77-B966-A6171BBC3DA6@gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<31B54506-7B08-4E77-B966-A6171BBC3DA6@gmail.com>
Message-ID: <CADiSq7dHzP+-p2zbGxOScGJGXBhj=gByEZJcrO=O5QEoMR731g@mail.gmail.com>

On Fri, Mar 1, 2013 at 9:39 AM, Doug Hellmann <doug.hellmann at gmail.com> wrote:
>
> On Feb 27, 2013, at 11:51 AM, Michael Foord wrote:
>
>> Hello all,
>>
>> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
>>
>> The agenda of topics for discussion so far includes the following:
>>
>> * A report on pypy status - Maciej and Armin
>> * Jython and IronPython status reports - Dino / Frank
>> * Packaging (Doug Hellmann and Monty Taylor at least)
>
> Since the time I suggested we add packaging to the agenda, Nick has set up a separate summit meeting for Friday evening. I don't know if it makes sense to leave this on the agenda for Wednesday or not.
>
> Nick, what do you think?

I think it's definitely worth my taking some time to explain my goals
for the Friday night session, and some broader things in terms of
where I'd like to see packaging going, but a lot of the key packaging
people aren't involved in Python language development *per se*, and
hence won't be at the language summit.

There's also one controversial point that *does* need to be raised at
the summit: I would like to make distutils-sig the true authority for
packaging standards, so we can stop cross-posting PEPs intended to
apply to packaging in *current* versions of Python to python-dev. The
split discussions suck, and most of the people that need to be
convinced in order for packaging standards to be supported in current
versions of Python aren't on python-dev, since it's a tooling issue
rather than a language design issue. Standard lib support is necessary
in the long run to provide a good "batteries included" experience, but
it's *not* the way to create the batteries in the first place. Until
these standards have been endorsed by the authors of *existing*
packaging tools, proposing them for stdlib addition is premature, but
has been perceived as necessary in the past due to the confused power
structure.

This means that those core developers that want a say in the future
direction of packaging and distribution of Python software would need
to be actively involved in the ongoing discussions on distutils-sig,
rather than relying on being given an explicit invitation to weigh in
at the last minute through a thread (or threads) on python-dev. The
requirement that BDFL-delegates for packaging and distribution related
PEPs also be experienced core developers will remain, however, as
"suitable for future stdlib inclusion" is an important overarching
requirement for packaging and distribution standards. Such delegates
will just be expected to participate actively in distutils-sig *as
well as* python-dev.

Proposals for *actual* standard library updates (to bring it into line
with updated packaging standards) would still be subject to python-dev
discussion and authority (and would *not* have their Discussions-To
header set). Such discussions aren't particularly relevant to most of
the packaging tool developers, since the standard library version
isn't updated frequently enough to be useful to them, and also isn't
available on older Python releases, so python-dev is a more
appropriate venue from both perspectives.

At the moment, python-dev, catalog-sig and distutils-sig create an
awkward trinity where decision making authority and the appropriate
venues for discussion are grossly unclear. I consider this to be one
of the key reasons that working on packaging issues has quite a high
incidence of developer burnout - it's hard to figure out who needs to
be convinced of what, so it's easy for the frustration levels to reach
the "this just isn't worth the hassle" stage (especially when trying
to bring python-dev members up to speed on discussions that may have
taken months on distutils-sig, and when many of the details are
awkward compromises forced by the need to support *existing* tools and
development processes on older versions of Python). Under my proposal,
the breakdown would be slightly clearer:

distutils-sig: overall authority for packaging and distribution
related standards, *including* the interfaces between index servers
(such as PyPI) and automated tools. If a PEP has "Discussions-To" set
to distutils-sig, announcements of new PEPs, new versions of those
PEPs, *and* their acceptance or rejection should be announced there,
and *not* on python-dev. The "Resolution" header will thus point to a
distutils-sig post rather than a python-dev one. distutils-sig will
focus on solutions that work for *current* versions of Python, while
keeping in mind the need for future stdlib support.

python-dev: authority over stdlib support for packaging and
distribution standards, and the "batteries included" experience of
interacting with those standards. Until a next generation distribution
infrastructure is firmly established (which may involve years of
running the legacy infrastructure and the next generation metadata 2.x
based infrastructure in parallel), the stdlib will typically trail the
upstream standards substantially, since many upstream enhancements
will run afoul of the standard library's "no new features" rule,
preventing their inclusion in maintenance releases of old Python
versions.

catalog-sig: authority over the PyPI index server (supported by the
infrastructure SIG for actual operation of the service). Key people
are expected to also participate in distutils-sig, as that is where
the expected interface exposed to automated tools will be defined. How
PyPI exposes the packaging and distribution standards to *users*
through its web UI will be up to catalog-sig.

The vehicle for *documenting* such a change in policy would be an
update to PEP 1 to indicate that the "Discussions-To" header should
always point to the list where any formal acceptance of rejection of
the PEP would be announced. If preliminary discussions take place on a
feeder list like python-ideas, or import-sig, or somewhere else, that
will be declared as explicitly irrelevant from the PEP metadata point
of view: the Discussions-To field would be documented as referring to
the list that any Resolution field will eventually reference.

Regards,
NIck.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Mar  2 17:49:13 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 02:49:13 +1000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAGE7PNJyo8d4SRODydmpGnnFP6+Ot-13WpXz3UUMiR3U4T06cQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADiSq7f7APRJJ+Wjgq66BC2rSmyhHLn3+tdEEPbOv53znxqeBQ@mail.gmail.com>
	<CAGE7PNJyo8d4SRODydmpGnnFP6+Ot-13WpXz3UUMiR3U4T06cQ@mail.gmail.com>
Message-ID: <CADiSq7c_wSVjhr=5jF+aWBG9hURts8jbNmsHsW1kC3=XK4um-g@mail.gmail.com>

On Sat, Mar 2, 2013 at 6:01 PM, Gregory P. Smith <greg at krypto.org> wrote:
> It'd mean smaller code objects and less bloat from constants (docstrings for
> one implementation vs another, etc) being in memory. Taken further, this
> could even be extended beyond implementations to platforms as we have some
> standard library code with alternate definitions within one file for windows
> vs posix, etc.

To plant seeds in the opposite direction, as you're considering this,
I suggest looking at:
- environment markers in PEP 345 and 426 for conditional selection
based on a constrained set of platform data
- compatibility tags in PEP 425 (and consider how they could be used
in relation to __pycache__ and bytecode-only distribution of platform
specific files)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Mar  2 17:58:18 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 02:58:18 +1000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <kgpt39$nc5$1@ger.gmane.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgpt39$nc5$1@ger.gmane.org>
Message-ID: <CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>

On Fri, Mar 1, 2013 at 7:41 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Michael Foord, 27.02.2013 17:51:
> It's also true that many of the topics above aren't really interesting for
> us, because we just inherit them with CPython, e.g. stdlib changes.
> Packaging is only relevant as far as it impacts the distribution of binary
> extensions, and the main changes appear to be outside of that area (which
> doesn't mean it's not truly wonderful that they are happening, Python
> packaging has seen a lot of great improvements during the last years and
> I'm very happy to see it getting even better).

I'm puzzled by this one. Did you leave out PEP 427 (the wheel format),
because it's already approved, and hence not likely to be discussed
much at the summit, or because you don't consider it to impact the
distribution of binary extensions (which would be rather odd, given
the nature of the PEP and the wheel format...)

>
> Interpreter initialisation would be interesting and Cython could
> potentially help in some spots here by making code easier to maintain and
> optimise, for example. We've had this discussion for the importlib
> bootstrapping and I'm sure there's more that could be done. It's sad to see
> so much C-level work go into areas that really don't need to be that low-level.

Cython's notion of embedding is the exact opposite of CPython's, so
I'm not at all clear on how Cython could help with PEP 432 at all.

> I'm not so happy with the argument clinic, but that's certainly also
> because I'm biased. I've written the argument unpacking code for Cython
> some years ago, so it's not surprising that I'm quite happy with that and
> fail to see the need for a totally new DSL *and* a totally new
> implementation, especially with its mapping to the slowish ParseTuple*()
> C-API functions. I've also not seen a good argument why the existing Py3
> function signatures can't do what the proposed DSL tries to achieve. They'd
> at least make it clear that the intention is to make things more
> Python-like, and would at the same time provide the documentation.

That's why Stefan Krah is writing a competing PEP - a number of us
already agree with you, and think the case needs to be made for
choosing something completely different like Argument Clinic
(especially given Guido's expressed tolerance for the idea of "/" as a
possible marker to indicate that the preceding parameters only support
positional arguments - that was in the context of Python discussion
where it was eventually deemed "not necessary", but becomes
interesting again in a C API signature discussion)

> And I'd really like to see a CPython summit
> happen at some point. There's so much interesting stuff going on in that
> area that it's worth getting some people together to move these things forward.

Yes, a CPython runtime summit some year would be interesting.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From fijall at gmail.com  Sat Mar  2 18:18:46 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 2 Mar 2013 19:18:46 +0200
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgpt39$nc5$1@ger.gmane.org>
	<CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>
Message-ID: <CAK5idxQUAdoJuTzv4hz_747Qvgf_wmRNhvmQtWZw8kxYvDpE5Q@mail.gmail.com>

>> And I'd really like to see a CPython summit
>> happen at some point. There's so much interesting stuff going on in that
>> area that it's worth getting some people together to move these things forward.
>
> Yes, a CPython runtime summit some year would be interesting.
>
> Cheers,
> Nick.

I don't see why CPython-specific stuff can't be discussed on the
language summit. After all, everyone can be not interested in a topic
X or topic Y. I would be even more than happy to contribute my
knowledge about building VMs w.r.t. CPython implementation as much as
I could.

Cheers,
fijal

From ncoghlan at gmail.com  Sat Mar  2 18:24:01 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 3 Mar 2013 03:24:01 +1000
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
Message-ID: <CADiSq7cEJxY+Y_J7E_i0xzb7-bL7ZJjtHsYw4=E97zFbnzyVfA@mail.gmail.com>

On Sun, Mar 3, 2013 at 2:16 AM, Brett Cannon <brett at python.org> wrote:
> On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I think you should keep it. A long running service that periodically
>> scans the importers for plugins doesn't care if modules take a few
>> extra seconds to show up, it just wants to see them eventually.
>> Installers (or filesystem copy or move operations!) have no way to
>> inform arbitrary processes that new files have been added.
>
>
> But if they are doing the scan they can also easily invalidate the caches
> before performing the scan.

"I just upgraded to Python 3.4, and now my server process isn't see new plugins"

That's a major backwards compatibility breach, and hence clearly
unacceptable in my view. Even the relatively *minor* compatibility
breach of becoming dependent on the filesystem timestamp resolution
for picking up added modules, creating a race condition between
writing the file and reading it back through the import system, has
caused people grief. When you're in a hole, the first thing to do is
to *stop digging*.

You can deprecate the heuristic if you want (and can figure out how),
but a definite -1 on removing it without at least the usual
deprecation period for backwards incompatible changes.

It may also be worth tweaking the wording of the upgrade note in the
What's New to mention the need to always invalidate the cache before
scanning for new modules if you want to reliably pick up new modules
created since the application started (at the moment the note really
only mentions it as something to do after *creating* a new module).

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stefan_ml at behnel.de  Sat Mar  2 19:30:21 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sat, 02 Mar 2013 19:30:21 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgpt39$nc5$1@ger.gmane.org>
	<CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>
Message-ID: <kgtgfq$6k6$1@ger.gmane.org>

Hi Nick,

thanks for the feedback.

Nick Coghlan, 02.03.2013 17:58:
> On Fri, Mar 1, 2013 at 7:41 PM, Stefan Behnel wrote:
>> Michael Foord, 27.02.2013 17:51:
>> It's also true that many of the topics above aren't really interesting for
>> us, because we just inherit them with CPython, e.g. stdlib changes.
>> Packaging is only relevant as far as it impacts the distribution of binary
>> extensions, and the main changes appear to be outside of that area (which
>> doesn't mean it's not truly wonderful that they are happening, Python
>> packaging has seen a lot of great improvements during the last years and
>> I'm very happy to see it getting even better).
> 
> I'm puzzled by this one. Did you leave out PEP 427 (the wheel format),
> because it's already approved, and hence not likely to be discussed
> much at the summit, or because you don't consider it to impact the
> distribution of binary extensions (which would be rather odd, given
> the nature of the PEP and the wheel format...)

I admit that the wheel format has been sailing mostly below my radar (I
guess much of the discussion about it is buried somewhere in the distutils
SIG archives?), but the last time it started blinking brightly enough to
have me take a look at the PEP, I didn't really see anything that was
relevant enough to Cython to pay much attention or even comment on it. As I
understand it, it's almost exclusively about naming and metadata. Cython
compiled extensions are in no way different from plain C extensions wrt
packaging. What works for those will work for Cython just fine.

Does it imply any changes in the build system that I should be aware of?
Cython usually just runs as a preprocessor for distutils extensions, before
even calling into setup(). The rest is just a plain old distutils extension
build.


>> Interpreter initialisation would be interesting and Cython could
>> potentially help in some spots here by making code easier to maintain and
>> optimise, for example. We've had this discussion for the importlib
>> bootstrapping and I'm sure there's more that could be done. It's sad to see
>> so much C-level work go into areas that really don't need to be that low-level.
> 
> Cython's notion of embedding is the exact opposite of CPython's, so
> I'm not at all clear on how Cython could help with PEP 432 at all.

I wasn't thinking about embedding CPython in a Cython compiled program.
That would appear like a rather strange setup here.

In the context of importlib, I proposed compiling init time Python code
into statically linked extension modules in order to speed it up and make
it independent of the parser and interpreter, as an alternative to freezing
it (which requires a working VM already and implies interpretation
overhead). I agree that Cython can't help in most of the early low-level
runtime bootstrap process, but once a minimum runtime is available, the
more high-level parts of the initialisation could be done in compiled
Python code, which other implementations might be able to reuse.


>> I'm not so happy with the argument clinic, but that's certainly also
>> because I'm biased. I've written the argument unpacking code for Cython
>> some years ago, so it's not surprising that I'm quite happy with that and
>> fail to see the need for a totally new DSL *and* a totally new
>> implementation, especially with its mapping to the slowish ParseTuple*()
>> C-API functions. I've also not seen a good argument why the existing Py3
>> function signatures can't do what the proposed DSL tries to achieve. They'd
>> at least make it clear that the intention is to make things more
>> Python-like, and would at the same time provide the documentation.
> 
> That's why Stefan Krah is writing a competing PEP - a number of us
> already agree with you, and think the case needs to be made for
> choosing something completely different like Argument Clinic

I'll happily provide my feedback to that approach. It might also have a
positive impact on the usage of Py3 argument annotations, which I think
merit some more visibility and "useful use cases".


> (especially given Guido's expressed tolerance for the idea of "/" as a
> possible marker to indicate that the preceding parameters only support
> positional arguments - that was in the context of Python discussion
> where it was eventually deemed "not necessary", but becomes
> interesting again in a C API signature discussion)

I've not really had that need myself yet, but I remember thinking of it at
least once while writing Cython's argument unpacking code. I think it would
get rid of a currently existing asymmetry between positional arguments and
keyword(-only) arguments, and would remove the risk of naming collisions
with positional arguments, most notably when **kwargs is used. And yes, I
agree that it would be most interesting for C signatures, just like kwonly
arguments are really handy there. It might not be all too hard to write up
a prototype in Cython. And I should be able to find a couple of places in
lxml where I could use this as an actual feature, so I might actually give
it a try when I find the time.

Stefan



From stefan at bytereef.org  Sat Mar  2 21:01:15 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 2 Mar 2013 21:01:15 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <kgtgfq$6k6$1@ger.gmane.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgpt39$nc5$1@ger.gmane.org>
	<CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>
	<kgtgfq$6k6$1@ger.gmane.org>
Message-ID: <20130302200115.GA3305@sleipnir.bytereef.org>

Stefan Behnel <stefan_ml at behnel.de> wrote:
> >> I'm not so happy with the argument clinic, but that's certainly also
> >> because I'm biased. I've written the argument unpacking code for Cython
> >> some years ago, so it's not surprising that I'm quite happy with that and
> >> fail to see the need for a totally new DSL *and* a totally new
> >> implementation, especially with its mapping to the slowish ParseTuple*()
> >> C-API functions. I've also not seen a good argument why the existing Py3
> >> function signatures can't do what the proposed DSL tries to achieve. They'd
> >> at least make it clear that the intention is to make things more
> >> Python-like, and would at the same time provide the documentation.
> > 
> > That's why Stefan Krah is writing a competing PEP - a number of us
> > already agree with you, and think the case needs to be made for
> > choosing something completely different like Argument Clinic
> 
> I'll happily provide my feedback to that approach. It might also have a
> positive impact on the usage of Py3 argument annotations, which I think
> merit some more visibility and "useful use cases".


BTW, I think so far no one has stepped forward to implement the custom
argument handlers. I've looked at Cython output and, as you say, most of
it is there already.

Is it possible to write a minimal version of the code generator that just
produces the argument handling code?

Approximately, how many lines of code would we be talking about?



Stefan Krah




From tjreedy at udel.edu  Sat Mar  2 21:32:02 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 02 Mar 2013 15:32:02 -0500
Subject: [Python-Dev] Disabling string interning for null and
	single-char causes segfaults
In-Reply-To: <CADiSq7fvccRpSWGPWwcY-5GFYzwt9FvO8fGXOK6zdDkfZQMA7Q@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<CADiSq7fvccRpSWGPWwcY-5GFYzwt9FvO8fGXOK6zdDkfZQMA7Q@mail.gmail.com>
Message-ID: <kgtnka$2ik$1@ger.gmane.org>

On 3/2/2013 10:08 AM, Nick Coghlan wrote:
> On Sat, Mar 2, 2013 at 1:24 AM, Stefan Bucur <stefan.bucur at gmail.com> wrote:
>> Hi,
>>
>> I'm working on an automated bug finding tool that I'm trying to apply on the
>> Python interpreter code (version 2.7.3). Because of early prototype
>> limitations, I needed to disable string interning in stringobject.c. More
>> precisely, I modified the PyString_FromStringAndSize and PyString_FromString
>> to no longer check for the null and single-char cases, and create instead a
>> new string every time (I can send the patch if needed).
>>
>> However, after applying this modification, when running "make test" I get a
>> segfault in the test___all__ test case.
>>
>> Before digging deeper into the issue, I wanted to ask here if there are any
>> implicit assumptions about string identity and interning throughout the
>> interpreter implementation. For instance, are two single-char strings having
>> the same content supposed to be identical objects?
>>
>> I'm assuming that it's either this, or some refcount bug in the interpreter
>> that manifests only when certain strings are no longer interned and thus
>> have a higher chance to get low refcount values.
>
> In theory, interning is supposed to be a pure optimisation, but it
> wouldn't surprise me if there are cases that assume the described
> strings are always interned (especially the null string case). Our
> test suite would never detect such bugs, as we never disable the
> interning.

Since it required patching functions rather than a configuration switch, 
it literally seems not be a supported option. If so, I would not 
consider it a bug for CPython to use the assumption of interning to run 
faster and I don't think it should be slowed down if that would be 
necessary to remove the assumption. (This is all assuming that the 
problem is not just a ref count bug.)

Stefan's question was about 2.7. I am just curious: does 3.3 still 
intern (some) unicode chars? Did the 256 interned bytes of 2.x carry 
over to 3.x?

> Whether or not we're interested in fixing such bugs would depend on
> the size of the patches needed to address them. From our point of
> view, such bugs are purely theoretical (as the assumption is always
> valid in an unpatched CPython build), so if the problem is too hard to
> diagnose or fix, we're more likely to declare that interning of at
> least those kinds of string values is required for correctness when
> creating modified versions of CPython.

-- 
Terry Jan Reedy


From stefan.bucur at gmail.com  Sat Mar  2 21:55:07 2013
From: stefan.bucur at gmail.com (Stefan Bucur)
Date: Sat, 2 Mar 2013 21:55:07 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CADiSq7fvccRpSWGPWwcY-5GFYzwt9FvO8fGXOK6zdDkfZQMA7Q@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<CADiSq7fvccRpSWGPWwcY-5GFYzwt9FvO8fGXOK6zdDkfZQMA7Q@mail.gmail.com>
Message-ID: <CAEg840W1KdbN4_6padfntpSGDGr-zopruGf2BLAkpzr3=JwknQ@mail.gmail.com>

On Sat, Mar 2, 2013 at 4:08 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Sat, Mar 2, 2013 at 1:24 AM, Stefan Bucur <stefan.bucur at gmail.com> wrote:
>> Hi,
>>
>> I'm working on an automated bug finding tool that I'm trying to apply on the
>> Python interpreter code (version 2.7.3). Because of early prototype
>> limitations, I needed to disable string interning in stringobject.c. More
>> precisely, I modified the PyString_FromStringAndSize and PyString_FromString
>> to no longer check for the null and single-char cases, and create instead a
>> new string every time (I can send the patch if needed).
>>
>> However, after applying this modification, when running "make test" I get a
>> segfault in the test___all__ test case.
>>
>> Before digging deeper into the issue, I wanted to ask here if there are any
>> implicit assumptions about string identity and interning throughout the
>> interpreter implementation. For instance, are two single-char strings having
>> the same content supposed to be identical objects?
>>
>> I'm assuming that it's either this, or some refcount bug in the interpreter
>> that manifests only when certain strings are no longer interned and thus
>> have a higher chance to get low refcount values.
>
> In theory, interning is supposed to be a pure optimisation, but it
> wouldn't surprise me if there are cases that assume the described
> strings are always interned (especially the null string case). Our
> test suite would never detect such bugs, as we never disable the
> interning.

I understand. In this case, I'll further investigate the issue, and
see what exactly is the cause of the crash.

>
> Whether or not we're interested in fixing such bugs would depend on
> the size of the patches needed to address them. From our point of
> view, such bugs are purely theoretical (as the assumption is always
> valid in an unpatched CPython build), so if the problem is too hard to
> diagnose or fix, we're more likely to declare that interning of at
> least those kinds of string values is required for correctness when
> creating modified versions of CPython.
>
> I'm not sure what kind of analyser you are writing, but if it relates
> to the CPython C API, you may be interested in
> https://gcc-python-plugin.readthedocs.org/en/latest/cpychecker.html

That's quite a neat tool, I didn't know about it! I guess that would
have saved me many hours of debugging obscure refcount bugs in my own
Python extensions :)

In any case, my analysis tool aims to find bugs in Python programs,
not in the CPython implementation itself. It works by performing
symbolic execution [1] on the Python interpreter, while it is
executing the target Python program. This means that the Python
interpreter memory space contains symbolic expressions (i.e.,
mathematical formulas over the program input) instead of "concrete"
values.

The interned strings are pesky for symbolic execution because the
PyObject* pointer allocated when creating an interned string depends
on the string contents, e.g., if the contents are already interned,
the old pointer is returned, otherwise a new object is created. So the
pointer itself becomes "symbolic", i.e., dependant on the input data,
which makes the analysis much more complicated.

Stefan

[1] http://en.wikipedia.org/wiki/Symbolic_execution

From stefan.bucur at gmail.com  Sat Mar  2 22:13:56 2013
From: stefan.bucur at gmail.com (Stefan Bucur)
Date: Sat, 2 Mar 2013 22:13:56 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <20130302163136.18b78885@pitrou.net>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<20130302163136.18b78885@pitrou.net>
Message-ID: <CAEg840ViGMH=Vx2wpx2CETWjS6xVvRmeQQZppA+JREhOftv1hg@mail.gmail.com>

On Sat, Mar 2, 2013 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Fri, 1 Mar 2013 16:24:42 +0100
> Stefan Bucur <stefan.bucur at gmail.com> wrote:
>>
>> However, after applying this modification, when running "make test" I get a
>> segfault in the test___all__ test case.
>>
>> Before digging deeper into the issue, I wanted to ask here if there are any
>> implicit assumptions about string identity and interning throughout the
>> interpreter implementation. For instance, are two single-char strings
>> having the same content supposed to be identical objects?
>
> From a language POV, no, but inside a specific interpreter such as
> CPython it may be a reasonable expectation.
>
>> I'm assuming that it's either this, or some refcount bug in the interpreter
>> that manifests only when certain strings are no longer interned and thus
>> have a higher chance to get low refcount values.
>
> Indeed, if it's a real bug it would be nice to get it fixed :-)

By the way, in that case, what would be the best way to debug such
type of ref count errors? I recently ran across this document [1],
which kind of applies to debugging focused on newly introduced code.
But when some changes potentially impact a good fraction of the
interpreter, where should I look first?

I'm asking since I re-ran the failing test with gdb, and the segfault
seems to occur when invoking the kill() syscall, so the error seems to
manifest at some later point than when the faulty code is executed.

Stefan

[1] http://www.python.org/doc/essays/refcnt/

From lukas.lueg at gmail.com  Sat Mar  2 23:49:52 2013
From: lukas.lueg at gmail.com (Lukas Lueg)
Date: Sat, 2 Mar 2013 23:49:52 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CAEg840ViGMH=Vx2wpx2CETWjS6xVvRmeQQZppA+JREhOftv1hg@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<20130302163136.18b78885@pitrou.net>
	<CAEg840ViGMH=Vx2wpx2CETWjS6xVvRmeQQZppA+JREhOftv1hg@mail.gmail.com>
Message-ID: <CAJF-kYmmShi51cU8Ta0hPOYYko1x=Cw7wkSE2Zc5SXycMwbFsw@mail.gmail.com>

Debugging a refcount bug? Good. Out of the door, line on the left, one
cross each.


2013/3/2 Stefan Bucur <stefan.bucur at gmail.com>

> On Sat, Mar 2, 2013 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net>
> wrote:
> > On Fri, 1 Mar 2013 16:24:42 +0100
> > Stefan Bucur <stefan.bucur at gmail.com> wrote:
> >>
> >> However, after applying this modification, when running "make test" I
> get a
> >> segfault in the test___all__ test case.
> >>
> >> Before digging deeper into the issue, I wanted to ask here if there are
> any
> >> implicit assumptions about string identity and interning throughout the
> >> interpreter implementation. For instance, are two single-char strings
> >> having the same content supposed to be identical objects?
> >
> > From a language POV, no, but inside a specific interpreter such as
> > CPython it may be a reasonable expectation.
> >
> >> I'm assuming that it's either this, or some refcount bug in the
> interpreter
> >> that manifests only when certain strings are no longer interned and thus
> >> have a higher chance to get low refcount values.
> >
> > Indeed, if it's a real bug it would be nice to get it fixed :-)
>
> By the way, in that case, what would be the best way to debug such
> type of ref count errors? I recently ran across this document [1],
> which kind of applies to debugging focused on newly introduced code.
> But when some changes potentially impact a good fraction of the
> interpreter, where should I look first?
>
> I'm asking since I re-ran the failing test with gdb, and the segfault
> seems to occur when invoking the kill() syscall, so the error seems to
> manifest at some later point than when the faulty code is executed.
>
> Stefan
>
> [1] http://www.python.org/doc/essays/refcnt/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/lukas.lueg%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/8936720f/attachment.html>

From brett at python.org  Sun Mar  3 00:38:15 2013
From: brett at python.org (Brett Cannon)
Date: Sat, 2 Mar 2013 18:38:15 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CADiSq7cEJxY+Y_J7E_i0xzb7-bL7ZJjtHsYw4=E97zFbnzyVfA@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
	<CADiSq7cEJxY+Y_J7E_i0xzb7-bL7ZJjtHsYw4=E97zFbnzyVfA@mail.gmail.com>
Message-ID: <CAP1=2W53Qe7mEMYOx6DbJGMkoTzPzWjH_ppHj_bS47D2LQVxGQ@mail.gmail.com>

On Sat, Mar 2, 2013 at 12:24 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Sun, Mar 3, 2013 at 2:16 AM, Brett Cannon <brett at python.org> wrote:
> > On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com>
> wrote:
> >> I think you should keep it. A long running service that periodically
> >> scans the importers for plugins doesn't care if modules take a few
> >> extra seconds to show up, it just wants to see them eventually.
> >> Installers (or filesystem copy or move operations!) have no way to
> >> inform arbitrary processes that new files have been added.
> >
> >
> > But if they are doing the scan they can also easily invalidate the caches
> > before performing the scan.
>
> "I just upgraded to Python 3.4, and now my server process isn't see new
> plugins"
>
> That's a major backwards compatibility breach, and hence clearly
> unacceptable in my view. Even the relatively *minor* compatibility
> breach of becoming dependent on the filesystem timestamp resolution
> for picking up added modules, creating a race condition between
> writing the file and reading it back through the import system, has
> caused people grief. When you're in a hole, the first thing to do is
> to *stop digging*.
>
> You can deprecate the heuristic if you want (and can figure out how),
> but a definite -1 on removing it without at least the usual
> deprecation period for backwards incompatible changes.
>

That part is easy: ImportWarning still exists so simply continuing to check
the directory and noticing when a difference exists that affects subsequent
imports and then raising the warning will handle that.


>
> It may also be worth tweaking the wording of the upgrade note in the
> What's New to mention the need to always invalidate the cache before
> scanning for new modules if you want to reliably pick up new modules
> created since the application started (at the moment the note really
> only mentions it as something to do after *creating* a new module).
>
>
As of right now with the check that's all that is needed, but yes, if the
deprecation does occur it would be worth changing it.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130302/de0a1b0e/attachment.html>

From solipsis at pitrou.net  Sun Mar  3 00:36:36 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 3 Mar 2013 00:36:36 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CAEg840ViGMH=Vx2wpx2CETWjS6xVvRmeQQZppA+JREhOftv1hg@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<20130302163136.18b78885@pitrou.net>
	<CAEg840ViGMH=Vx2wpx2CETWjS6xVvRmeQQZppA+JREhOftv1hg@mail.gmail.com>
Message-ID: <20130303003636.505ed760@pitrou.net>

On Sat, 2 Mar 2013 22:13:56 +0100
Stefan Bucur <stefan.bucur at gmail.com> wrote:

> On Sat, Mar 2, 2013 at 4:31 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> > On Fri, 1 Mar 2013 16:24:42 +0100
> > Stefan Bucur <stefan.bucur at gmail.com> wrote:
> >>
> >> However, after applying this modification, when running "make test" I get a
> >> segfault in the test___all__ test case.
> >>
> >> Before digging deeper into the issue, I wanted to ask here if there are any
> >> implicit assumptions about string identity and interning throughout the
> >> interpreter implementation. For instance, are two single-char strings
> >> having the same content supposed to be identical objects?
> >
> > From a language POV, no, but inside a specific interpreter such as
> > CPython it may be a reasonable expectation.
> >
> >> I'm assuming that it's either this, or some refcount bug in the interpreter
> >> that manifests only when certain strings are no longer interned and thus
> >> have a higher chance to get low refcount values.
> >
> > Indeed, if it's a real bug it would be nice to get it fixed :-)
> 
> By the way, in that case, what would be the best way to debug such
> type of ref count errors? I recently ran across this document [1],
> which kind of applies to debugging focused on newly introduced code.

That documents looks a bit outdated (1998!).
I would suggest you enable core dumps (`ulimit -c unlimited`), then let
Python crash and inspect the stack trace with gdb.
You will get better results if using a debug build and the modern gdb
inspection helpers:
http://docs.python.org/devguide/gdb.html

Oh, by the way, it would be better to do your work on Python 3 rather
than 2.7. Either the `default` branch or the `3.3` branch, I guess.
See http://docs.python.org/devguide/setup.html#checkout

Regards

Antoine.

From solipsis at pitrou.net  Sun Mar  3 00:51:57 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 3 Mar 2013 00:51:57 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <855C0532-804F-48BB-BE16-2604D2DF0AE4@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<20130227195021.0101c919@pitrou.net>
	<855C0532-804F-48BB-BE16-2604D2DF0AE4@voidspace.org.uk>
Message-ID: <20130303005157.5911a233@pitrou.net>

On Thu, 28 Feb 2013 11:39:52 +0000
Michael Foord <michael at voidspace.org.uk> wrote:
> > 
> > Perhaps someone wants to discuss
> > http://www.python.org/dev/peps/pep-0428/, but I won't be there and the
> > PEP isn't terribly up-to-date either :-)
> 
> If you can find someone familiar with pathlib to champion the discussion it is more likely to happen and be productive... Getting the PEP up to date before the summit will also help. (I very much like the *idea* of pathlib and the bits I've seen / read through - but I haven't used it in anger yet so I don't feel qualified to champion it myself.)

I've made the PEP up-to-date now.
http://mail.python.org/pipermail/python-ideas/2013-March/019731.html

Regards

Antoine.

From erik.m.bray at gmail.com  Sun Mar  3 02:09:49 2013
From: erik.m.bray at gmail.com (Erik Bray)
Date: Sat, 2 Mar 2013 20:09:49 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
Message-ID: <CAOTD34a08p_NHRDOcvm39Pyw7tO3LSK=5_K4L_Snx-LjrXtLxA@mail.gmail.com>

On Sat, Mar 2, 2013 at 10:36 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> In addition, it may be appropriate for importlib to offer a
> "write_module" method that accepts (module name, target path,
> contents). This would:
>
> 1. Allow in-process caches to be invalidated implicitly and
> selectively when new modules are created
> 2. Allow importers to abstract write access in addition to read access
> 3. Allow the import system to complain at time of writing if the
> desired module name and target path don't actually match given the
> current import system state.

+1 to write_module().  This would be useful in general, I think.
Though perhaps the best solution to the original problem is to more
forcefully document: "If you're writing a module and expect to be able
to import it immediately within the same process, it's necessary to
manually invalidate the directory cache."

I might go a little further and suggest adding a function to only
invalidate the cache for the relevant directory (the proposed
write_module() function could do this).  This can already be done with
something like:

dirname = os.path.dirname(module_filename)
sys.path_importer_cache[dirname].invalidate_caches()

But that's a bit onerous considering that this wasn't even necessary
before 3.3.  There should be an easier way to do this, as there's no
sense in invalidating all the directory caches if one is only writing
new modules to a specific directory or directories.

Erik

From solipsis at pitrou.net  Sun Mar  3 02:16:30 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 3 Mar 2013 02:16:30 +0100
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
Message-ID: <20130303021630.3f5340d5@pitrou.net>

On Sat, 2 Mar 2013 11:16:28 -0500
Brett Cannon <brett at python.org> wrote:
> > In addition, it may be appropriate for importlib to offer a
> > "write_module" method that accepts (module name, target path,
> > contents). This would:
> >
> > 1. Allow in-process caches to be invalidated implicitly and
> > selectively when new modules are created
> 
> I don't think that's necessary. If people don't want to blindly clear all
> caches for a file they can write the file, search the keys in
> sys.path_importer_cache for the longest prefix for the newly created file,
> and then call the invalidate_cache() method on that explicit finder.

That's too complicated for non-import experts IMHO.

Regards

Antoine.



From trent at snakebite.org  Sun Mar  3 02:29:40 2013
From: trent at snakebite.org (Trent Nelson)
Date: Sat, 2 Mar 2013 20:29:40 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
Message-ID: <20130303012939.GC62205@snakebite.org>

On Wed, Feb 27, 2013 at 08:51:16AM -0800, Michael Foord wrote:
> If you have other items you'd like to discuss please let me know and I
> can add them to the agenda.

    Hmm, seems like this might be a good forum to introduce the
    parallel/async stuff I've been working on the past few months.
    TL;DR version is I've come up with an alternative approach for
    exploiting multiple cores that doesn't rely on GIL-removal or
    STM (and has a negligible performance overhead when executing
    single-threaded code).  (For those that are curious, it lives
    in the px branch of the sandbox/trent repo on hg.p.o, albeit
    in a very experimental/prototype/proof-of-concept state (i.e.
    it's an unorganized, undocumented, uncommented hackfest); on
    the plus side, it works.  Sort of.)

    Second suggestion: perhaps a little segment on Snakebite?  What
    it is, what's available to committers, feedback/kvetching from
    those who have already used it, etc.

    (I forgot the format of these summits -- is there a projector?)

        Trent.

From stefan_ml at behnel.de  Sun Mar  3 11:14:41 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 03 Mar 2013 11:14:41 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130302200115.GA3305@sleipnir.bytereef.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgpt39$nc5$1@ger.gmane.org>
	<CADiSq7da5beuHEzkT-+MS6dtH4pydJ-Kr-Kq_i8ppnA3aO5RwQ@mail.gmail.com>
	<kgtgfq$6k6$1@ger.gmane.org>
	<20130302200115.GA3305@sleipnir.bytereef.org>
Message-ID: <kgv7qd$v30$1@ger.gmane.org>

Stefan Krah, 02.03.2013 21:01:
> Stefan Behnel wrote:
>>>> I'm not so happy with the argument clinic, but that's certainly also
>>>> because I'm biased. I've written the argument unpacking code for Cython
>>>> some years ago, so it's not surprising that I'm quite happy with that and
>>>> fail to see the need for a totally new DSL *and* a totally new
>>>> implementation, especially with its mapping to the slowish ParseTuple*()
>>>> C-API functions. I've also not seen a good argument why the existing Py3
>>>> function signatures can't do what the proposed DSL tries to achieve. They'd
>>>> at least make it clear that the intention is to make things more
>>>> Python-like, and would at the same time provide the documentation.
>>>
>>> That's why Stefan Krah is writing a competing PEP - a number of us
>>> already agree with you, and think the case needs to be made for
>>> choosing something completely different like Argument Clinic
>>
>> I'll happily provide my feedback to that approach. It might also have a
>> positive impact on the usage of Py3 argument annotations, which I think
>> merit some more visibility and "useful use cases".
> 
> 
> BTW, I think so far no one has stepped forward to implement the custom
> argument handlers. I've looked at Cython output and, as you say, most of
> it is there already.
> 
> Is it possible to write a minimal version of the code generator that just
> produces the argument handling code?

It should be possible, although it does have a lot of dependencies on
Cython's type system, so a part of that would have to be extracted as well
or at least emulated. Conversion functions are based on it, for example.

However, I think it would actually be easiest to just let Cython generate
the module interface completely. I.e. you'd remove all code that currently
deals with Python function signatures from the C module, only leaving the
bare C API, and then generate a Cython interface module like this:

  cdef extern from *:
      object original_c_xyzfunc(object x, int y, double z)

  def xyzfunc(x, int y=0, *, double z=1.0):
      "docstring goes here"
      return original_c_xyzfunc(x,y,z)

Finally, #include the generated C file at the end of the original module.
There'd be a bit of a hassle with the module init function, I guess. Maybe
renaming it in the Cython C code (even just a #define) and calling it from
the original module would work. Or do it the other way round and add a hook
function somewhere that does the manually written parts of the module
setup. Sounds simple enough in both cases, although I'm sure there's lots
of little details. Extension types and their methods are certainly part of
those details ...

Stefan



From solipsis at pitrou.net  Sun Mar  3 12:29:48 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 3 Mar 2013 12:29:48 +0100
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
	<CADiSq7cEJxY+Y_J7E_i0xzb7-bL7ZJjtHsYw4=E97zFbnzyVfA@mail.gmail.com>
	<CAP1=2W53Qe7mEMYOx6DbJGMkoTzPzWjH_ppHj_bS47D2LQVxGQ@mail.gmail.com>
Message-ID: <20130303122948.67bcc39b@pitrou.net>

On Sat, 2 Mar 2013 18:38:15 -0500
Brett Cannon <brett at python.org> wrote:
> >
> > You can deprecate the heuristic if you want (and can figure out how),
> > but a definite -1 on removing it without at least the usual
> > deprecation period for backwards incompatible changes.
> >
> 
> That part is easy: ImportWarning still exists so simply continuing to check
> the directory and noticing when a difference exists that affects subsequent
> imports and then raising the warning will handle that.

Won't that raise spurious ImportWarnings for people who don't actually
care about that?

Regards

Antoine.



From doug.hellmann at gmail.com  Sun Mar  3 15:11:06 2013
From: doug.hellmann at gmail.com (Doug Hellmann)
Date: Sun, 3 Mar 2013 09:11:06 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADiSq7dHzP+-p2zbGxOScGJGXBhj=gByEZJcrO=O5QEoMR731g@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<31B54506-7B08-4E77-B966-A6171BBC3DA6@gmail.com>
	<CADiSq7dHzP+-p2zbGxOScGJGXBhj=gByEZJcrO=O5QEoMR731g@mail.gmail.com>
Message-ID: <61AACA32-A8E5-4AF6-9246-C5CB99C397D4@gmail.com>


On Mar 2, 2013, at 11:41 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:

> On Fri, Mar 1, 2013 at 9:39 AM, Doug Hellmann <doug.hellmann at gmail.com> wrote:
>> 
>> On Feb 27, 2013, at 11:51 AM, Michael Foord wrote:
>> 
>>> Hello all,
>>> 
>>> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
>>> 
>>> The agenda of topics for discussion so far includes the following:
>>> 
>>> * A report on pypy status - Maciej and Armin
>>> * Jython and IronPython status reports - Dino / Frank
>>> * Packaging (Doug Hellmann and Monty Taylor at least)
>> 
>> Since the time I suggested we add packaging to the agenda, Nick has set up a separate summit meeting for Friday evening. I don't know if it makes sense to leave this on the agenda for Wednesday or not.
>> 
>> Nick, what do you think?
> 
> I think it's definitely worth my taking some time to explain my goals
> for the Friday night session, and some broader things in terms of
> where I'd like to see packaging going, but a lot of the key packaging
> people aren't involved in Python language development *per se*, and
> hence won't be at the language summit.

OK, a summary seems like a good idea. 


> 
> There's also one controversial point that *does* need to be raised at
> the summit: I would like to make distutils-sig the true authority for
> packaging standards, so we can stop cross-posting PEPs intended to
> apply to packaging in *current* versions of Python to python-dev. The
> split discussions suck, and most of the people that need to be
> convinced in order for packaging standards to be supported in current
> versions of Python aren't on python-dev, since it's a tooling issue
> rather than a language design issue. Standard lib support is necessary
> in the long run to provide a good "batteries included" experience, but
> it's *not* the way to create the batteries in the first place. Until
> these standards have been endorsed by the authors of *existing*
> packaging tools, proposing them for stdlib addition is premature, but
> has been perceived as necessary in the past due to the confused power
> structure.

+1 -- anything to reduce the confusion about where to get involved :)

> 
> This means that those core developers that want a say in the future
> direction of packaging and distribution of Python software would need
> to be actively involved in the ongoing discussions on distutils-sig,
> rather than relying on being given an explicit invitation to weigh in
> at the last minute through a thread (or threads) on python-dev. The
> requirement that BDFL-delegates for packaging and distribution related
> PEPs also be experienced core developers will remain, however, as
> "suitable for future stdlib inclusion" is an important overarching
> requirement for packaging and distribution standards. Such delegates
> will just be expected to participate actively in distutils-sig *as
> well as* python-dev.
> 
> Proposals for *actual* standard library updates (to bring it into line
> with updated packaging standards) would still be subject to python-dev
> discussion and authority (and would *not* have their Discussions-To
> header set). Such discussions aren't particularly relevant to most of
> the packaging tool developers, since the standard library version
> isn't updated frequently enough to be useful to them, and also isn't
> available on older Python releases, so python-dev is a more
> appropriate venue from both perspectives.
> 
> At the moment, python-dev, catalog-sig and distutils-sig create an
> awkward trinity where decision making authority and the appropriate
> venues for discussion are grossly unclear. I consider this to be one
> of the key reasons that working on packaging issues has quite a high
> incidence of developer burnout - it's hard to figure out who needs to
> be convinced of what, so it's easy for the frustration levels to reach
> the "this just isn't worth the hassle" stage (especially when trying
> to bring python-dev members up to speed on discussions that may have
> taken months on distutils-sig, and when many of the details are
> awkward compromises forced by the need to support *existing* tools and
> development processes on older versions of Python). Under my proposal,
> the breakdown would be slightly clearer:
> 
> distutils-sig: overall authority for packaging and distribution
> related standards, *including* the interfaces between index servers
> (such as PyPI) and automated tools. If a PEP has "Discussions-To" set
> to distutils-sig, announcements of new PEPs, new versions of those
> PEPs, *and* their acceptance or rejection should be announced there,
> and *not* on python-dev. The "Resolution" header will thus point to a
> distutils-sig post rather than a python-dev one. distutils-sig will
> focus on solutions that work for *current* versions of Python, while
> keeping in mind the need for future stdlib support.
> 
> python-dev: authority over stdlib support for packaging and
> distribution standards, and the "batteries included" experience of
> interacting with those standards. Until a next generation distribution
> infrastructure is firmly established (which may involve years of
> running the legacy infrastructure and the next generation metadata 2.x
> based infrastructure in parallel), the stdlib will typically trail the
> upstream standards substantially, since many upstream enhancements
> will run afoul of the standard library's "no new features" rule,
> preventing their inclusion in maintenance releases of old Python
> versions.
> 
> catalog-sig: authority over the PyPI index server (supported by the
> infrastructure SIG for actual operation of the service). Key people
> are expected to also participate in distutils-sig, as that is where
> the expected interface exposed to automated tools will be defined. How
> PyPI exposes the packaging and distribution standards to *users*
> through its web UI will be up to catalog-sig.
> 
> The vehicle for *documenting* such a change in policy would be an
> update to PEP 1 to indicate that the "Discussions-To" header should
> always point to the list where any formal acceptance of rejection of
> the PEP would be announced. If preliminary discussions take place on a
> feeder list like python-ideas, or import-sig, or somewhere else, that
> will be declared as explicitly irrelevant from the PEP metadata point
> of view: the Discussions-To field would be documented as referring to
> the list that any Resolution field will eventually reference.
> 
> Regards,
> NIck.
> 
> -- 
> Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From brett at python.org  Sun Mar  3 18:08:00 2013
From: brett at python.org (Brett Cannon)
Date: Sun, 3 Mar 2013 12:08:00 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <20130303021630.3f5340d5@pitrou.net>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
	<20130303021630.3f5340d5@pitrou.net>
Message-ID: <CAP1=2W4VEd1o2LvqbE=ZTbKyptg8k3ZinU76xkxao0htEPBE9A@mail.gmail.com>

On Sat, Mar 2, 2013 at 8:16 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat, 2 Mar 2013 11:16:28 -0500
> Brett Cannon <brett at python.org> wrote:
> > > In addition, it may be appropriate for importlib to offer a
> > > "write_module" method that accepts (module name, target path,
> > > contents). This would:
> > >
> > > 1. Allow in-process caches to be invalidated implicitly and
> > > selectively when new modules are created
> >
> > I don't think that's necessary. If people don't want to blindly clear all
> > caches for a file they can write the file, search the keys in
> > sys.path_importer_cache for the longest prefix for the newly created
> file,
> > and then call the invalidate_cache() method on that explicit finder.
>
> That's too complicated for non-import experts IMHO.
>

Which is why they can just call importlib.import_module().
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130303/96c780b4/attachment.html>

From brett at python.org  Sun Mar  3 18:12:27 2013
From: brett at python.org (Brett Cannon)
Date: Sun, 3 Mar 2013 12:12:27 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <20130303122948.67bcc39b@pitrou.net>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
	<CADiSq7cEJxY+Y_J7E_i0xzb7-bL7ZJjtHsYw4=E97zFbnzyVfA@mail.gmail.com>
	<CAP1=2W53Qe7mEMYOx6DbJGMkoTzPzWjH_ppHj_bS47D2LQVxGQ@mail.gmail.com>
	<20130303122948.67bcc39b@pitrou.net>
Message-ID: <CAP1=2W6=MP6D1TOb+tmztvgWzVwrn_y+Js0XJ-x=n-4pr-3LdQ@mail.gmail.com>

On Sun, Mar 3, 2013 at 6:29 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat, 2 Mar 2013 18:38:15 -0500
> Brett Cannon <brett at python.org> wrote:
> > >
> > > You can deprecate the heuristic if you want (and can figure out how),
> > > but a definite -1 on removing it without at least the usual
> > > deprecation period for backwards incompatible changes.
> > >
> >
> > That part is easy: ImportWarning still exists so simply continuing to
> check
> > the directory and noticing when a difference exists that affects
> subsequent
> > imports and then raising the warning will handle that.
>
> Won't that raise spurious ImportWarnings for people who don't actually
> care about that?
>

It shouldn't. If the implementation I have in my head works (set of
original files, another set of what mtime says is there to know what would
not have been found w/o the cache invalidation), then it will only come up
when someone would break in Python 3.5 if the cache invalidation is
removed. Plus warnings are off by default as it is.


>
> Regards
>
> Antoine.
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/brett%40python.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130303/42418fd5/attachment.html>

From brett at python.org  Sun Mar  3 18:31:56 2013
From: brett at python.org (Brett Cannon)
Date: Sun, 3 Mar 2013 12:31:56 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CAP1=2W4VEd1o2LvqbE=ZTbKyptg8k3ZinU76xkxao0htEPBE9A@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
	<20130303021630.3f5340d5@pitrou.net>
	<CAP1=2W4VEd1o2LvqbE=ZTbKyptg8k3ZinU76xkxao0htEPBE9A@mail.gmail.com>
Message-ID: <CAP1=2W5TzOM5-kr8onoH=1ExRAU7dW1e32Pn7pHKWTe9Rhv0Fg@mail.gmail.com>

On Sun, Mar 3, 2013 at 12:08 PM, Brett Cannon <brett at python.org> wrote:

>
>
>
> On Sat, Mar 2, 2013 at 8:16 PM, Antoine Pitrou <solipsis at pitrou.net>wrote:
>
>> On Sat, 2 Mar 2013 11:16:28 -0500
>> Brett Cannon <brett at python.org> wrote:
>> > > In addition, it may be appropriate for importlib to offer a
>> > > "write_module" method that accepts (module name, target path,
>> > > contents). This would:
>> > >
>> > > 1. Allow in-process caches to be invalidated implicitly and
>> > > selectively when new modules are created
>> >
>> > I don't think that's necessary. If people don't want to blindly clear
>> all
>> > caches for a file they can write the file, search the keys in
>> > sys.path_importer_cache for the longest prefix for the newly created
>> file,
>> > and then call the invalidate_cache() method on that explicit finder.
>>
>> That's too complicated for non-import experts IMHO.
>>
>
> Which is why they can just call importlib.import_module().
>

That obviously should have said importlib.invalidate_caches(). =)

But how about this as a compromise over introducing write_module():
invalidate_caches() can take a path for something to specifically
invalidate. The path can then be passed to the invalidate_caches() on
sys.meta_path. In the case of PathFinder it would take that path, try to
find the directory in sys.path_importer_cache, and then invalidate the most
specific finder for that path (if there is one that has any directory
prefix match).

Lots of little details to specify (e.g. absolute path forced anywhere in
case a relative path is passed in by sys.path is all absolute paths? How do
we know something is a file if it has not been written yet?), but this
would prevent importlib from subsuming file writing specifically for source
files and minimize performance overhead of invalidating all caches for a
single file.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130303/5e6880bc/attachment.html>

From robertc at robertcollins.net  Mon Mar  4 06:24:21 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 4 Mar 2013 18:24:21 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
Message-ID: <CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>

On 28 February 2013 05:51, Michael Foord <michael at voidspace.org.uk> wrote:
> Hello all,
>
> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
>
> The agenda of topics for discussion so far includes the following:
>
> * A report on pypy status - Maciej and Armin
> * Jython and IronPython status reports - Dino / Frank
> * Packaging (Doug Hellmann and Monty Taylor at least)
> * Cleaning up interpreter initialisation (both in hopes of finding areas
>   to rationalise and hence speed things up, as well as making things
>   more embedding friendly). Nick Coghlan
> * Adding new async capabilities to the standard library (Guido)
> * cffi and the standard library - Maciej
> * flufl.enum and the standard library - Barry Warsaw
> * The argument clinic - Larry Hastings
>
> If you have other items you'd like to discuss please let me know and I can add them to the agenda.

I'd like to talk about overhauling - not tweaking, overhauling - the
standard library testing facilities.

-Rob


--
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From guido at python.org  Mon Mar  4 06:54:00 2013
From: guido at python.org (Guido van Rossum)
Date: Sun, 3 Mar 2013 21:54:00 -0800
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
Message-ID: <CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>

On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins
<robertc at robertcollins.net> wrote:
> I'd like to talk about overhauling - not tweaking, overhauling - the
> standard library testing facilities.

That seems like too big a topic and too vague a description to discuss
usefully. Perhaps you have a specific proposal? Or at least just a use
case that's poorly covered?

TBH, your choice of words is ambiguous -- are you interested in
overhauling the facilities for testing *of* the standard library (i.e.
the 'test' package), or the testing facilities *provided by* the
standard library (i.e. the unittest module)?

-- 
--Guido van Rossum (python.org/~guido)

From robertc at robertcollins.net  Mon Mar  4 07:26:12 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 4 Mar 2013 19:26:12 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
Message-ID: <CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>

On 4 March 2013 18:54, Guido van Rossum <guido at python.org> wrote:
> On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins
> <robertc at robertcollins.net> wrote:
>> I'd like to talk about overhauling - not tweaking, overhauling - the
>> standard library testing facilities.
>
> That seems like too big a topic and too vague a description to discuss
> usefully. Perhaps you have a specific proposal? Or at least just a use
> case that's poorly covered?

I have both - I have a draft implementation for a new test result API
(and forwards and backwards compat code etc), and use cases that drive
it. I started a thread here -
http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html
, with blog posts
https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/
https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/
https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/
https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/

They are focused on subunit, but much of subunit's friction has been
due to issues encountered from the stdlibrary TestResult API - in
particular three things:
 - the single-active-test model that the current API (or at least
implementation) has.
 - the expectation that all test outcomes will originate from the same
interpreter (or something with a live traceback object)
 - the inability to supply details about errors other than the exception

All of which start to bite rather deep when working on massively
parallel test environments.

It is of course possible for subunit and related tools to run their
own implementation, but it seems ideal to me to have a common API
which regular unittest, nose, py.test and others can all agree on and
use : better reuse for pretty printers, GUI displays and the like
depend on some common API.

> TBH, your choice of words is ambiguous -- are you interested in
> overhauling the facilities for testing *of* the standard library (i.e.
> the 'test' package), or the testing facilities *provided by* the
> standard library (i.e. the unittest module)?

Sorry! Testing facilities provided by the standard library. They
should naturally facilitate testing of the standard library too.

-Rob

> --
> --Guido van Rossum (python.org/~guido)



-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From ncoghlan at gmail.com  Mon Mar  4 07:40:48 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 4 Mar 2013 16:40:48 +1000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
Message-ID: <CADiSq7fDyN2YVCtoU9taT6Lwae2GR6a70vME1Rko6++YkH=E4A@mail.gmail.com>

On Mon, Mar 4, 2013 at 4:26 PM, Robert Collins
<robertc at robertcollins.net> wrote:
> On 4 March 2013 18:54, Guido van Rossum <guido at python.org> wrote:
>> On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins
>> <robertc at robertcollins.net> wrote:
>>> I'd like to talk about overhauling - not tweaking, overhauling - the
>>> standard library testing facilities.
>>
>> That seems like too big a topic and too vague a description to discuss
>> usefully. Perhaps you have a specific proposal? Or at least just a use
>> case that's poorly covered?
>
> I have both - I have a draft implementation for a new test result API
> (and forwards and backwards compat code etc), and use cases that drive
> it. I started a thread here -
> http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html
> , with blog posts
> https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/
> https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/
> https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/
> https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/
>
> They are focused on subunit, but much of subunit's friction has been
> due to issues encountered from the stdlibrary TestResult API - in
> particular three things:
>  - the single-active-test model that the current API (or at least
> implementation) has.
>  - the expectation that all test outcomes will originate from the same
> interpreter (or something with a live traceback object)
>  - the inability to supply details about errors other than the exception
>
> All of which start to bite rather deep when working on massively
> parallel test environments.

Your feedback on http://bugs.python.org/issue16997 would be greatly appreciated.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From robertc at robertcollins.net  Mon Mar  4 07:46:07 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Mon, 4 Mar 2013 19:46:07 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADiSq7fDyN2YVCtoU9taT6Lwae2GR6a70vME1Rko6++YkH=E4A@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<CADiSq7fDyN2YVCtoU9taT6Lwae2GR6a70vME1Rko6++YkH=E4A@mail.gmail.com>
Message-ID: <CAJ3HoZ3hzg89sWN9=d8afMdo2DE0gSyJW_GdezJWFmSw=F1N8Q@mail.gmail.com>

On 4 March 2013 19:40, Nick Coghlan <ncoghlan at gmail.com> wrote:

> Your feedback on http://bugs.python.org/issue16997 would be greatly appreciated.

Done directly to Antoine on IRC the other day in a conversation with
him and Michael about the compatability impact of subtests. Happy to
do a full code review if that would be useful.

-Rob


-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From ncoghlan at gmail.com  Mon Mar  4 07:58:04 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 4 Mar 2013 16:58:04 +1000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ3hzg89sWN9=d8afMdo2DE0gSyJW_GdezJWFmSw=F1N8Q@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<CADiSq7fDyN2YVCtoU9taT6Lwae2GR6a70vME1Rko6++YkH=E4A@mail.gmail.com>
	<CAJ3HoZ3hzg89sWN9=d8afMdo2DE0gSyJW_GdezJWFmSw=F1N8Q@mail.gmail.com>
Message-ID: <CADiSq7eZoDYZWiO-oWTOraQVN3py=2pPZ4VuSUj+st3maZSfWw@mail.gmail.com>

On Mon, Mar 4, 2013 at 4:46 PM, Robert Collins
<robertc at robertcollins.net> wrote:
> On 4 March 2013 19:40, Nick Coghlan <ncoghlan at gmail.com> wrote:
>
>> Your feedback on http://bugs.python.org/issue16997 would be greatly appreciated.
>
> Done directly to Antoine on IRC the other day in a conversation with
> him and Michael about the compatability impact of subtests. Happy to
> do a full code review if that would be useful.

The extra set of eyes couldn't hurt (and if you can spot a better way
to tie @expectedfailure into the rest of the test running machinery,
that would be great. Making sure that decorator doesn't break is the
ugliest part of the whole patch)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From solipsis at pitrou.net  Mon Mar  4 10:29:13 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 4 Mar 2013 10:29:13 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<CADiSq7fDyN2YVCtoU9taT6Lwae2GR6a70vME1Rko6++YkH=E4A@mail.gmail.com>
	<CAJ3HoZ3hzg89sWN9=d8afMdo2DE0gSyJW_GdezJWFmSw=F1N8Q@mail.gmail.com>
Message-ID: <20130304102913.030ef355@pitrou.net>

Le Mon, 4 Mar 2013 19:46:07 +1300,
Robert Collins <robertc at robertcollins.net> a ?crit :
> On 4 March 2013 19:40, Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> > Your feedback on http://bugs.python.org/issue16997 would be greatly
> > appreciated.
> 
> Done directly to Antoine on IRC the other day in a conversation with
> him and Michael about the compatability impact of subtests. Happy to
> do a full code review if that would be useful.

Indeed and some of the changes in the latest patch stem from that
conversation.

Regards

Antoine.



From storchaka at gmail.com  Mon Mar  4 16:32:21 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 04 Mar 2013 17:32:21 +0200
Subject: [Python-Dev] Disabling string interning for null and
	single-char causes segfaults
In-Reply-To: <kgtnka$2ik$1@ger.gmane.org>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<CADiSq7fvccRpSWGPWwcY-5GFYzwt9FvO8fGXOK6zdDkfZQMA7Q@mail.gmail.com>
	<kgtnka$2ik$1@ger.gmane.org>
Message-ID: <kh2eq7$gdi$1@ger.gmane.org>

On 02.03.13 22:32, Terry Reedy wrote:
> I am just curious: does 3.3 still
> intern (some) unicode chars? Did the 256 interned bytes of 2.x carry
> over to 3.x?

Yes, Python 3 interns an empty string and first 256 Unicode characters.



From storchaka at gmail.com  Mon Mar  4 16:40:54 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 04 Mar 2013 17:40:54 +0200
Subject: [Python-Dev] Disabling string interning for null and
	single-char causes segfaults
In-Reply-To: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
Message-ID: <kh2fa6$lg2$1@ger.gmane.org>

On 01.03.13 17:24, Stefan Bucur wrote:
> Before digging deeper into the issue, I wanted to ask here if there are
> any implicit assumptions about string identity and interning throughout
> the interpreter implementation. For instance, are two single-char
> strings having the same content supposed to be identical objects?

I think this is not a bug if the code relies on the fact that an empty 
string is a singleton. This obviously is an immutable object and there 
is no public method to create different empty string. But a user can 
create different 1-character strings with same value (first create 
uninitialized a 1-character string and than fill a content). If some 
code fails when none of 1-character strings are interned, this obviously 
is a bug.



From barry at python.org  Mon Mar  4 17:29:37 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 11:29:37 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
Message-ID: <20130304112937.520651d4@anarchist.wooz.org>

On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:

>It is of course possible for subunit and related tools to run their
>own implementation, but it seems ideal to me to have a common API
>which regular unittest, nose, py.test and others can all agree on and
>use : better reuse for pretty printers, GUI displays and the like
>depend on some common API.

And One True Way of invoking and/or discovering how to invoke, a package's
test suite.

-Barry

From brian at python.org  Mon Mar  4 17:30:12 2013
From: brian at python.org (Brian Curtin)
Date: Mon, 4 Mar 2013 10:30:12 -0600
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
Message-ID: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>

The full announcement is at
http://blog.python.org/2013/03/introducing-electronic-contributor.html,
but a summary follows.

We've now moved to an electronic Contributor License Agreement form at
http://www.python.org/psf/contrib/contrib-form/ which will hopefully
ease the signing and sending of forms for our potential contributors.
The form shows the required fields whether you're signing as an
individual or a representative of an organization, and removes the
need to print, scan, fax, etc.

When a new contributor fills in the form, they are emailed a copy of
the form and asked to confirm the email address that they used (and
received that copy at). Upon confirming, the signed form is sent to
the PSF Administrator and filed away.

The signature can either be generated from your typed name, or you can
draw or upload your actual written signature if you choose.

From brett at python.org  Mon Mar  4 17:34:34 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 4 Mar 2013 11:34:34 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304112937.520651d4@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
Message-ID: <CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>

On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw <barry at python.org> wrote:

> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
>
> >It is of course possible for subunit and related tools to run their
> >own implementation, but it seems ideal to me to have a common API
> >which regular unittest, nose, py.test and others can all agree on and
> >use : better reuse for pretty printers, GUI displays and the like
> >depend on some common API.
>
> And One True Way of invoking and/or discovering how to invoke, a package's
> test suite.
>

How does unittest's test discovery not solve that?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/c669419e/attachment.html>

From brett at python.org  Mon Mar  4 17:36:15 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 4 Mar 2013 11:36:15 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
Message-ID: <CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>

On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin <brian at python.org> wrote:

> The full announcement is at
> http://blog.python.org/2013/03/introducing-electronic-contributor.html,
> but a summary follows.
>
> We've now moved to an electronic Contributor License Agreement form at
> http://www.python.org/psf/contrib/contrib-form/ which will hopefully
> ease the signing and sending of forms for our potential contributors.
> The form shows the required fields whether you're signing as an
> individual or a representative of an organization, and removes the
> need to print, scan, fax, etc.
>
> When a new contributor fills in the form, they are emailed a copy of
> the form and asked to confirm the email address that they used (and
> received that copy at). Upon confirming, the signed form is sent to
> the PSF Administrator and filed away.
>
> The signature can either be generated from your typed name, or you can
> draw or upload your actual written signature if you choose.


With this in place I would like to propose that all patches submitted to
bugs.python.org must come from someone who has signed the CLA before we
consider committing it (if you want to be truly paranoid we could say that
we won't even look at the code w/o a CLA).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/c89639e3/attachment.html>

From cournape at gmail.com  Mon Mar  4 17:44:16 2013
From: cournape at gmail.com (David Cournapeau)
Date: Mon, 4 Mar 2013 16:44:16 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
Message-ID: <CAGY4rcUqxSMyyb4CP1dnfsCm8W-mKEuwonAE_bvMe4ixCO+czQ@mail.gmail.com>

On Mon, Mar 4, 2013 at 4:34 PM, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw <barry at python.org> wrote:
>>
>> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
>>
>> >It is of course possible for subunit and related tools to run their
>> >own implementation, but it seems ideal to me to have a common API
>> >which regular unittest, nose, py.test and others can all agree on and
>> >use : better reuse for pretty printers, GUI displays and the like
>> >depend on some common API.
>>
>> And One True Way of invoking and/or discovering how to invoke, a package's
>> test suite.
>
>
> How does unittest's test discovery not solve that?

It is not always obvious how to test a package when one is not
familiar with it. Are the tests in pkgname/tests or tests or ... ?

In the scientific community, we have used the convention of making the
test suite available at runtime with pkgname.tests().

David

From brett at python.org  Mon Mar  4 17:47:34 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 4 Mar 2013 11:47:34 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAGY4rcUqxSMyyb4CP1dnfsCm8W-mKEuwonAE_bvMe4ixCO+czQ@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<CAGY4rcUqxSMyyb4CP1dnfsCm8W-mKEuwonAE_bvMe4ixCO+czQ@mail.gmail.com>
Message-ID: <CAP1=2W5F=fq6p9ftEnsq=2ZVr+TOLBaWF2PFTnAGnqwy8+1p1w@mail.gmail.com>

On Mon, Mar 4, 2013 at 11:44 AM, David Cournapeau <cournape at gmail.com>wrote:

> On Mon, Mar 4, 2013 at 4:34 PM, Brett Cannon <brett at python.org> wrote:
> >
> >
> >
> > On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw <barry at python.org> wrote:
> >>
> >> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
> >>
> >> >It is of course possible for subunit and related tools to run their
> >> >own implementation, but it seems ideal to me to have a common API
> >> >which regular unittest, nose, py.test and others can all agree on and
> >> >use : better reuse for pretty printers, GUI displays and the like
> >> >depend on some common API.
> >>
> >> And One True Way of invoking and/or discovering how to invoke, a
> package's
> >> test suite.
> >
> >
> > How does unittest's test discovery not solve that?
>
> It is not always obvious how to test a package when one is not
> familiar with it. Are the tests in pkgname/tests or tests or ... ?
>

I would argue that's a packaging problem and not a testing infrastructure
in the stdlib problem. If we want to standardize on always having the tests
in a 'tests' sub-package that's fine, but I don't see unittest or subtest
directly controlling that short of some registration hook that has to be
called to declare where the tests are.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/f044e747/attachment.html>

From barry at python.org  Mon Mar  4 17:51:04 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 11:51:04 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
Message-ID: <20130304115104.224493ef@anarchist.wooz.org>

On Mar 04, 2013, at 11:34 AM, Brett Cannon wrote:

>> And One True Way of invoking and/or discovering how to invoke, a package's
>> test suite.
>
>How does unittest's test discovery not solve that?

I should have added "from the command line".  E.g. is it:

$ python -m unittest discover
$ python setup.py test
$ python setup.py nosetests
$ python -m nose test
$ nosetests-X.Y

Besides having a multitude of choices, there's almost no way to automatically
discover (e.g. by metadata inspection or some such) how to invoke the tests.
You're often lucky if there's a README.test and it's still accurate.

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/4c353f64/attachment.pgp>

From skip at pobox.com  Mon Mar  4 18:02:04 2013
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 4 Mar 2013 11:02:04 -0600
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
Message-ID: <CANc-5UzLrnoz7m2y-+PW4TZpjRvhrHgQQdvRJUHVF2jqPpW0Ow@mail.gmail.com>

On Mon, Mar 4, 2013 at 10:30 AM, Brian Curtin <brian at python.org> wrote:

> The full announcement is at
> http://blog.python.org/2013/03/introducing-electronic-contributor.html,
> but a summary follows.
> ...


Brian,

Do you want old-timers like me who have a wet-signed fax gathering dust in
a  box at PSF World Headquarters to execute the electronic contributor
agreement?  While not strictly necessary, I suspect it might be nice for
you to have all agreements in a common form.

Skip
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/9fcc6d30/attachment.html>

From brian at python.org  Mon Mar  4 18:04:25 2013
From: brian at python.org (Brian Curtin)
Date: Mon, 4 Mar 2013 11:04:25 -0600
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CANc-5UzLrnoz7m2y-+PW4TZpjRvhrHgQQdvRJUHVF2jqPpW0Ow@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CANc-5UzLrnoz7m2y-+PW4TZpjRvhrHgQQdvRJUHVF2jqPpW0Ow@mail.gmail.com>
Message-ID: <CAD+XWwqZDSuxz3a2fgSwWkzjQ-RVvmYURcebboi51qQBWvjC2w@mail.gmail.com>

On Mon, Mar 4, 2013 at 11:02 AM, Skip Montanaro <skip at pobox.com> wrote:
>
>
> On Mon, Mar 4, 2013 at 10:30 AM, Brian Curtin <brian at python.org> wrote:
>>
>> The full announcement is at
>> http://blog.python.org/2013/03/introducing-electronic-contributor.html,
>> but a summary follows.
>> ...
>
>
> Brian,
>
> Do you want old-timers like me who have a wet-signed fax gathering dust in a
> box at PSF World Headquarters to execute the electronic contributor
> agreement?  While not strictly necessary, I suspect it might be nice for you
> to have all agreements in a common form.

I'll check on that, but I don't think it's necessary since the
gathered data is no different.

From solipsis at pitrou.net  Mon Mar  4 19:41:48 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 4 Mar 2013 19:41:48 +0100
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
Message-ID: <20130304194148.7e6b42f5@pitrou.net>

On Mon, 4 Mar 2013 11:51:04 -0500
Barry Warsaw <barry at python.org> wrote:
> On Mar 04, 2013, at 11:34 AM, Brett Cannon wrote:
> 
> >> And One True Way of invoking and/or discovering how to invoke, a package's
> >> test suite.
> >
> >How does unittest's test discovery not solve that?
> 
> I should have added "from the command line".  E.g. is it:
> 
> $ python -m unittest discover
> $ python setup.py test
> $ python setup.py nosetests
> $ python -m nose test
> $ nosetests-X.Y
> 
> Besides having a multitude of choices, there's almost no way to automatically
> discover (e.g. by metadata inspection or some such) how to invoke the tests.
> You're often lucky if there's a README.test and it's still accurate.

I hope we can have a "pytest" utility that does the right thing in
3.4 :-)
Typing "python -m unittest discover" is too cumbersome.

Regards

Antoine.



From amauryfa at gmail.com  Mon Mar  4 20:06:31 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Mar 2013 20:06:31 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <kh2fa6$lg2$1@ger.gmane.org>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<kh2fa6$lg2$1@ger.gmane.org>
Message-ID: <CAGmFidbpEjZ6cgk=6jQ8MGeTHx41tDDM1pYxSmZv4-XR5wrggA@mail.gmail.com>

2013/3/4 Serhiy Storchaka <storchaka at gmail.com>

> On 01.03.13 17:24, Stefan Bucur wrote:
>
>> Before digging deeper into the issue, I wanted to ask here if there are
>> any implicit assumptions about string identity and interning throughout
>> the interpreter implementation. For instance, are two single-char
>> strings having the same content supposed to be identical objects?
>>
>
> I think this is not a bug if the code relies on the fact that an empty
> string is a singleton. This obviously is an immutable object and there is
> no public method to create different empty string.


Really?

>>> x = u'\xe9'.encode('ascii', 'ignore')
>>> x == '', x is ''
(True, False)


-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/d703ebcc/attachment.html>

From fijall at gmail.com  Mon Mar  4 20:08:50 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 4 Mar 2013 21:08:50 +0200
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304194148.7e6b42f5@pitrou.net>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
Message-ID: <CAK5idxRzbeCFBJ10KyVpAn1cJ0c_N_ugU1Jts-=qt7SffHsp5Q@mail.gmail.com>

On Mon, Mar 4, 2013 at 8:41 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 4 Mar 2013 11:51:04 -0500
> Barry Warsaw <barry at python.org> wrote:
>> On Mar 04, 2013, at 11:34 AM, Brett Cannon wrote:
>>
>> >> And One True Way of invoking and/or discovering how to invoke, a package's
>> >> test suite.
>> >
>> >How does unittest's test discovery not solve that?
>>
>> I should have added "from the command line".  E.g. is it:
>>
>> $ python -m unittest discover
>> $ python setup.py test
>> $ python setup.py nosetests
>> $ python -m nose test
>> $ nosetests-X.Y
>>
>> Besides having a multitude of choices, there's almost no way to automatically
>> discover (e.g. by metadata inspection or some such) how to invoke the tests.
>> You're often lucky if there's a README.test and it's still accurate.
>
> I hope we can have a "pytest" utility that does the right thing in
> 3.4 :-)
> Typing "python -m unittest discover" is too cumbersome.
>
> Regards
>
> Antoine.

Please pick a different name though, pytest is already widely used.

From guido at python.org  Mon Mar  4 20:11:54 2013
From: guido at python.org (Guido van Rossum)
Date: Mon, 4 Mar 2013 11:11:54 -0800
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CAGmFidbpEjZ6cgk=6jQ8MGeTHx41tDDM1pYxSmZv4-XR5wrggA@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<kh2fa6$lg2$1@ger.gmane.org>
	<CAGmFidbpEjZ6cgk=6jQ8MGeTHx41tDDM1pYxSmZv4-XR5wrggA@mail.gmail.com>
Message-ID: <CAP7+vJJytUOH5au9cFbhGN5h+e313_NiAErhtiVXfj_kYkj1Zw@mail.gmail.com>

On Mon, Mar 4, 2013 at 11:06 AM, Amaury Forgeot d'Arc
<amauryfa at gmail.com> wrote:
>
>
> 2013/3/4 Serhiy Storchaka <storchaka at gmail.com>
>>
>> On 01.03.13 17:24, Stefan Bucur wrote:
>>>
>>> Before digging deeper into the issue, I wanted to ask here if there are
>>> any implicit assumptions about string identity and interning throughout
>>> the interpreter implementation. For instance, are two single-char
>>> strings having the same content supposed to be identical objects?
>>
>>
>> I think this is not a bug if the code relies on the fact that an empty
>> string is a singleton. This obviously is an immutable object and there is no
>> public method to create different empty string.
>
>
> Really?
>
>>>> x = u'\xe9'.encode('ascii', 'ignore')
>>>> x == '', x is ''
> (True, False)

Code that relies on this is incorrect (the language doesn't guarantee
interning) but nevertheless given the intention of the implementation,
that behavior of encode() is also a bug.

-- 
--Guido van Rossum (python.org/~guido)

From barry at python.org  Mon Mar  4 20:14:57 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 14:14:57 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304194148.7e6b42f5@pitrou.net>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
Message-ID: <20130304141457.603b4965@anarchist.wooz.org>

On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:

>> $ python -m unittest discover
>> $ python setup.py test
>> $ python setup.py nosetests
>> $ python -m nose test
>> $ nosetests-X.Y
>> 
>> Besides having a multitude of choices, there's almost no way to
>> automatically discover (e.g. by metadata inspection or some such) how to
>> invoke the tests.  You're often lucky if there's a README.test and it's
>> still accurate.
>
>I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
>Typing "python -m unittest discover" is too cumbersome.

Where is this work being done (e.g. is there a PEP)?

One thing to keep in mind is how to invoke this on a system with multiple
versions of Python available.  For example, in Debian, a decision was recently
made to drop all the nosetests-X.Y scripts from /usr/bin[1].

This makes sense when you think about having at least two major versions of
Python (2.x and 3.x) and maybe up to four (2.6, 2.7, 3.2, 3.3), *plus* debug
versions of each.  Add to that, we don't actually know at package build time
which versions of Python you might have installed on your system.

A suggestion was made to provide a main entry point so that `pythonX.Y -m
nose` would work, which makes sense to me and was adopted by the
nose-devs[2].

So while a top level `pytest` command may make sense, it also might not ;).
While PEP 426 has a way to declare test dependencies (a good thing), it seems
to have no way to declare how to actually run the tests.

Cheers,
-Barry

[1] Start of thread:
    http://comments.gmane.org/gmane.linux.debian.devel.python/8572

[2] https://github.com/nose-devs/nose/issues/634

From amauryfa at gmail.com  Mon Mar  4 20:20:00 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Mon, 4 Mar 2013 20:20:00 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CAP7+vJJytUOH5au9cFbhGN5h+e313_NiAErhtiVXfj_kYkj1Zw@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<kh2fa6$lg2$1@ger.gmane.org>
	<CAGmFidbpEjZ6cgk=6jQ8MGeTHx41tDDM1pYxSmZv4-XR5wrggA@mail.gmail.com>
	<CAP7+vJJytUOH5au9cFbhGN5h+e313_NiAErhtiVXfj_kYkj1Zw@mail.gmail.com>
Message-ID: <CAGmFidbOB39-JFHKgLSxR_5MdP59TN+kJNzkf91mYsTSNUwU0w@mail.gmail.com>

2013/3/4 Guido van Rossum <guido at python.org>

> >>>> x = u'\xe9'.encode('ascii', 'ignore')
> >>>> x == '', x is ''
> > (True, False)
>
> Code that relies on this is incorrect (the language doesn't guarantee
> interning) but nevertheless given the intention of the implementation,
> that behavior of encode() is also a bug.
>

The example above is obviously from python2.7; there is a similar example
with python3.2:
>>> x = b'\xe9\xe9'.decode('ascii', 'ignore')
>>> x == '', x is ''
(True, False)

...but this bug has been fixed in 3.3: PyUnicode_Resize() always returns
the unicode_empty singleton.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/9bf5a2d8/attachment.html>

From berker.peksag at gmail.com  Mon Mar  4 20:26:18 2013
From: berker.peksag at gmail.com (=?UTF-8?Q?Berker_Peksa=C4=9F?=)
Date: Mon, 4 Mar 2013 21:26:18 +0200
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304141457.603b4965@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
Message-ID: <CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>

On Mon, Mar 4, 2013 at 9:14 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>
>>> $ python -m unittest discover
>>> $ python setup.py test
>>> $ python setup.py nosetests
>>> $ python -m nose test
>>> $ nosetests-X.Y
>>>
>>> Besides having a multitude of choices, there's almost no way to
>>> automatically discover (e.g. by metadata inspection or some such) how to
>>> invoke the tests.  You're often lucky if there's a README.test and it's
>>> still accurate.
>>
>>I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
>>Typing "python -m unittest discover" is too cumbersome.
>
> Where is this work being done (e.g. is there a PEP)?

There is an open issue on the tracker: http://bugs.python.org/issue14266

--Berker

From benno at benno.id.au  Mon Mar  4 20:31:33 2013
From: benno at benno.id.au (Ben Leslie)
Date: Tue, 5 Mar 2013 06:31:33 +1100
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
Message-ID: <CABZ0LtDiMN20fht_CH5PZzd+73vT+Metr28u82ke1iHTxSbcRg@mail.gmail.com>

On Tue, Mar 5, 2013 at 3:30 AM, Brian Curtin <brian at python.org> wrote:

> The full announcement is at
> http://blog.python.org/2013/03/introducing-electronic-contributor.html,
> but a summary follows.
>
> We've now moved to an electronic Contributor License Agreement form at
> http://www.python.org/psf/contrib/contrib-form/ which will hopefully
> ease the signing and sending of forms for our potential contributors.
> The form shows the required fields whether you're signing as an
> individual or a representative of an organization, and removes the
> need to print, scan, fax, etc.
>
> When a new contributor fills in the form, they are emailed a copy of
> the form and asked to confirm the email address that they used (and
> received that copy at). Upon confirming, the signed form is sent to
> the PSF Administrator and filed away.
>
> The signature can either be generated from your typed name, or you can
> draw or upload your actual written signature if you choose.
>

I had been procrastinating on filling in the paper version, but having this
means no excuse.

The process was very simple and straight forward. (The only difficult part
was actually working out my python bugs username).

Thanks for taking the administrative effort to get this all in place.

Cheers,

Benno
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130305/bd240c45/attachment.html>

From robertc at robertcollins.net  Mon Mar  4 21:00:38 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 09:00:38 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
Message-ID: <CAJ3HoZ1iVOhYA=_RT-T_8GZybyRtxUJ6eBs3XXeXag+Pw+E-cw@mail.gmail.com>

On 5 March 2013 05:34, Brett Cannon <brett at python.org> wrote:
>
>
>
> On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw <barry at python.org> wrote:
>>
>> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
>>
>> >It is of course possible for subunit and related tools to run their
>> >own implementation, but it seems ideal to me to have a common API
>> >which regular unittest, nose, py.test and others can all agree on and
>> >use : better reuse for pretty printers, GUI displays and the like
>> >depend on some common API.
>>
>> And One True Way of invoking and/or discovering how to invoke, a package's
>> test suite.
>
>
> How does unittest's test discovery not solve that?

Three reasons
 a) There are some bugs (all filed I think) - I intend to hack on
these in the near future - that prevent discovery working at all for
some use cases.
 b) discovery requires magic parameters that are project specific
(e.g. is it 'discover .' or 'discover . lib' to run it). This is
arguably a setup.py/packaging entrypoint issue.
 c) Test suites written for e.g. Twisted, or nose, or other
non-stdunit-runner-compatible test runners will fail to execute even
when discovered correctly.

There are ways to solve this without addressing a/b/c - just defining
a standard command to run that signals success/failure with it's exit
code. Packages can export a particular flavour of that in their
setup.py if they have exceptional needs, and do nothing in the common
case. That doesn't solve 'how to introspect a package test suite' but
for distro packagers - and large scale CI integration - that doesn't
matter.

For instance testrepository offers a setuptools extension to let it be
used trivially, I believe nose does something similar.

Having something that would let *any* test suite spit out folk's
favourite test protocol de jour would be brilliant of course :).
[junit-xml, subunit, TAP, ...]

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From dholth at gmail.com  Mon Mar  4 21:02:35 2013
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 4 Mar 2013 15:02:35 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
Message-ID: <CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>

On Mon, Mar 4, 2013 at 2:26 PM, Berker Peksa? <berker.peksag at gmail.com> wrote:
> On Mon, Mar 4, 2013 at 9:14 PM, Barry Warsaw <barry at python.org> wrote:
>> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>>
>>>> $ python -m unittest discover
>>>> $ python setup.py test
>>>> $ python setup.py nosetests
>>>> $ python -m nose test
>>>> $ nosetests-X.Y
>>>>
>>>> Besides having a multitude of choices, there's almost no way to
>>>> automatically discover (e.g. by metadata inspection or some such) how to
>>>> invoke the tests.  You're often lucky if there's a README.test and it's
>>>> still accurate.
>>>
>>>I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
>>>Typing "python -m unittest discover" is too cumbersome.
>>
>> Where is this work being done (e.g. is there a PEP)?
>
> There is an open issue on the tracker: http://bugs.python.org/issue14266
>
> --Berker

setup.py's setup(test_suite="x")... not sure if this is a distutils or
setuptools feature. PEP 426 has an extension mechanism that could do
the job.

From robertc at robertcollins.net  Mon Mar  4 21:07:15 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 09:07:15 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304115104.224493ef@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
Message-ID: <CAJ3HoZ1jU7J3gYppKkMTCeDEMqv7w9Mav=+avFQUXqNamS_x5g@mail.gmail.com>

On 5 March 2013 05:51, Barry Warsaw <barry at python.org> wrote:
> I should have added "from the command line".  E.g. is it:
>
> $ python -m unittest discover
> $ python setup.py test
> $ python setup.py nosetests
> $ python -m nose test
> $ nosetests-X.Y

$ testr run

:)

> Besides having a multitude of choices, there's almost no way to automatically
> discover (e.g. by metadata inspection or some such) how to invoke the tests.
> You're often lucky if there's a README.test and it's still accurate.

<pimping own horse>
If there is a .testr.conf you can run 'testr init; testr run'. Thats
the defined entry point for testr, and .testr.conf can specify running
make, or setup.py build or whatever else is needed to run tests.
</pimp>

I would love to see a declaritive interface so that you can tell that
is what you should run.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From barry at python.org  Mon Mar  4 21:14:39 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 15:14:39 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
Message-ID: <20130304151439.611dd9aa@anarchist.wooz.org>

On Mar 04, 2013, at 03:02 PM, Daniel Holth wrote:

>setup.py's setup(test_suite="x")... not sure if this is a distutils or
>setuptools feature. PEP 426 has an extension mechanism that could do
>the job.

Shouldn't "testing" be a first order feature?

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/ce69e590/attachment-0001.pgp>

From dholth at gmail.com  Mon Mar  4 21:40:43 2013
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 4 Mar 2013 15:40:43 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304151439.611dd9aa@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
	<20130304151439.611dd9aa@anarchist.wooz.org>
Message-ID: <CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>

On Mon, Mar 4, 2013 at 3:14 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 04, 2013, at 03:02 PM, Daniel Holth wrote:
>
>>setup.py's setup(test_suite="x")... not sure if this is a distutils or
>>setuptools feature. PEP 426 has an extension mechanism that could do
>>the job.
>
> Shouldn't "testing" be a first order feature?

Unfortunately there are so many potential first-order features that
we've had to leave some out in order to save time. "How to run the
tests" is not something that you need to know when searching PyPI for
a distribution and its dependencies.

From barry at python.org  Mon Mar  4 21:45:05 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 15:45:05 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
	<20130304151439.611dd9aa@anarchist.wooz.org>
	<CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>
Message-ID: <20130304154505.016092cf@anarchist.wooz.org>

On Mar 04, 2013, at 03:40 PM, Daniel Holth wrote:

>On Mon, Mar 4, 2013 at 3:14 PM, Barry Warsaw <barry at python.org> wrote:
>> On Mar 04, 2013, at 03:02 PM, Daniel Holth wrote:
>>
>>>setup.py's setup(test_suite="x")... not sure if this is a distutils or
>>>setuptools feature. PEP 426 has an extension mechanism that could do
>>>the job.
>>
>> Shouldn't "testing" be a first order feature?
>
>Unfortunately there are so many potential first-order features that
>we've had to leave some out in order to save time. "How to run the
>tests" is not something that you need to know when searching PyPI for
>a distribution and its dependencies.

Although "has unittests that I can run" might be a deciding factor of which of
the many implementations of a particular feature you might choose.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/c2c15afd/attachment.pgp>

From tjreedy at udel.edu  Mon Mar  4 21:46:48 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 04 Mar 2013 15:46:48 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
Message-ID: <kh317n$ifo$1@ger.gmane.org>

On 3/4/2013 11:36 AM, Brett Cannon wrote:
>
>
>
> On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin <brian at python.org
> <mailto:brian at python.org>> wrote:
>
>     The full announcement is at
>     http://blog.python.org/2013/03/introducing-electronic-contributor.html,
>     but a summary follows.
>
>     We've now moved to an electronic Contributor License Agreement form at
>     http://www.python.org/psf/contrib/contrib-form/ which will hopefully
>     ease the signing and sending of forms for our potential contributors.
>     The form shows the required fields whether you're signing as an
>     individual or a representative of an organization, and removes the
>     need to print, scan, fax, etc.
>
>     When a new contributor fills in the form, they are emailed a copy of
>     the form and asked to confirm the email address that they used (and
>     received that copy at). Upon confirming, the signed form is sent to
>     the PSF Administrator and filed away.
>
>     The signature can either be generated from your typed name, or you can
>     draw or upload your actual written signature if you choose.
>
>
> With this in place I would like to propose that all patches submitted to
> bugs.python.org <http://bugs.python.org> must come from someone who has
> signed the CLA before we consider committing it (if you want to be truly
> paranoid we could say that we won't even look at the code w/o a CLA).

Either policy could be facilitated by tracker changes. In order to see 
the file upload box, one must login and the tracker knows who has a CLA 
on file (as indicated by a * suffix on the name). If a file is uploaded 
by someone without, a box could popup with the link to the e-form and a 
message that a CLA is required.

-- 
Terry Jan Reedy


From victor.stinner at gmail.com  Mon Mar  4 21:46:49 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 4 Mar 2013 21:46:49 +0100
Subject: [Python-Dev] Disabling string interning for null and
 single-char causes segfaults
In-Reply-To: <CAGmFidbOB39-JFHKgLSxR_5MdP59TN+kJNzkf91mYsTSNUwU0w@mail.gmail.com>
References: <CAEg840WBPjNM9Kpv6gha7JBYsea3tbb6j4vwKvKbTtDCUgZavQ@mail.gmail.com>
	<kh2fa6$lg2$1@ger.gmane.org>
	<CAGmFidbpEjZ6cgk=6jQ8MGeTHx41tDDM1pYxSmZv4-XR5wrggA@mail.gmail.com>
	<CAP7+vJJytUOH5au9cFbhGN5h+e313_NiAErhtiVXfj_kYkj1Zw@mail.gmail.com>
	<CAGmFidbOB39-JFHKgLSxR_5MdP59TN+kJNzkf91mYsTSNUwU0w@mail.gmail.com>
Message-ID: <CAMpsgwYc915JHi9LUCYggrtVmKBQgg4t7gs2v=_wnfATh=xf9w@mail.gmail.com>

Hi,

2013/3/4 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
> The example above is obviously from python2.7; there is a similar example
> with python3.2:
>>>> x = b'\xe9\xe9'.decode('ascii', 'ignore')
>>>> x == '', x is ''
> (True, False)
>
> ...but this bug has been fixed in 3.3: PyUnicode_Resize() always returns the
> unicode_empty singleton.

Yeah, I tried to reuse singletons (empty string and latin-1 single
letters) as much as possible to reduce memory footprint, not to ensure
that an empty string is always the '' singleton.

I wouldn't call this a bug.

Victor

From solipsis at pitrou.net  Mon Mar  4 21:46:24 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 4 Mar 2013 21:46:24 +0100
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org>
Message-ID: <20130304214624.0f6dec8c@pitrou.net>

On Mon, 04 Mar 2013 15:46:48 -0500
Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/4/2013 11:36 AM, Brett Cannon wrote:
> >
> >
> >
> > On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin <brian at python.org
> > <mailto:brian at python.org>> wrote:
> >
> >     The full announcement is at
> >     http://blog.python.org/2013/03/introducing-electronic-contributor.html,
> >     but a summary follows.
> >
> >     We've now moved to an electronic Contributor License Agreement form at
> >     http://www.python.org/psf/contrib/contrib-form/ which will hopefully
> >     ease the signing and sending of forms for our potential contributors.
> >     The form shows the required fields whether you're signing as an
> >     individual or a representative of an organization, and removes the
> >     need to print, scan, fax, etc.
> >
> >     When a new contributor fills in the form, they are emailed a copy of
> >     the form and asked to confirm the email address that they used (and
> >     received that copy at). Upon confirming, the signed form is sent to
> >     the PSF Administrator and filed away.
> >
> >     The signature can either be generated from your typed name, or you can
> >     draw or upload your actual written signature if you choose.
> >
> >
> > With this in place I would like to propose that all patches submitted to
> > bugs.python.org <http://bugs.python.org> must come from someone who has
> > signed the CLA before we consider committing it (if you want to be truly
> > paranoid we could say that we won't even look at the code w/o a CLA).
> 
> Either policy could be facilitated by tracker changes. In order to see 
> the file upload box, one must login and the tracker knows who has a CLA 
> on file (as indicated by a * suffix on the name). If a file is uploaded 
> by someone without, a box could popup with the link to the e-form and a 
> message that a CLA is required.

And how about people who upload something else than a patch?

Regards

Antoine.



From pje at telecommunity.com  Mon Mar  4 21:52:14 2013
From: pje at telecommunity.com (PJ Eby)
Date: Mon, 4 Mar 2013 15:52:14 -0500
Subject: [Python-Dev] Planning on removing cache invalidation for file
	finders
In-Reply-To: <CAP1=2W5TzOM5-kr8onoH=1ExRAU7dW1e32Pn7pHKWTe9Rhv0Fg@mail.gmail.com>
References: <CAP1=2W48FLHFYeakvAx6yHpnQGs_a1K-s_V5-Vq2f_simAOD1g@mail.gmail.com>
	<CADiSq7fwWN_=ufnB2qoGSECpTf5XTrPE5FRp62yYqcBvgxM_fQ@mail.gmail.com>
	<CAP1=2W6JdinOam+AN_JR6y0eeDr95sdOVeSgj9zmd=58cd3YaQ@mail.gmail.com>
	<20130303021630.3f5340d5@pitrou.net>
	<CAP1=2W4VEd1o2LvqbE=ZTbKyptg8k3ZinU76xkxao0htEPBE9A@mail.gmail.com>
	<CAP1=2W5TzOM5-kr8onoH=1ExRAU7dW1e32Pn7pHKWTe9Rhv0Fg@mail.gmail.com>
Message-ID: <CALeMXf7XaLwn4TXTtNyzm0qTTnLZ8Qus=KrxgB1hGWr2cdK55A@mail.gmail.com>

On Sun, Mar 3, 2013 at 12:31 PM, Brett Cannon <brett at python.org> wrote:
> But how about this as a compromise over introducing write_module():
> invalidate_caches() can take a path for something to specifically
> invalidate. The path can then be passed to the invalidate_caches() on
> sys.meta_path. In the case of PathFinder it would take that path, try to
> find the directory in sys.path_importer_cache, and then invalidate the most
> specific finder for that path (if there is one that has any directory prefix
> match).
>
> Lots of little details to specify (e.g. absolute path forced anywhere in
> case a relative path is passed in by sys.path is all absolute paths? How do
> we know something is a file if it has not been written yet?), but this would
> prevent importlib from subsuming file writing specifically for source files
> and minimize performance overhead of invalidating all caches for a single
> file.

ISTR that when we were first discussing caching, I'd proposed a
TTL-based workaround for the timestamp granularity problem, and it was
mooted because somebody already proposed and implemented a similar
idea.  But my approach -- or at least the one I have in mind now --
would provide an "eventual consistency" guarantee, while still
allowing fast startup times.

However I think the experience with this heuristic so far shows that
the real problem isn't that the heuristic doesn't work for the normal
case; it works fine for that.  Instead, what happens is that *it
doesn't work when you generate modules*.

And *that* problem can be fixed without even invalidating the caches:
it can be fixed by doing some extra work when writing a module - e.g.
by making sure the directory mtime changes again after the module is
written.

For example, create the module under a temporary name, verify that the
directory mtime is different than it was before, then keep renaming it
to different temporary names until the mtime changes again, then
rename it to the final name.  (This would be very fast on some
platforms, much slower on others, but the OS itself would tell you
when it had worked.)  A utility routine to "write_module()" or
"write_package()" would be easier to find than advice that says to
invalidate the cache under thus-and-such conditions, as it would show
up in searches for writing modules dynamically or creating modules
dynamically, where you could only search for info about the cache if
you knew the cache existed.

From eliben at gmail.com  Mon Mar  4 22:26:57 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Mon, 4 Mar 2013 13:26:57 -0800
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
Message-ID: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>

[Splitting into a separate thread]

Do we really need to overthink something that requires a trivial alias to
set up for one's own convenience?

Picking a Python version (as Barry mentions) is just one of the problems.
What's wrong with:

alias rupytests='python3 -m unittest discover"
alias runpytests2='python2 -m unittest discover"

?

Don't get me wrong, I love the "discover" option and agree that it should
be the recommended way to go - but isn't this largely a documentation issue?

Eli






On Mon, Mar 4, 2013 at 11:14 AM, Barry Warsaw <barry at python.org> wrote:

> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>
> >> $ python -m unittest discover
> >> $ python setup.py test
> >> $ python setup.py nosetests
> >> $ python -m nose test
> >> $ nosetests-X.Y
> >>
> >> Besides having a multitude of choices, there's almost no way to
> >> automatically discover (e.g. by metadata inspection or some such) how to
> >> invoke the tests.  You're often lucky if there's a README.test and it's
> >> still accurate.
> >
> >I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
> >Typing "python -m unittest discover" is too cumbersome.
>
> Where is this work being done (e.g. is there a PEP)?
>
> One thing to keep in mind is how to invoke this on a system with multiple
> versions of Python available.  For example, in Debian, a decision was
> recently
> made to drop all the nosetests-X.Y scripts from /usr/bin[1].
>
> This makes sense when you think about having at least two major versions of
> Python (2.x and 3.x) and maybe up to four (2.6, 2.7, 3.2, 3.3), *plus*
> debug
> versions of each.  Add to that, we don't actually know at package build
> time
> which versions of Python you might have installed on your system.
>
> A suggestion was made to provide a main entry point so that `pythonX.Y -m
> nose` would work, which makes sense to me and was adopted by the
> nose-devs[2].
>
> So while a top level `pytest` command may make sense, it also might not ;).
> While PEP 426 has a way to declare test dependencies (a good thing), it
> seems
> to have no way to declare how to actually run the tests.
>
> Cheers,
> -Barry
>
> [1] Start of thread:
>     http://comments.gmane.org/gmane.linux.debian.devel.python/8572
>
> [2] https://github.com/nose-devs/nose/issues/634
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/eliben%40gmail.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/98629198/attachment.html>

From breamoreboy at yahoo.co.uk  Mon Mar  4 22:33:55 2013
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Mon, 04 Mar 2013 21:33:55 +0000
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh317n$ifo$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org>
Message-ID: <kh33sr$b2k$1@ger.gmane.org>

On 04/03/2013 20:46, Terry Reedy wrote:
> On 3/4/2013 11:36 AM, Brett Cannon wrote:
>>
>>
>>
>> On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin <brian at python.org
>> <mailto:brian at python.org>> wrote:
>>
>>     The full announcement is at
>>
>> http://blog.python.org/2013/03/introducing-electronic-contributor.html,
>>     but a summary follows.
>>
>>     We've now moved to an electronic Contributor License Agreement
>> form at
>>     http://www.python.org/psf/contrib/contrib-form/ which will hopefully
>>     ease the signing and sending of forms for our potential contributors.
>>     The form shows the required fields whether you're signing as an
>>     individual or a representative of an organization, and removes the
>>     need to print, scan, fax, etc.
>>
>>     When a new contributor fills in the form, they are emailed a copy of
>>     the form and asked to confirm the email address that they used (and
>>     received that copy at). Upon confirming, the signed form is sent to
>>     the PSF Administrator and filed away.
>>
>>     The signature can either be generated from your typed name, or you
>> can
>>     draw or upload your actual written signature if you choose.
>>
>>
>> With this in place I would like to propose that all patches submitted to
>> bugs.python.org <http://bugs.python.org> must come from someone who has
>> signed the CLA before we consider committing it (if you want to be truly
>> paranoid we could say that we won't even look at the code w/o a CLA).
>
> Either policy could be facilitated by tracker changes. In order to see
> the file upload box, one must login and the tracker knows who has a CLA
> on file (as indicated by a * suffix on the name). If a file is uploaded
> by someone without, a box could popup with the link to the e-form and a
> message that a CLA is required.
>

People already use the bug tracker as an excuse not to contribute, 
wouldn't this requirement make the situation worse?

-- 
Cheers.

Mark Lawrence


From solipsis at pitrou.net  Mon Mar  4 22:28:48 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 4 Mar 2013 22:28:48 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
Message-ID: <20130304222848.5006bf3b@pitrou.net>

On Mon, 4 Mar 2013 13:26:57 -0800
Eli Bendersky <eliben at gmail.com> wrote:
> [Splitting into a separate thread]
> 
> Do we really need to overthink something that requires a trivial alias to
> set up for one's own convenience?
> 
> Picking a Python version (as Barry mentions) is just one of the problems.
> What's wrong with:
> 
> alias rupytests='python3 -m unittest discover"
> alias runpytests2='python2 -m unittest discover"
> 
> ?
> 
> Don't get me wrong, I love the "discover" option and agree that it should
> be the recommended way to go - but isn't this largely a documentation issue?

I would personally call it a typing issue :-) "python -m unittest
discover" is just too long.

Regards

Antoine.



From robertc at robertcollins.net  Mon Mar  4 23:01:38 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 11:01:38 +1300
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
Message-ID: <CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>

On 5 March 2013 10:26, Eli Bendersky <eliben at gmail.com> wrote:
> [Splitting into a separate thread]
>
> Do we really need to overthink something that requires a trivial alias to
> set up for one's own convenience?

The big thing is automated tools, not developers.

When distributors want to redistribute packages they want to be sure
they work. Running the tests is a pretty good signal for that, but
having every package slightly different adds to the work they need to
do. Being able to do 'setup.py test' consistently, everywhere - that
would be great.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From brett at python.org  Mon Mar  4 23:04:13 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 4 Mar 2013 17:04:13 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304154505.016092cf@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
	<20130304151439.611dd9aa@anarchist.wooz.org>
	<CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>
	<20130304154505.016092cf@anarchist.wooz.org>
Message-ID: <CAP1=2W5o_zZKe1MDyXnPnmm7YZqXmVW_Hru5+BQwks29QoPJhg@mail.gmail.com>

On Mon, Mar 4, 2013 at 3:45 PM, Barry Warsaw <barry at python.org> wrote:

> On Mar 04, 2013, at 03:40 PM, Daniel Holth wrote:
>
> >On Mon, Mar 4, 2013 at 3:14 PM, Barry Warsaw <barry at python.org> wrote:
> >> On Mar 04, 2013, at 03:02 PM, Daniel Holth wrote:
> >>
> >>>setup.py's setup(test_suite="x")... not sure if this is a distutils or
> >>>setuptools feature. PEP 426 has an extension mechanism that could do
> >>>the job.
> >>
> >> Shouldn't "testing" be a first order feature?
> >
> >Unfortunately there are so many potential first-order features that
> >we've had to leave some out in order to save time. "How to run the
> >tests" is not something that you need to know when searching PyPI for
> >a distribution and its dependencies.
>
> Although "has unittests that I can run" might be a deciding factor of
> which of
> the many implementations of a particular feature you might choose.
>

Sure, but that has nothing to do with programmatic package discovery.
That's something you will have to do as a person in making a qualitative
decision along the same lines as API design. Flipping a bit in a config
file saying "I have tests" doesn't say much beyond you flipped a bit, e.g.
no idea on coverage, quality, etc.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/40f1498d/attachment.html>

From brett at python.org  Mon Mar  4 23:08:11 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 4 Mar 2013 17:08:11 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh33sr$b2k$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
Message-ID: <CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>

On Mon, Mar 4, 2013 at 4:33 PM, Mark Lawrence <breamoreboy at yahoo.co.uk>wrote:

> On 04/03/2013 20:46, Terry Reedy wrote:
>
>> On 3/4/2013 11:36 AM, Brett Cannon wrote:
>>
>>>
>>>
>>>
>>> On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin <brian at python.org
>>> <mailto:brian at python.org>> wrote:
>>>
>>>     The full announcement is at
>>>
>>> http://blog.python.org/2013/**03/introducing-electronic-**
>>> contributor.html<http://blog.python.org/2013/03/introducing-electronic-contributor.html>
>>> ,
>>>     but a summary follows.
>>>
>>>     We've now moved to an electronic Contributor License Agreement
>>> form at
>>>     http://www.python.org/psf/**contrib/contrib-form/<http://www.python.org/psf/contrib/contrib-form/>which will hopefully
>>>     ease the signing and sending of forms for our potential contributors.
>>>     The form shows the required fields whether you're signing as an
>>>     individual or a representative of an organization, and removes the
>>>     need to print, scan, fax, etc.
>>>
>>>     When a new contributor fills in the form, they are emailed a copy of
>>>     the form and asked to confirm the email address that they used (and
>>>     received that copy at). Upon confirming, the signed form is sent to
>>>     the PSF Administrator and filed away.
>>>
>>>     The signature can either be generated from your typed name, or you
>>> can
>>>     draw or upload your actual written signature if you choose.
>>>
>>>
>>> With this in place I would like to propose that all patches submitted to
>>> bugs.python.org <http://bugs.python.org> must come from someone who has
>>> signed the CLA before we consider committing it (if you want to be truly
>>> paranoid we could say that we won't even look at the code w/o a CLA).
>>>
>>
>> Either policy could be facilitated by tracker changes. In order to see
>> the file upload box, one must login and the tracker knows who has a CLA
>> on file (as indicated by a * suffix on the name). If a file is uploaded
>> by someone without, a box could popup with the link to the e-form and a
>> message that a CLA is required.
>>
>>
> People already use the bug tracker as an excuse not to contribute,
> wouldn't this requirement make the situation worse?


Depends on your paranoia. If you're worried about accidentally lifting IP
merely by reading someone's source code, then you wouldn't want to touch
code without the CLA signed.

Now I'm not that paranoid, but I'm still not about to commit someone's code
now without the CLA signed to make sure we are legally covered for the
patch. If someone chooses not to contribute because of the CLA that's fine,
but since we have already told at least Anatoly that we won't accept
patches from him until he signs the CLA I'm not going to start acting
differently towards others. I view legally covering our ass by having
someone fill in a form is worth the potential loss of some contribution in
the grand scheme of things.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/e1638343/attachment.html>

From barry at python.org  Mon Mar  4 23:14:39 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 17:14:39 -0500
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
Message-ID: <20130304171439.7c37c840@anarchist.wooz.org>

On Mar 05, 2013, at 11:01 AM, Robert Collins wrote:

>The big thing is automated tools, not developers.

Exactly.

-Barry

From barry at python.org  Mon Mar  4 23:24:55 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 17:24:55 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAP1=2W5o_zZKe1MDyXnPnmm7YZqXmVW_Hru5+BQwks29QoPJhg@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
	<20130304151439.611dd9aa@anarchist.wooz.org>
	<CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>
	<20130304154505.016092cf@anarchist.wooz.org>
	<CAP1=2W5o_zZKe1MDyXnPnmm7YZqXmVW_Hru5+BQwks29QoPJhg@mail.gmail.com>
Message-ID: <20130304172455.5ef0aa5e@anarchist.wooz.org>

On Mar 04, 2013, at 05:04 PM, Brett Cannon wrote:

>Sure, but that has nothing to do with programmatic package discovery.
>That's something you will have to do as a person in making a qualitative
>decision along the same lines as API design. Flipping a bit in a config
>file saying "I have tests" doesn't say much beyond you flipped a bit, e.g.
>no idea on coverage, quality, etc.

What I'm looking for is something that automated tools can use to easily
discover how to run a package's tests.  I want it to be dead simple for
developers of a package to declare how their tests are to be run, and what
extra dependencies they might need.  It seems like PEP 426 only addresses the
latter.  Maybe that's fine and a different PEP is needed to describe automated
test discover, but I still think it's an important use case.

Imagine:

 * Every time you upload a package to PyPI, snakebite runs your test suite on
   a variety of Python versions and platforms.  You get a nice link to the
   Jenkins results so you and your users get a good sense of overall package
   quality.

 * You have an automated gatekeeper that will prevent commits or uploads if
   your coverage or test results get worse instead of better.

 * Distro packagers can build tools that auto-discover the tests so that they
   are run automatically when the package is built, ensuring high quality
   packages specifically targeted to those distros.

As a community, we know how important tests are, so I think our tools should
reflect that and make it easy for those tests to be expressed.  As a selfish
side-effect, I want to reduce the amount of guesswork I need to perform in
order to know how to run a package's test when I `$vcs clone` their
repository. ;)

Cheers,
-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/42f5f2b0/attachment.pgp>

From tjreedy at udel.edu  Tue Mar  5 00:23:46 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 04 Mar 2013 18:23:46 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <20130304214624.0f6dec8c@pitrou.net>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <20130304214624.0f6dec8c@pitrou.net>
Message-ID: <kh3ae1$b4g$1@ger.gmane.org>

On 3/4/2013 3:46 PM, Antoine Pitrou wrote:
> On Mon, 04 Mar 2013 15:46:48 -0500
> Terry Reedy <tjreedy at udel.edu> wrote:
>> On 3/4/2013 11:36 AM, Brett Cannon wrote:

>>> With this in place I would like to propose that all patches submitted to
>>> bugs.python.org <http://bugs.python.org> must come from someone who has
>>> signed the CLA before we consider committing it (if you want to be truly
>>> paranoid we could say that we won't even look at the code w/o a CLA).

While I regard CLAs as partly being a form of legal theater, I regard 
our participation as necessary, both to make explicit to contributors 
what should be implicit in the act of submission *and* to show to 
copyright holders a good-faith effort to not improperly incorporate 
their code.

Note: no one expected the Linux copyright challenge, nor our European 
trademark challenge, but they happened. I expect there will be more 
challenges to open source projects, perhaps some legitimate as the 
number of contributors increases.

>> Either policy could be facilitated by tracker changes. In order to see
>> the file upload box, one must login and the tracker knows who has a CLA
>> on file (as indicated by a * suffix on the name). If a file is uploaded
>> by someone without, a box could popup with the link to the e-form and a
>> message that a CLA is required.
>
> And how about people who upload something else than a patch?

Limit the popup to files with .diff or .patch extension. Reviewers can 
check for '*' for the occasionally patch lacking that.

-- 
Terry Jan Reedy


From eliben at gmail.com  Tue Mar  5 00:47:37 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Mon, 4 Mar 2013 15:47:37 -0800
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <20130304222848.5006bf3b@pitrou.net>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<20130304222848.5006bf3b@pitrou.net>
Message-ID: <CAF-Rda-28cRSiMqc_5yzNAtMk5SWUdaofuiHtT_pzpJ-_Bq+TA@mail.gmail.com>

On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 4 Mar 2013 13:26:57 -0800
> Eli Bendersky <eliben at gmail.com> wrote:
> > [Splitting into a separate thread]
> >
> > Do we really need to overthink something that requires a trivial alias to
> > set up for one's own convenience?
> >
> > Picking a Python version (as Barry mentions) is just one of the problems.
> > What's wrong with:
> >
> > alias rupytests='python3 -m unittest discover"
> > alias runpytests2='python2 -m unittest discover"
> >
> > ?
> >
> > Don't get me wrong, I love the "discover" option and agree that it should
> > be the recommended way to go - but isn't this largely a documentation
> issue?
>
> I would personally call it a typing issue :-) "python -m unittest
> discover" is just too long.
>

Command-line options for advanced capabilities can get long, yes. It's not
a reason to add an extra layer, which is extra complexity, IMHO. The user
is free to create his own shortcuts if this is too much typing. Moreover,
many projects already have a way to run "all tests" from their root
directory.

As a case in point, we also have the useful:

$ python -m SimpleHTTPServer

So why not create a new "pyserve" command to reduce the amount of typing?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/cabd57a8/attachment.html>

From eliben at gmail.com  Tue Mar  5 00:49:50 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Mon, 4 Mar 2013 15:49:50 -0800
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <20130304171439.7c37c840@anarchist.wooz.org>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
Message-ID: <CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>

On Mon, Mar 4, 2013 at 2:14 PM, Barry Warsaw <barry at python.org> wrote:

> On Mar 05, 2013, at 11:01 AM, Robert Collins wrote:
>
> >The big thing is automated tools, not developers.
>
> Exactly.
>

I don't understand. Is "python -m unittest discover" too much typing for
automatic tools? If anything, it's much more portable across Python
versions since any new coommand/script won't be added before 3.4, while the
longer version works in 2.7 and 3.2+

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/cee86b14/attachment.html>

From robertc at robertcollins.net  Tue Mar  5 01:09:39 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 13:09:39 +1300
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
Message-ID: <CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>

On 5 March 2013 12:49, Eli Bendersky <eliben at gmail.com> wrote:
>
> On Mon, Mar 4, 2013 at 2:14 PM, Barry Warsaw <barry at python.org> wrote:
>>
>> On Mar 05, 2013, at 11:01 AM, Robert Collins wrote:
>>
>> >The big thing is automated tools, not developers.
>>
>> Exactly.
>
> I don't understand. Is "python -m unittest discover" too much typing for
> automatic tools? If anything, it's much more portable across Python versions
> since any new coommand/script won't be added before 3.4, while the longer
> version works in 2.7 and 3.2+

It isn't about length. It is about knowing that *that* is what to type
(and btw that exact command cannot run twisted's tests, among many
other projects tests).

Perhaps we are talking about different things. A top level script to
run tests is interesting, but orthogonal to the thing Barry was asking
for.

-Rob


-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From michael at voidspace.org.uk  Tue Mar  5 01:13:19 2013
From: michael at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:13:19 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130303012939.GC62205@snakebite.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<20130303012939.GC62205@snakebite.org>
Message-ID: <1BC5ABCC-7683-4D74-85AB-622A57E9F38D@voidspace.org.uk>


On 3 Mar 2013, at 01:29, Trent Nelson <trent at snakebite.org> wrote:

> On Wed, Feb 27, 2013 at 08:51:16AM -0800, Michael Foord wrote:
>> If you have other items you'd like to discuss please let me know and I
>> can add them to the agenda.
> 
>    Hmm, seems like this might be a good forum to introduce the
>    parallel/async stuff I've been working on the past few months.
>    TL;DR version is I've come up with an alternative approach for
>    exploiting multiple cores that doesn't rely on GIL-removal or
>    STM (and has a negligible performance overhead when executing
>    single-threaded code).  (For those that are curious, it lives
>    in the px branch of the sandbox/trent repo on hg.p.o, albeit
>    in a very experimental/prototype/proof-of-concept state (i.e.
>    it's an unorganized, undocumented, uncommented hackfest); on
>    the plus side, it works.  Sort of.)
> 
>    Second suggestion: perhaps a little segment on Snakebite?  What
>    it is, what's available to committers, feedback/kvetching from
>    those who have already used it, etc.
> 

I've added both to the agenda.

>    (I forgot the format of these summits -- is there a projector?)
> 

I've asked for a projector, yes.

Michael

>        Trent.


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From tjreedy at udel.edu  Tue Mar  5 01:16:27 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon, 04 Mar 2013 19:16:27 -0500
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304172455.5ef0aa5e@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
	<20130304151439.611dd9aa@anarchist.wooz.org>
	<CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>
	<20130304154505.016092cf@anarchist.wooz.org>
	<CAP1=2W5o_zZKe1MDyXnPnmm7YZqXmVW_Hru5+BQwks29QoPJhg@mail.gmail.com>
	<20130304172455.5ef0aa5e@anarchist.wooz.org>
Message-ID: <kh3dgr$4qj$1@ger.gmane.org>

On 3/4/2013 5:24 PM, Barry Warsaw wrote:

> What I'm looking for is something that automated tools can use to easily
> discover how to run a package's tests.  I want it to be dead simple for
> developers of a package to declare how their tests are to be run, and what

I am writing a package that has tests for each module (which I so far 
run individually for each module) using a custom test framework. I am 
planning to add a function to the package to run all of them. Should I 
call it 'testall', 'test_all', 'runtests', or something else? I really 
do not care. It would be used like this.
   import xxx; xxx.testall()
Of course, this would not work with the stdlib since /lib is not a 
package that can be imported. I could put the same code in the top level 
of a module, to be run when imported (but that would not work with 
re-imports), or put the function in my test module. I am willing to 
adjust to a standard when there is one.

What I do suggest is that package developers should only have to provide 
one standard entry point that hides all package-specific details. I 
presume the side-effect spec would be error messages to sdterr. Any 
return requirements should be a simple as possible, as in all pass 
True/False, or (number run, number fail) by whatever counting method the 
package/test framework uses. (Note: my framework does not count tests, 
as I only care about failure messages, but testall could count modules 
tested and those with a failure.)

> extra dependencies they might need.  It seems like PEP 426 only addresses the
> latter.  Maybe that's fine and a different PEP is needed to describe automated
> test discover, but I still think it's an important use case.

New PEP.

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Tue Mar  5 01:20:00 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 5 Mar 2013 10:20:00 +1000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304141457.603b4965@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
Message-ID: <CADiSq7fhsv4vbthpopMMUFc5dQM6OVYoq_+AvKWoH46GALXc5w@mail.gmail.com>

On 5 Mar 2013 05:21, "Barry Warsaw" <barry at python.org> wrote:
>
> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>
> >> $ python -m unittest discover
> >> $ python setup.py test
> >> $ python setup.py nosetests
> >> $ python -m nose test
> >> $ nosetests-X.Y
> >>
> >> Besides having a multitude of choices, there's almost no way to
> >> automatically discover (e.g. by metadata inspection or some such) how
to
> >> invoke the tests.  You're often lucky if there's a README.test and it's
> >> still accurate.
> >
> >I hope we can have a "pytest" utility that does the right thing in 3.4
:-)
> >Typing "python -m unittest discover" is too cumbersome.
>
> Where is this work being done (e.g. is there a PEP)?
>
> One thing to keep in mind is how to invoke this on a system with multiple
> versions of Python available.  For example, in Debian, a decision was
recently
> made to drop all the nosetests-X.Y scripts from /usr/bin[1].
>
> This makes sense when you think about having at least two major versions
of
> Python (2.x and 3.x) and maybe up to four (2.6, 2.7, 3.2, 3.3), *plus*
debug
> versions of each.  Add to that, we don't actually know at package build
time
> which versions of Python you might have installed on your system.
>
> A suggestion was made to provide a main entry point so that `pythonX.Y -m
> nose` would work, which makes sense to me and was adopted by the
> nose-devs[2].
>
> So while a top level `pytest` command may make sense, it also might not
;).
> While PEP 426 has a way to declare test dependencies (a good thing), it
seems
> to have no way to declare how to actually run the tests.

Metadata 2.0 won't cover that, 2.1 probably will. Please give us time to
solve problems incrementally rather than trying to fix everything at once.

Regards,
Nick.

>
> Cheers,
> -Barry
>
> [1] Start of thread:
>     http://comments.gmane.org/gmane.linux.debian.devel.python/8572
>
> [2] https://github.com/nose-devs/nose/issues/634
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130305/5ca1aae4/attachment.html>

From fuzzyman at voidspace.org.uk  Tue Mar  5 01:21:49 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:21:49 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
Message-ID: <E48BC372-3B57-414B-BD43-835784AED2C6@voidspace.org.uk>


On 4 Mar 2013, at 06:26, Robert Collins <robertc at robertcollins.net> wrote:

> On 4 March 2013 18:54, Guido van Rossum <guido at python.org> wrote:
>> On Sun, Mar 3, 2013 at 9:24 PM, Robert Collins
>> <robertc at robertcollins.net> wrote:
>>> I'd like to talk about overhauling - not tweaking, overhauling - the
>>> standard library testing facilities.
>> 
>> That seems like too big a topic and too vague a description to discuss
>> usefully. Perhaps you have a specific proposal? Or at least just a use
>> case that's poorly covered?
> 
> I have both - I have a draft implementation for a new test result API
> (and forwards and backwards compat code etc), and use cases that drive
> it. I started a thread here -
> http://lists.idyll.org/pipermail/testing-in-python/2013-February/005434.html
> , with blog posts
> https://rbtcollins.wordpress.com/2013/02/14/time-to-revise-the-subunit-protocol/
> https://rbtcollins.wordpress.com/2013/02/15/more-subunit-needs/
> https://rbtcollins.wordpress.com/2013/02/19/first-experience-implementing-streamresult/
> https://rbtcollins.wordpress.com/2013/02/23/simpler-is-better/
> 
> They are focused on subunit, but much of subunit's friction has been
> due to issues encountered from the stdlibrary TestResult API - in
> particular three things:
> - the single-active-test model that the current API (or at least
> implementation) has.
> - the expectation that all test outcomes will originate from the same
> interpreter (or something with a live traceback object)
> - the inability to supply details about errors other than the exception
> 
> All of which start to bite rather deep when working on massively
> parallel test environments.
> 
> It is of course possible for subunit and related tools to run their
> own implementation, but it seems ideal to me to have a common API
> which regular unittest, nose, py.test and others can all agree on and
> use : better reuse for pretty printers, GUI displays and the like
> depend on some common API.
> 
>> TBH, your choice of words is ambiguous -- are you interested in
>> overhauling the facilities for testing *of* the standard library (i.e.
>> the 'test' package), or the testing facilities *provided by* the
>> standard library (i.e. the unittest module)?
> 
> Sorry! Testing facilities provided by the standard library. They
> should naturally facilitate testing of the standard library too.


We can certainly talk about it - although as Guido says, something specific may be easier to have a useful discussion about.

Reading through your blog articles it seemed like a whole lot of subunit context was required to understand the specific proposal you're making for the TestResult. It also *seems* like you're redesigning the TestResult for a single use case (distributed testing) with an api that looks quite "odd" for anything that isn't that use case. I'd rather see how we can make the TestResult play *better* with those requirements. That discussion probably belongs in another thread - or at the summit.

Michael

> 
> -Rob
> 
>> --
>> --Guido van Rossum (python.org/~guido)
> 
> 
> 
> -- 
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Cloud Services
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From fuzzyman at voidspace.org.uk  Tue Mar  5 01:21:58 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:21:58 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
Message-ID: <C92BB22E-7A90-4363-B82D-C0170CFFD063@voidspace.org.uk>


On 4 Mar 2013, at 19:26, Berker Peksa? <berker.peksag at gmail.com> wrote:

> On Mon, Mar 4, 2013 at 9:14 PM, Barry Warsaw <barry at python.org> wrote:
>> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>> 
>>>> $ python -m unittest discover
>>>> $ python setup.py test
>>>> $ python setup.py nosetests
>>>> $ python -m nose test
>>>> $ nosetests-X.Y
>>>> 
>>>> Besides having a multitude of choices, there's almost no way to
>>>> automatically discover (e.g. by metadata inspection or some such) how to
>>>> invoke the tests.  You're often lucky if there's a README.test and it's
>>>> still accurate.
>>> 
>>> I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
>>> Typing "python -m unittest discover" is too cumbersome.
>> 
>> Where is this work being done (e.g. is there a PEP)?
> 
> There is an open issue on the tracker: http://bugs.python.org/issue14266
> 


Indeed, and unittest2 (the backport) which has to work with Python 2.6 (where "python -m package_name" doesn't work) has "unit2" as a shortcut. So it has an advantage over the standard library version here.

I'd like to see pyunit as a short-cut for "python -m unittest discover", with a "pyunit-3.x" variant too. 

Barry objects that Linux distributions won't want to support all of these, which is frankly their problem.

Michael

> --Berker
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From robertc at robertcollins.net  Tue Mar  5 01:23:25 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 13:23:25 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <E48BC372-3B57-414B-BD43-835784AED2C6@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<E48BC372-3B57-414B-BD43-835784AED2C6@voidspace.org.uk>
Message-ID: <CAJ3HoZ3HGcW73hsm_suDrzCwSZ=LCpmH95Fx9ekGMHE6zNsa_g@mail.gmail.com>

On 5 March 2013 13:21, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>

> We can certainly talk about it - although as Guido says, something specific may be easier to have a useful discussion about.
>
> Reading through your blog articles it seemed like a whole lot of subunit context was required to understand the specific
> proposal you're making for the TestResult. It also *seems* like you're redesigning the TestResult for a single use case
> (distributed testing) with an api that looks quite "odd" for anything that isn't that use case. I'd rather see how we can
> make the TestResult play *better* with those requirements. That discussion probably belongs in another thread - or at
> the summit.

Right - all I wanted was to flag that you and I and any other
interested parties should discuss this at the summit :).

-Rob






-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From fuzzyman at voidspace.org.uk  Tue Mar  5 01:33:20 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:33:20 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAP1=2W6PpZEmjeoYSxjFLURqwPZwnmoOsXS+w5Wq3g4yhY676Q@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<kgn1dd$88j$1@ger.gmane.org>
	<F0ECF91B-5E2A-4114-9423-A7A50A7FA0F1@voidspace.org.uk>
	<CAP1=2W6PpZEmjeoYSxjFLURqwPZwnmoOsXS+w5Wq3g4yhY676Q@mail.gmail.com>
Message-ID: <E89FEE51-E5B2-4BF7-AF5D-001CE55CE9DE@voidspace.org.uk>


On 28 Feb 2013, at 13:49, Brett Cannon <brett at python.org> wrote:

> 
> 
> 
> On Thu, Feb 28, 2013 at 6:34 AM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> 
> On 28 Feb 2013, at 07:36, Georg Brandl <g.brandl at gmx.net> wrote:
> 
> > Am 27.02.2013 17:51, schrieb Michael Foord:
> >> Hello all,
> >>
> >> PyCon, and the Python Language Summit, is nearly upon us. We have a good number of people confirmed to attend. If you are intending to come to the language summit but haven't let me know please do so.
> >>
> >> The agenda of topics for discussion so far includes the following:
> >>
> >> * A report on pypy status - Maciej and Armin
> >> * Jython and IronPython status reports - Dino / Frank
> >> * Packaging (Doug Hellmann and Monty Taylor at least)
> >> * Cleaning up interpreter initialisation (both in hopes of finding areas
> >>  to rationalise and hence speed things up, as well as making things
> >>  more embedding friendly). Nick Coghlan
> >> * Adding new async capabilities to the standard library (Guido)
> >> * cffi and the standard library - Maciej
> >> * flufl.enum and the standard library - Barry Warsaw
> >> * The argument clinic - Larry Hastings
> >>
> >> If you have other items you'd like to discuss please let me know and I can add them to the agenda.
> >
> > May I in absentia propose at least a short discussion of the XML fixes
> > and accompanying security releases?  FWIW, for 3.2 and 3.3 I have no
> > objections to secure-by-default.
> >
> 
> Sure. It would be good if someone who *will* be there can champion the discussion.
> 
> While Christian is in the best position to discuss this, I did review his various monkeypatch fixes + expat patches so I can attempt to answer any questions people may have.

I've put you next to the topic in the agenda Brett :-)

Michael

--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From fuzzyman at voidspace.org.uk  Tue Mar  5 01:35:51 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:35:51 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
Message-ID: <EC99491B-2DBC-42E4-856F-C453A05F329F@voidspace.org.uk>


On 4 Mar 2013, at 20:02, Daniel Holth <dholth at gmail.com> wrote:

> On Mon, Mar 4, 2013 at 2:26 PM, Berker Peksa? <berker.peksag at gmail.com> wrote:
>> On Mon, Mar 4, 2013 at 9:14 PM, Barry Warsaw <barry at python.org> wrote:
>>> On Mar 04, 2013, at 07:41 PM, Antoine Pitrou wrote:
>>> 
>>>>> $ python -m unittest discover
>>>>> $ python setup.py test
>>>>> $ python setup.py nosetests
>>>>> $ python -m nose test
>>>>> $ nosetests-X.Y
>>>>> 
>>>>> Besides having a multitude of choices, there's almost no way to
>>>>> automatically discover (e.g. by metadata inspection or some such) how to
>>>>> invoke the tests.  You're often lucky if there's a README.test and it's
>>>>> still accurate.
>>>> 
>>>> I hope we can have a "pytest" utility that does the right thing in 3.4 :-)
>>>> Typing "python -m unittest discover" is too cumbersome.
>>> 
>>> Where is this work being done (e.g. is there a PEP)?
>> 
>> There is an open issue on the tracker: http://bugs.python.org/issue14266
>> 
>> --Berker
> 
> setup.py's setup(test_suite="x")... not sure if this is a distutils or
> setuptools feature. PEP 426 has an extension mechanism that could do
> the job.


This is a setuptools extension. There was some discussion for packaging/distutils2 of having test support but I have no idea if that has been picked up for the new bunch of packaging related work.

Michael

> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From eliben at gmail.com  Tue Mar  5 01:35:41 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Mon, 4 Mar 2013 16:35:41 -0800
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
Message-ID: <CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>

On Mon, Mar 4, 2013 at 4:09 PM, Robert Collins <robertc at robertcollins.net>wrote:

> On 5 March 2013 12:49, Eli Bendersky <eliben at gmail.com> wrote:
> >
> > On Mon, Mar 4, 2013 at 2:14 PM, Barry Warsaw <barry at python.org> wrote:
> >>
> >> On Mar 05, 2013, at 11:01 AM, Robert Collins wrote:
> >>
> >> >The big thing is automated tools, not developers.
> >>
> >> Exactly.
> >
> > I don't understand. Is "python -m unittest discover" too much typing for
> > automatic tools? If anything, it's much more portable across Python
> versions
> > since any new coommand/script won't be added before 3.4, while the longer
> > version works in 2.7 and 3.2+
>
> It isn't about length. It is about knowing that *that* is what to type
> (and btw that exact command cannot run twisted's tests, among many
> other projects tests).
>
> Perhaps we are talking about different things. A top level script to
> run tests is interesting, but orthogonal to the thing Barry was asking
> for.
>

Perhaps :-)
I'm specifically referring to a new top-level script that will run all
unittests in discovery mode from the current directory, as a shortcut to
"python -m unittest discover". ISTM this is at leas in part what was
discussed, and my email was in this context.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130304/65883813/attachment.html>

From fuzzyman at voidspace.org.uk  Tue Mar  5 01:39:40 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:39:40 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130301193803.4156607c@pitrou.net>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
	<20130301193803.4156607c@pitrou.net>
Message-ID: <D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>


On 1 Mar 2013, at 18:38, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Fri, 1 Mar 2013 09:32:23 -0500
> Barry Warsaw <barry at python.org> wrote:
>> 
>>> On the other hand in some ways Jython is sort of like Python on a
>>> weird virtual OS that lets the real OS bleed through some. This may
>>> still need to be checked in that way (there's are still checks of <if
>>> os.name == 'nt'> right?)
>> 
>> Yeah, but that all ooooold code ;)
> 
> Hmm, what do you mean? `os.name == 'nt'` is still the proper way to
> test that we're running on a Windows system (more accurately, over the
> Windows API).
> 

It has been used incorrectly in a few places in the Python standard library - Windows support code that would work correctly on IronPython is skipped because os.name is *not* 'nt' on IronPython.  That was the case in the past anyway. It's quite some time since I've used IronPython now.

Michael

> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From robertc at robertcollins.net  Tue Mar  5 01:41:01 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 13:41:01 +1300
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
Message-ID: <CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>

On 5 March 2013 13:35, Eli Bendersky <eliben at gmail.com> wrote:

> Perhaps :-)
> I'm specifically referring to a new top-level script that will run all
> unittests in discovery mode from the current directory, as a shortcut to
> "python -m unittest discover". ISTM this is at leas in part what was
> discussed, and my email was in this context.

So that is interesting, but its not sufficient to meet the automation
need Barry is calling out, unless all test suites can be run by
'python -m unittest discover' with no additional parameters [and a
pretty large subset cannot].

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From breamoreboy at yahoo.co.uk  Tue Mar  5 01:51:24 2013
From: breamoreboy at yahoo.co.uk (Mark Lawrence)
Date: Tue, 05 Mar 2013 00:51:24 +0000
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
Message-ID: <kh3fbr$juf$1@ger.gmane.org>

On 04/03/2013 22:08, Brett Cannon wrote:
>
>
>
> On Mon, Mar 4, 2013 at 4:33 PM, Mark Lawrence <breamoreboy at yahoo.co.uk
> <mailto:breamoreboy at yahoo.co.uk>> wrote:
>
>     On 04/03/2013 20:46, Terry Reedy wrote:
>
>         On 3/4/2013 11:36 AM, Brett Cannon wrote:
>
>
>
>
>             On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin
>             <brian at python.org <mailto:brian at python.org>
>             <mailto:brian at python.org <mailto:brian at python.org>>> wrote:
>
>                  The full announcement is at
>
>             http://blog.python.org/2013/__03/introducing-electronic-__contributor.html
>             <http://blog.python.org/2013/03/introducing-electronic-contributor.html>,
>                  but a summary follows.
>
>                  We've now moved to an electronic Contributor License
>             Agreement
>             form at
>             http://www.python.org/psf/__contrib/contrib-form/
>             <http://www.python.org/psf/contrib/contrib-form/> which will
>             hopefully
>                  ease the signing and sending of forms for our potential
>             contributors.
>                  The form shows the required fields whether you're
>             signing as an
>                  individual or a representative of an organization, and
>             removes the
>                  need to print, scan, fax, etc.
>
>                  When a new contributor fills in the form, they are
>             emailed a copy of
>                  the form and asked to confirm the email address that
>             they used (and
>                  received that copy at). Upon confirming, the signed
>             form is sent to
>                  the PSF Administrator and filed away.
>
>                  The signature can either be generated from your typed
>             name, or you
>             can
>                  draw or upload your actual written signature if you choose.
>
>
>             With this in place I would like to propose that all patches
>             submitted to
>             bugs.python.org <http://bugs.python.org>
>             <http://bugs.python.org> must come from someone who has
>             signed the CLA before we consider committing it (if you want
>             to be truly
>             paranoid we could say that we won't even look at the code
>             w/o a CLA).
>
>
>         Either policy could be facilitated by tracker changes. In order
>         to see
>         the file upload box, one must login and the tracker knows who
>         has a CLA
>         on file (as indicated by a * suffix on the name). If a file is
>         uploaded
>         by someone without, a box could popup with the link to the
>         e-form and a
>         message that a CLA is required.
>
>
>     People already use the bug tracker as an excuse not to contribute,
>     wouldn't this requirement make the situation worse?
>
>
> Depends on your paranoia. If you're worried about accidentally lifting
> IP merely by reading someone's source code, then you wouldn't want to
> touch code without the CLA signed.
>
> Now I'm not that paranoid, but I'm still not about to commit someone's
> code now without the CLA signed to make sure we are legally covered for
> the patch. If someone chooses not to contribute because of the CLA
> that's fine, but since we have already told at least Anatoly that we
> won't accept patches from him until he signs the CLA I'm not going to
> start acting differently towards others. I view legally covering our ass
> by having someone fill in a form is worth the potential loss of some
> contribution in the grand scheme of things.
>
>

Who's talking source code, you're previously mentioned *ALL* patches 
needing a CLA.  Does this mean you have to sign a CLA for a one line 
documentation patch?  What is the definition of a patch, an actual patch 
file or a proposal for a change that is given within a bug tracker message?

-- 
Cheers.

Mark Lawrence


From fuzzyman at voidspace.org.uk  Tue Mar  5 01:50:20 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:50:20 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ3HGcW73hsm_suDrzCwSZ=LCpmH95Fx9ekGMHE6zNsa_g@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<E48BC372-3B57-414B-BD43-835784AED2C6@voidspace.org.uk>
	<CAJ3HoZ3HGcW73hsm_suDrzCwSZ=LCpmH95Fx9ekGMHE6zNsa_g@mail.gmail.com>
Message-ID: <17397A0A-200F-41F5-ACF1-2E3C2338865E@voidspace.org.uk>


On 5 Mar 2013, at 00:23, Robert Collins <robertc at robertcollins.net> wrote:

> On 5 March 2013 13:21, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> 
> 
>> We can certainly talk about it - although as Guido says, something specific may be easier to have a useful discussion about.
>> 
>> Reading through your blog articles it seemed like a whole lot of subunit context was required to understand the specific
>> proposal you're making for the TestResult. It also *seems* like you're redesigning the TestResult for a single use case
>> (distributed testing) with an api that looks quite "odd" for anything that isn't that use case. I'd rather see how we can
>> make the TestResult play *better* with those requirements. That discussion probably belongs in another thread - or at
>> the summit.
> 
> Right - all I wanted was to flag that you and I and any other
> interested parties should discuss this at the summit :).

I've added a testing topic to the agenda. At the very least you could outline your streaming test result proposal, or kick off a meta discussion. We'll probably time limit the discussion so some specific focus will make it more productive - or maybe you can get a feel for how open to major changes in this area other python devs are.

Michael

> 
> -Rob
> 
> 
> 
> 
> 
> 
> -- 
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Cloud Services


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From robertc at robertcollins.net  Tue Mar  5 01:52:51 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 13:52:51 +1300
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <17397A0A-200F-41F5-ACF1-2E3C2338865E@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<E48BC372-3B57-414B-BD43-835784AED2C6@voidspace.org.uk>
	<CAJ3HoZ3HGcW73hsm_suDrzCwSZ=LCpmH95Fx9ekGMHE6zNsa_g@mail.gmail.com>
	<17397A0A-200F-41F5-ACF1-2E3C2338865E@voidspace.org.uk>
Message-ID: <CAJ3HoZ0kc1VqntDOHrq8972mKW678NPi2aagBBrse-MAxwu1aQ@mail.gmail.com>

On 5 March 2013 13:50, Michael Foord <fuzzyman at voidspace.org.uk> wrote:

>> Right - all I wanted was to flag that you and I and any other
>> interested parties should discuss this at the summit :).
>
> I've added a testing topic to the agenda. At the very least you could outline your streaming test result proposal, or kick off a meta discussion. We'll probably time limit the discussion so some specific focus will make it more productive - or maybe you can get a feel for how open to major changes in this area other python devs are.


Cool. I can step through the core use cases and differences to what
TestResult is in pretty short order. We can spider out from there as
folk desire.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From fuzzyman at voidspace.org.uk  Tue Mar  5 01:54:33 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:54:33 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <20130304172455.5ef0aa5e@anarchist.wooz.org>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<20130304115104.224493ef@anarchist.wooz.org>
	<20130304194148.7e6b42f5@pitrou.net>
	<20130304141457.603b4965@anarchist.wooz.org>
	<CAF4280LCRJ7WwZ84=B51vKHbGa5z+qPZR=B=YZuV+RskwZaw2A@mail.gmail.com>
	<CAG8k2+6zEPNHaMi5gouWTayRYHDb+qiAG3-VgAVmoPGqs7pH0Q@mail.gmail.com>
	<20130304151439.611dd9aa@anarchist.wooz.org>
	<CAG8k2+6J0TMKPx+E9yAH_-rGSRsDpeUKj=FurNTHiS4vgD-BFA@mail.gmail.com>
	<20130304154505.016092cf@anarchist.wooz.org>
	<CAP1=2W5o_zZKe1MDyXnPnmm7YZqXmVW_Hru5+BQwks29QoPJhg@mail.gmail.com>
	<20130304172455.5ef0aa5e@anarchist.wooz.org>
Message-ID: <EB0A3772-FD69-420E-A12D-B3D56DDB5296@voidspace.org.uk>


On 4 Mar 2013, at 22:24, Barry Warsaw <barry at python.org> wrote:

> On Mar 04, 2013, at 05:04 PM, Brett Cannon wrote:
> 
>> Sure, but that has nothing to do with programmatic package discovery.
>> That's something you will have to do as a person in making a qualitative
>> decision along the same lines as API design. Flipping a bit in a config
>> file saying "I have tests" doesn't say much beyond you flipped a bit, e.g.
>> no idea on coverage, quality, etc.
> 
> What I'm looking for is something that automated tools can use to easily
> discover how to run a package's tests.  I want it to be dead simple for
> developers of a package to declare how their tests are to be run, and what
> extra dependencies they might need.  It seems like PEP 426 only addresses the
> latter.  Maybe that's fine and a different PEP is needed to describe automated
> test discover, but I still think it's an important use case.
> 
> Imagine:
> 
> * Every time you upload a package to PyPI, snakebite runs your test suite on
>   a variety of Python versions and platforms.  You get a nice link to the
>   Jenkins results so you and your users get a good sense of overall package
>   quality.
> 
> * You have an automated gatekeeper that will prevent commits or uploads if
>   your coverage or test results get worse instead of better.
> 
> * Distro packagers can build tools that auto-discover the tests so that they
>   are run automatically when the package is built, ensuring high quality
>   packages specifically targeted to those distros.
> 
> As a community, we know how important tests are, so I think our tools should
> reflect that and make it easy for those tests to be expressed.  As a selfish
> side-effect, I want to reduce the amount of guesswork I need to perform in
> order to know how to run a package's test when I `$vcs clone` their
> repository. ;)
> 


Distutils2 had a way of specifying this in the metadata. It looks like this hasn't made it into the reboot:

	 http://alexis.notmyidea.org/distutils2/distutils/newcommands.html

Michael 

> Cheers,
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From fuzzyman at voidspace.org.uk  Tue Mar  5 01:56:42 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 00:56:42 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAJ3HoZ1iVOhYA=_RT-T_8GZybyRtxUJ6eBs3XXeXag+Pw+E-cw@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CAJ3HoZ14jVr=9AR6cwnx8G5ZyWmp3gtCTuu7DPOB0UmT7fs+JQ@mail.gmail.com>
	<CAP7+vJ+qarVjnDpHPtk4d+G1RCN2_idpn0sYZidrj-pAVsu3EQ@mail.gmail.com>
	<CAJ3HoZ3vYBFpKdypjWjDb_MVbFEvYubebMP4=mMFBxACzufh5w@mail.gmail.com>
	<20130304112937.520651d4@anarchist.wooz.org>
	<CAP1=2W7XPh0npM82p_5RGg_dZVT4+XG=+qnU73UQ2kEQW+hxVA@mail.gmail.com>
	<CAJ3HoZ1iVOhYA=_RT-T_8GZybyRtxUJ6eBs3XXeXag+Pw+E-cw@mail.gmail.com>
Message-ID: <CC0BAF03-25CE-4DA9-9891-8ADF3B7806FC@voidspace.org.uk>


On 4 Mar 2013, at 20:00, Robert Collins <robertc at robertcollins.net> wrote:

> On 5 March 2013 05:34, Brett Cannon <brett at python.org> wrote:
>> 
>> 
>> 
>> On Mon, Mar 4, 2013 at 11:29 AM, Barry Warsaw <barry at python.org> wrote:
>>> 
>>> On Mar 04, 2013, at 07:26 PM, Robert Collins wrote:
>>> 
>>>> It is of course possible for subunit and related tools to run their
>>>> own implementation, but it seems ideal to me to have a common API
>>>> which regular unittest, nose, py.test and others can all agree on and
>>>> use : better reuse for pretty printers, GUI displays and the like
>>>> depend on some common API.
>>> 
>>> And One True Way of invoking and/or discovering how to invoke, a package's
>>> test suite.
>> 
>> 
>> How does unittest's test discovery not solve that?
> 
> Three reasons
> a) There are some bugs (all filed I think) - I intend to hack on
> these in the near future - that prevent discovery working at all for
> some use cases.


The only discovery related issues I'm aware of are:

* Issue 16079 (filed by you) - trivial to fix just needs a test
* Issue 15010 obscure and unlikely to be an issue for standard discovery

I'm not aware of any other discovery related issues. Please let me know (or add me as nosy) to them.

> b) discovery requires magic parameters that are project specific
> (e.g. is it 'discover .' or 'discover . lib' to run it). This is
> arguably a setup.py/packaging entrypoint issue.


This was addressed by Barry - and yes discovery has to be done with the right parameters. If you layout your project in a particular way then "python -m unittest discover" in the project root will just work. This is project specific metadata though and not a particular problem of any testing library.

> c) Test suites written for e.g. Twisted, or nose, or other
> non-stdunit-runner-compatible test runners will fail to execute even
> when discovered correctly.
> 
> There are ways to solve this without addressing a/b/c - just defining
> a standard command to run that signals success/failure with it's exit
> code. Packages can export a particular flavour of that in their
> setup.py if they have exceptional needs, and do nothing in the common
> case. That doesn't solve 'how to introspect a package test suite' but
> for distro packagers - and large scale CI integration - that doesn't
> matter.
> 
> For instance testrepository offers a setuptools extension to let it be
> used trivially, I believe nose does something similar.
> 

unittest2 also has setuptools compatible test command.

> Having something that would let *any* test suite spit out folk's
> favourite test protocol de jour would be brilliant of course :).
> [junit-xml, subunit, TAP, ...]
> 

Yes.

Michael

> -Rob
> 
> -- 
> Robert Collins <rbtcollins at hp.com>
> Distinguished Technologist
> HP Cloud Services
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From dholth at gmail.com  Tue Mar  5 03:24:57 2013
From: dholth at gmail.com (Daniel Holth)
Date: Mon, 4 Mar 2013 21:24:57 -0500
Subject: [Python-Dev] running tests; mebs
Message-ID: <CAG8k2+6tkv-1V0htd+UTDj9ZTuNWAefU9moZt-C58eNhHy=qbg@mail.gmail.com>

>> As a community, we know how important tests are, so I think our tools should
>> reflect that and make it easy for those tests to be expressed.  As a selfish
>> side-effect, I want to reduce the amount of guesswork I need to perform in
>> order to know how to run a package's test when I `$vcs clone` their
>> repository. ;)
>>
>
>
> Distutils2 had a way of specifying this in the metadata. It looks like this hasn't made it into the reboot:
>
>          http://alexis.notmyidea.org/distutils2/distutils/newcommands.html
>
> Michael
>
>> Cheers,
>> -Barry

I'm not aware of a reboot of the setup.py replacement / improvement
effort. The work that has been done has proceeded backwards from the
installer end of things.

I had a proposal called "mebs, not an actual project". A completely
plugin-based system would recognize any sdist format and provide a
minimal, consistent interface. Add tests to the below text from
October.

...

A very simple meta-build system "mebs" is used to recognize sdists and
build binary packages. Build systems provide plugins having three
methods,

.recognize()
.metadata()
.build()

An installer downloads an sdist. For each installed build plugin,
.recognize(dir) is called. The first plugin to return True is used.

From barry at python.org  Tue Mar  5 03:43:15 2013
From: barry at python.org (Barry Warsaw)
Date: Mon, 4 Mar 2013 21:43:15 -0500
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
Message-ID: <20130304214315.3037ebe3@limelight.wooz.org>

On Mar 05, 2013, at 01:09 PM, Robert Collins wrote:

>It isn't about length. It is about knowing that *that* is what to type
>(and btw that exact command cannot run twisted's tests, among many
>other projects tests).

>Perhaps we are talking about different things. A top level script to
>run tests is interesting, but orthogonal to the thing Barry was asking
>for.

Right, two different things.

-Barry

From stephen at xemacs.org  Tue Mar  5 04:13:44 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 05 Mar 2013 12:13:44 +0900
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh33sr$b2k$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
Message-ID: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp>

Mark Lawrence writes:

 > People already use the bug tracker as an excuse not to contribute, 
 > wouldn't this requirement make the situation worse?

A failure to sign the CLA is already a decision not to contribute to
the distribution, no matter how noisy they are on the tracker and
list.  I think that pretty much any upload is potential content for
inclusion in Python.  For example, uploading a log of an interactive
session reproducing a bug could easily evolve into contribution of a
doctest.

Since the proposed page only triggers on uploads, I think we're in
"yes, we really do want this person's CLA" territory.  The procedure
is actually rather cool.  As Eli says, the tough part is finding your
user name, but OpenID or browser memory makes that reasonably close to
trivial for many people.

It's true that people upload "one-line documentation patches," and
these don't require a CLA under even the most paranoid interpretation
of US law.  The FSF's guideline is 16 lines, I believe.  However, the
FSF's guideline also says those 16 lines are lifetime cumulative (per
copyrighted work, but we're only talking about one, Python).  In my
experience (with a different project, so FWIW) somebody who goes to
the trouble of uploading a doc typo patch is likely to be a repeat
offender, whereas "drive-by" contributors who just need that one
feature so their web2.0 app works as desired are often going to be in
16-line territory anyway.

This argument doesn't catch 100% of those who might be deterred by the
popup, but it's definitely enough to make the popup worthwhile.

IANAL-but-I-like-a-good-license-flamewar-as-much-as-the-next-guy-ly y'rs,

From jdhardy at gmail.com  Tue Mar  5 06:39:12 2013
From: jdhardy at gmail.com (Jeff Hardy)
Date: Mon, 4 Mar 2013 21:39:12 -0800
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
	<20130301193803.4156607c@pitrou.net>
	<D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>
Message-ID: <CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>

On Mon, Mar 4, 2013 at 4:39 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>
> On 1 Mar 2013, at 18:38, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
>> On Fri, 1 Mar 2013 09:32:23 -0500
>> Barry Warsaw <barry at python.org> wrote:
>>>
>>>> On the other hand in some ways Jython is sort of like Python on a
>>>> weird virtual OS that lets the real OS bleed through some. This may
>>>> still need to be checked in that way (there's are still checks of <if
>>>> os.name == 'nt'> right?)
>>>
>>> Yeah, but that all ooooold code ;)
>>
>> Hmm, what do you mean? `os.name == 'nt'` is still the proper way to
>> test that we're running on a Windows system (more accurately, over the
>> Windows API).
>>
>
> It has been used incorrectly in a few places in the Python standard library - Windows support code that would work correctly on IronPython is skipped because os.name is *not* 'nt' on IronPython.  That was the case in the past anyway. It's quite some time since I've used IronPython now.

I think you misremembered - there's lots of code that uses
`sys.platform == 'win32'` to detect Windows, but sys.platform is 'cli'
for IronPython. I'm pretty sure `os.name has always been 'nt' (when
running on Windows), and if not, it definitely is now.

Jython sets os.name to 'java' (IIRC), so there isn't a uniform way to
detect Windows across all implementations.

- Jeff

From regebro at gmail.com  Tue Mar  5 08:02:09 2013
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 5 Mar 2013 08:02:09 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
Message-ID: <CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>

On Tue, Mar 5, 2013 at 1:41 AM, Robert Collins
<robertc at robertcollins.net> wrote:
> So that is interesting, but its not sufficient to meet the automation
> need Barry is calling out, unless all test suites can be run by
> 'python -m unittest discover' with no additional parameters [and a
> pretty large subset cannot].

But can they be changed so they are? That's gotta be the important bit.

What's needed here is not a tool that can run all unittests in
existence, but an official way for automated tools to run tests, with
the ability for any test and test framework to hook into that, so that
you can run any test suite automatically from an automated tool. The,
once that mechanism has been identified/implemented, we need to tell
everybody to do this.

I don't care much what that mechanism is, but I think the easiest way
to get there is to tell people to extend distutils with a test command
(or use Distribute) and perhaps add such a command in 3.4 that will do
the unittest discover thingy. I remember looking into zope.testrunner
hooking into that mechanism as well, but I don't remember what the
outcome was.

//Lennart

From donald.stufft at gmail.com  Tue Mar  5 08:11:44 2013
From: donald.stufft at gmail.com (Donald Stufft)
Date: Tue, 5 Mar 2013 02:11:44 -0500
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
Message-ID: <FED6DE3A5AE140FD884CDD899F20A043@gmail.com>

On Tuesday, March 5, 2013 at 2:02 AM, Lennart Regebro wrote:
> On Tue, Mar 5, 2013 at 1:41 AM, Robert Collins
> <robertc at robertcollins.net (mailto:robertc at robertcollins.net)> wrote:
> > So that is interesting, but its not sufficient to meet the automation
> > need Barry is calling out, unless all test suites can be run by
> > 'python -m unittest discover' with no additional parameters [and a
> > pretty large subset cannot].
> > 
> 
> 
> But can they be changed so they are? That's gotta be the important bit.
> 
> What's needed here is not a tool that can run all unittests in
> existence, but an official way for automated tools to run tests, with
> the ability for any test and test framework to hook into that, so that
> you can run any test suite automatically from an automated tool. The,
> once that mechanism has been identified/implemented, we need to tell
> everybody to do this.
> 
> I don't care much what that mechanism is, but I think the easiest way
> to get there is to tell people to extend distutils with a test command
> (or use Distribute) and perhaps add such a command in 3.4 that will do
> the unittest discover thingy. I remember looking into zope.testrunner
> hooking into that mechanism as well, but I don't remember what the
> outcome was.
> 
> 

Doesn't setuptools/distribute already have a setup.py test command? That
seems like the easiest way forward? 
> 
> //Lennart
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org (mailto:Python-Dev at python.org)
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald.stufft%40gmail.com
> 
> 


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130305/061b4ba3/attachment.html>

From robertc at robertcollins.net  Tue Mar  5 08:13:02 2013
From: robertc at robertcollins.net (Robert Collins)
Date: Tue, 5 Mar 2013 20:13:02 +1300
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
Message-ID: <CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>

On 5 March 2013 20:02, Lennart Regebro <regebro at gmail.com> wrote:
> On Tue, Mar 5, 2013 at 1:41 AM, Robert Collins
> <robertc at robertcollins.net> wrote:
>> So that is interesting, but its not sufficient to meet the automation
>> need Barry is calling out, unless all test suites can be run by
>> 'python -m unittest discover' with no additional parameters [and a
>> pretty large subset cannot].
>
> But can they be changed so they are? That's gotta be the important bit.

In principle maybe. Need to talk with the trial developers, nose
developers, py.test developers etc - to get consensus on a number of
internal API friction points.

> What's needed here is not a tool that can run all unittests in
> existence, but an official way for automated tools to run tests, with
> the ability for any test and test framework to hook into that, so that
> you can run any test suite automatically from an automated tool. The,
> once that mechanism has been identified/implemented, we need to tell
> everybody to do this.

I think the command line is the right place to do that - declare as
metadata the command line to run a packages tests.

> I don't care much what that mechanism is, but I think the easiest way
> to get there is to tell people to extend distutils with a test command
> (or use Distribute) and perhaps add such a command in 3.4 that will do
> the unittest discover thingy. I remember looking into zope.testrunner
> hooking into that mechanism as well, but I don't remember what the
> outcome was.

Agreed.

-Rob

-- 
Robert Collins <rbtcollins at hp.com>
Distinguished Technologist
HP Cloud Services

From solipsis at pitrou.net  Tue Mar  5 08:19:16 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 5 Mar 2013 08:19:16 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<20130304222848.5006bf3b@pitrou.net>
	<CAF-Rda-28cRSiMqc_5yzNAtMk5SWUdaofuiHtT_pzpJ-_Bq+TA@mail.gmail.com>
Message-ID: <20130305081916.27f38906@pitrou.net>

On Mon, 4 Mar 2013 15:47:37 -0800
Eli Bendersky <eliben at gmail.com> wrote:
> On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> 
> > On Mon, 4 Mar 2013 13:26:57 -0800
> > Eli Bendersky <eliben at gmail.com> wrote:
> > > [Splitting into a separate thread]
> > >
> > > Do we really need to overthink something that requires a trivial alias to
> > > set up for one's own convenience?
> > >
> > > Picking a Python version (as Barry mentions) is just one of the problems.
> > > What's wrong with:
> > >
> > > alias rupytests='python3 -m unittest discover"
> > > alias runpytests2='python2 -m unittest discover"
> > >
> > > ?
> > >
> > > Don't get me wrong, I love the "discover" option and agree that it should
> > > be the recommended way to go - but isn't this largely a documentation
> > issue?
> >
> > I would personally call it a typing issue :-) "python -m unittest
> > discover" is just too long.
> >
> 
> Command-line options for advanced capabilities can get long, yes.

The whole point is that discovery is not "advanced capability", it's
pretty basic by today's standards. So it should actually be the default
behaviour (like it is with nose).

Regards

Antoine.



From tjreedy at udel.edu  Tue Mar  5 08:36:41 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 05 Mar 2013 02:36:41 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh3fbr$juf$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
	<kh3fbr$juf$1@ger.gmane.org>
Message-ID: <kh47aa$ho1$1@ger.gmane.org>

On 3/4/2013 7:51 PM, Mark Lawrence wrote:

> Who's talking source code, you're previously mentioned *ALL* patches
> needing a CLA.  Does this mean you have to sign a CLA for a one line
> documentation patch?

It it is a one char typo, I would not bother downloading the patch, or 
adding a person to ACKS. If the patch is big enough to download and 
apply, then I want a CLA. If a person is sophisticated enough to submit 
a respository file diff, they are likely to submit more, and I want them 
to feel encouraged to do so by already having done the CLA. If we do not 
get it with the first submission, then when? Who keeps track of 
cumulative lines?

> What is the definition of a patch, an actual patch file

Usually.

> or a proposal for a change that is given within a bug tracker message?

I view a proposal for a change as just an idea. Such usually get 
re-written by whoever creates an actual patch.

I would like the link to the e-form to be accessible somewhere on the 
tracker so I can refer people to it easily.

-- 
Terry Jan Reedy


From ncoghlan at gmail.com  Tue Mar  5 09:25:38 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 5 Mar 2013 18:25:38 +1000
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
Message-ID: <CADiSq7fjfuqF2bnbZZfii1ru_aYuSub83vPSrY8vNYAsWx0SHw@mail.gmail.com>

On Tue, Mar 5, 2013 at 5:02 PM, Lennart Regebro <regebro at gmail.com> wrote:
> I don't care much what that mechanism is, but I think the easiest way
> to get there is to tell people to extend distutils with a test command
> (or use Distribute) and perhaps add such a command in 3.4 that will do
> the unittest discover thingy. I remember looking into zope.testrunner
> hooking into that mechanism as well, but I don't remember what the
> outcome was.

There is no easy way forward at this point in time. There just isn't.
If people want to dispute that claim, please feel free to solve all
the other problems distutils-sig is trying to tackle, so we can pay
attention to this one.

We'll get to this eventually - there are just several other more
important things ahead of it in the queue for packaging and
distribution infrastructure enhancements (and python-dev is not the
group that will solve them).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From stephen at xemacs.org  Tue Mar  5 09:31:03 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 05 Mar 2013 17:31:03 +0900
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh47aa$ho1$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
	<kh3fbr$juf$1@ger.gmane.org> <kh47aa$ho1$1@ger.gmane.org>
Message-ID: <87a9qijliw.fsf@uwakimon.sk.tsukuba.ac.jp>

Terry Reedy writes:

 > > or a proposal for a change that is given within a bug tracker message?
 > 
 > I view a proposal for a change as just an idea. Such usually get 
 > re-written by whoever creates an actual patch.

Precisely how U.S. law would view it, implying no copyright issue.

If this really need further discussion, it should move to
python-legal.


From glyph at twistedmatrix.com  Tue Mar  5 10:02:46 2013
From: glyph at twistedmatrix.com (Glyph)
Date: Tue, 5 Mar 2013 01:02:46 -0800
Subject: [Python-Dev] built-in Python test runner (was: Python Language
	Summit at PyCon: Agenda)
In-Reply-To: <CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>
Message-ID: <183E8AF2-3D1D-4780-9015-686E94635A9F@twistedmatrix.com>

On Mar 4, 2013, at 11:13 PM, Robert Collins <robertc at robertcollins.net> wrote:

> In principle maybe. Need to talk with the trial developers, nose
> developers, py.test developers etc - to get consensus on a number of
> internal API friction points.

Some of trial's lessons might be also useful for the stdlib going forward, given the hope of doing some event-loop stuff in the core.

But, I feel like this might be too much to cover at the language summit; there could be a test frameworks summit of its own, of about equivalent time and scope, and we'd still have a lot to discuss.

Is there a unit testing SIG someone from Twisted ought to be a member of, to represent Trial, and to get consensus on these points going forward?

-glyph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130305/d81f97ff/attachment.html>

From holger.krekel at gmail.com  Tue Mar  5 11:15:28 2013
From: holger.krekel at gmail.com (Holger Krekel)
Date: Tue, 5 Mar 2013 11:15:28 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <183E8AF2-3D1D-4780-9015-686E94635A9F@twistedmatrix.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>
	<183E8AF2-3D1D-4780-9015-686E94635A9F@twistedmatrix.com>
Message-ID: <CAPJJ-p+z05Z5dOH0WYF5kYkU63myT-fYhiLndWi=HtSLU+KOAA@mail.gmail.com>

On Tue, Mar 5, 2013 at 10:02 AM, Glyph <glyph at twistedmatrix.com> wrote:

> On Mar 4, 2013, at 11:13 PM, Robert Collins <robertc at robertcollins.net>
> wrote:
>
> In principle maybe. Need to talk with the trial developers, nose
> developers, py.test developers etc - to get consensus on a number of
> internal API friction points.
>
>
> Some of trial's lessons might be also useful for the stdlib going forward,
> given the hope of doing some event-loop stuff in the core.
>
> But, I feel like this might be too much to cover at the language summit;
> there could be a test frameworks summit of its own, of about equivalent
> time and scope, and we'd still have a lot to discuss.
>
> Is there a unit testing SIG someone from Twisted ought to be a member of,
> to represent Trial, and to get consensus on these points going forward?
>
>
The testing-in-python list is pretty much where most test tool authors hang
out, see  http://lists.idyll.org/listinfo/testing-in-python

Also, maybe related, i am heading the "tox" effort which many people use to
have a frontend for their testing process, see http://tox.testrun.org -- it
has a somewhat different focus in that it sets up virtualenv and install
test specific dependencies.  However, positional arguments (often used to
select tests) can be configured to be passed on to the test runner of
choice. I was considering extending tox to directly support nose, pytest,
unittest and trial "drivers" and offer a unified (minimal) command line
API.  Am open to collaboration on that.

cheers,
holger



> -glyph
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/holger.krekel%40gmail.com
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130305/b11c7494/attachment-0001.html>

From fuzzyman at voidspace.org.uk  Tue Mar  5 11:52:20 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 10:52:20 +0000
Subject: [Python-Dev] built-in Python test runner (was: Python Language
	Summit at PyCon: Agenda)
In-Reply-To: <20130305081916.27f38906@pitrou.net>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<20130304222848.5006bf3b@pitrou.net>
	<CAF-Rda-28cRSiMqc_5yzNAtMk5SWUdaofuiHtT_pzpJ-_Bq+TA@mail.gmail.com>
	<20130305081916.27f38906@pitrou.net>
Message-ID: <C04B3074-FB2A-430A-AB38-27D1403CF68C@voidspace.org.uk>


On 5 Mar 2013, at 07:19, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Mon, 4 Mar 2013 15:47:37 -0800
> Eli Bendersky <eliben at gmail.com> wrote:
>> On Mon, Mar 4, 2013 at 1:28 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> 
>>> On Mon, 4 Mar 2013 13:26:57 -0800
>>> Eli Bendersky <eliben at gmail.com> wrote:
>>>> [Splitting into a separate thread]
>>>> 
>>>> Do we really need to overthink something that requires a trivial alias to
>>>> set up for one's own convenience?
>>>> 
>>>> Picking a Python version (as Barry mentions) is just one of the problems.
>>>> What's wrong with:
>>>> 
>>>> alias rupytests='python3 -m unittest discover"
>>>> alias runpytests2='python2 -m unittest discover"
>>>> 
>>>> ?
>>>> 
>>>> Don't get me wrong, I love the "discover" option and agree that it should
>>>> be the recommended way to go - but isn't this largely a documentation
>>> issue?
>>> 
>>> I would personally call it a typing issue :-) "python -m unittest
>>> discover" is just too long.
>>> 
>> 
>> Command-line options for advanced capabilities can get long, yes.
> 
> The whole point is that discovery is not "advanced capability", it's
> pretty basic by today's standards. So it should actually be the default
> behaviour (like it is with nose).
> 


For Python 3.3 onwards "python -m unittest" does run test discovery by default. However if you want to provide parameters you still need the "discover" subcommand to disambiguate from the other command line options. So I agree - a shorthand command would be an improvement.

Michael



> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From fuzzyman at voidspace.org.uk  Tue Mar  5 11:55:34 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 10:55:34 +0000
Subject: [Python-Dev] built-in Python test runner (was: Python Language
	Summit at PyCon: Agenda)
In-Reply-To: <183E8AF2-3D1D-4780-9015-686E94635A9F@twistedmatrix.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>
	<183E8AF2-3D1D-4780-9015-686E94635A9F@twistedmatrix.com>
Message-ID: <5F687C3C-7145-4473-A08D-E4786D38E945@voidspace.org.uk>


On 5 Mar 2013, at 09:02, Glyph <glyph at twistedmatrix.com> wrote:

> On Mar 4, 2013, at 11:13 PM, Robert Collins <robertc at robertcollins.net> wrote:
> 
>> In principle maybe. Need to talk with the trial developers, nose
>> developers, py.test developers etc - to get consensus on a number of
>> internal API friction points.
> 
> Some of trial's lessons might be also useful for the stdlib going forward, given the hope of doing some event-loop stuff in the core.
> 
> But, I feel like this might be too much to cover at the language summit; there could be a test frameworks summit of its own, of about equivalent time and scope, and we'd still have a lot to discuss.
> 
> Is there a unit testing SIG someone from Twisted ought to be a member of, to represent Trial, and to get consensus on these points going forward?


The testing-on-python mailing list is probably the best place (and if doesn't have that status already I'd be keen to elevate it to "official sig for Python testing issues" status).

	http://lists.idyll.org/listinfo/testing-in-python

Like the "massively distributed testing" use case, I'd be very happy for the standard library testing capabilities to better support this use case - but I wouldn't like to design their apis around that as the sole use case. :-)

Michael

> 
> -glyph
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From fuzzyman at voidspace.org.uk  Tue Mar  5 12:07:52 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 5 Mar 2013 11:07:52 +0000
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
	<20130301193803.4156607c@pitrou.net>
	<D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>
	<CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>
Message-ID: <3BF8EF8F-98CF-47C7-A498-D043E27BC200@voidspace.org.uk>


On 5 Mar 2013, at 05:39, Jeff Hardy <jdhardy at gmail.com> wrote:

> On Mon, Mar 4, 2013 at 4:39 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> 
>> On 1 Mar 2013, at 18:38, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> 
>>> On Fri, 1 Mar 2013 09:32:23 -0500
>>> Barry Warsaw <barry at python.org> wrote:
>>>> 
>>>>> On the other hand in some ways Jython is sort of like Python on a
>>>>> weird virtual OS that lets the real OS bleed through some. This may
>>>>> still need to be checked in that way (there's are still checks of <if
>>>>> os.name == 'nt'> right?)
>>>> 
>>>> Yeah, but that all ooooold code ;)
>>> 
>>> Hmm, what do you mean? `os.name == 'nt'` is still the proper way to
>>> test that we're running on a Windows system (more accurately, over the
>>> Windows API).
>>> 
>> 
>> It has been used incorrectly in a few places in the Python standard library - Windows support code that would work correctly on IronPython is skipped because os.name is *not* 'nt' on IronPython.  That was the case in the past anyway. It's quite some time since I've used IronPython now.
> 
> I think you misremembered - there's lots of code that uses
> `sys.platform == 'win32'` to detect Windows, but sys.platform is 'cli'
> for IronPython. I'm pretty sure `os.name has always been 'nt' (when
> running on Windows), and if not, it definitely is now.
> 
> Jython sets os.name to 'java' (IIRC), so there isn't a uniform way to
> detect Windows across all implementations.
> 


Ah, I'm sure you're correct. 

Thanks.

> - Jeff


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From ezio.melotti at gmail.com  Tue Mar  5 13:04:46 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Tue, 5 Mar 2013 14:04:46 +0200
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh317n$ifo$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org>
Message-ID: <CACBhJdGUrm0Q3XVESpPomiyNh3UTO+BN-TgfYtfSBYG=ipa_BA@mail.gmail.com>

Hi,

On Mon, Mar 4, 2013 at 10:46 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/4/2013 11:36 AM, Brett Cannon wrote:
>> On Mon, Mar 4, 2013 at 11:30 AM, Brian Curtin <brian at python.org
>> <mailto:brian at python.org>> wrote:
>>
>> With this in place I would like to propose that all patches submitted to
>> bugs.python.org <http://bugs.python.org> must come from someone who has
>>
>> signed the CLA before we consider committing it (if you want to be truly
>> paranoid we could say that we won't even look at the code w/o a CLA).
>
>
> Either policy could be facilitated by tracker changes. In order to see the
> file upload box, one must login and the tracker knows who has a CLA on file
> (as indicated by a * suffix on the name). If a file is uploaded by someone
> without, a box could popup with the link to the e-form and a message that a
> CLA is required.
>

http://psf.upfronthosting.co.za/roundup/meta/issue461

Best Regards,
Ezio Melotti

> --
> Terry Jan Reedy
>

From devel at baptiste-carvello.net  Tue Mar  5 17:48:07 2013
From: devel at baptiste-carvello.net (Baptiste Carvello)
Date: Tue, 05 Mar 2013 17:48:07 +0100
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>	<kh317n$ifo$1@ger.gmane.org>
	<kh33sr$b2k$1@ger.gmane.org>
	<87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <kh57mb$hbl$1@ger.gmane.org>

Le 05/03/2013 04:13, Stephen J. Turnbull a ?crit :
> Mark Lawrence writes:
> 
>  > People already use the bug tracker as an excuse not to contribute, 
>  > wouldn't this requirement make the situation worse?
> 
> A failure to sign the CLA is already a decision not to contribute to
> the distribution

my 2 cents as an occasional contributor of minor patches: I understand
that the scarce resource is reviewer time, so I would definitely accept
to sign the CLA with my next contribution before a reviewer invests his
time in it.

However, please don't make the popup too pushy. I abhor websites which
push people into entering legally binding agreements "with one click"
without the opportunity to study them carefully (personnally, this would
not be a problem as I already know what the CLA is about, but other
contributors might not).

Also, please keep the possibility to use the old paper-based signing
procedure. I for one don't consider so-called "electronic signatures"
based on email address verification (as opposed to real crypto) to be as
good as a handwritten signature, and I don't want to legitimize them by
using them.

Cheers,
Baptiste


From fwierzbicki at gmail.com  Tue Mar  5 17:55:06 2013
From: fwierzbicki at gmail.com (fwierzbicki at gmail.com)
Date: Tue, 5 Mar 2013 08:55:06 -0800
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
	<20130301193803.4156607c@pitrou.net>
	<D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>
	<CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>
Message-ID: <CADrh4zKhSheCsfmm=R_rZTFzwxvVks5V_SznEFt1Q-i6mJCQOw@mail.gmail.com>

On Mon, Mar 4, 2013 at 9:39 PM, Jeff Hardy <jdhardy at gmail.com> wrote:
> I think you misremembered - there's lots of code that uses
> `sys.platform == 'win32'` to detect Windows, but sys.platform is 'cli'
> for IronPython. I'm pretty sure `os.name has always been 'nt' (when
> running on Windows), and if not, it definitely is now.
>
> Jython sets os.name to 'java' (IIRC), so there isn't a uniform way to
> detect Windows across all implementations.
I've been thinking that this is a bit of a historical mistake on our
part. I'm strongly considering setting os.name properly in Jython3.

-Frank

From fwierzbicki at gmail.com  Tue Mar  5 18:00:14 2013
From: fwierzbicki at gmail.com (fwierzbicki at gmail.com)
Date: Tue, 5 Mar 2013 09:00:14 -0800
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADrh4zKhSheCsfmm=R_rZTFzwxvVks5V_SznEFt1Q-i6mJCQOw@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
	<20130301193803.4156607c@pitrou.net>
	<D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>
	<CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>
	<CADrh4zKhSheCsfmm=R_rZTFzwxvVks5V_SznEFt1Q-i6mJCQOw@mail.gmail.com>
Message-ID: <CADrh4zKQ=a7Z4mApQ5XM2zOp67uwo8WvLgw3Cjk6AOrSS4gG+Q@mail.gmail.com>

On Tue, Mar 5, 2013 at 8:55 AM, fwierzbicki at gmail.com
<fwierzbicki at gmail.com> wrote:
> I've been thinking that this is a bit of a historical mistake on our
> part. I'm strongly considering setting os.name properly in Jython3.
In fairness to Jython implementers past - it wasn't a mistake but a
deliberate design choice at the time since in the old days we where
going for the now defunct "100% Java" thing (does anyone remember
that?) so this is more of "the design has evolved such that Jython's
os.name now feels incorrect".

I'll stop having conversations with myself in public now, sorry :)

-Frank

From barry at python.org  Tue Mar  5 20:49:00 2013
From: barry at python.org (Barry Warsaw)
Date: Tue, 5 Mar 2013 14:49:00 -0500
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
Message-ID: <20130305144900.5ecf90ba@resist.wooz.org>

On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote:

>Doesn't setuptools/distribute already have a setup.py test command? That
>seems like the easiest way forward? 

Yes, and in theory it can make `python setup.py test` work well.  But there
are lots of little details (such as API differences for ensuring that doctests
run, "additional tests" discovery, etc.) that make this often not work so well
in practice.  Some of that is social and some of it is technical.  I still
claim that including test suite information in a package's metadata would be a
win, but maybe that's just too much to hope for right now.

-Barry

From dholth at gmail.com  Tue Mar  5 21:20:24 2013
From: dholth at gmail.com (Daniel Holth)
Date: Tue, 5 Mar 2013 15:20:24 -0500
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <20130305144900.5ecf90ba@resist.wooz.org>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
	<20130305144900.5ecf90ba@resist.wooz.org>
Message-ID: <CAG8k2+5a_s8UC4N=Y4d8FBCBiYaaLQbf+OjQBwQ9evGDdfE3zw@mail.gmail.com>

On Tue, Mar 5, 2013 at 2:49 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote:
>
>>Doesn't setuptools/distribute already have a setup.py test command? That
>>seems like the easiest way forward?
>
> Yes, and in theory it can make `python setup.py test` work well.  But there
> are lots of little details (such as API differences for ensuring that doctests
> run, "additional tests" discovery, etc.) that make this often not work so well
> in practice.  Some of that is social and some of it is technical.  I still
> claim that including test suite information in a package's metadata would be a
> win, but maybe that's just too much to hope for right now.

It would be a win, but "parsing the metadata" is just not what happens
right now, let alone writing anything about which and where the
modules are defined in the sdist. We can barely install packages by
using the dependency metadata from PKG-INFO; pip always re-generates
it from "setup.py egg_info".

Your testing metadata prototype would only have to write two lines to
the metadata instead of one a-la: Extension: flufl; flufl/test_suite:
nose.collector; document the extension; write some tool to actually
parse the metadata and invoke the tests; it may become a core feature
in the next version, or having a monolithic specification may become
less important.

Thanks,

Daniel Holth

From brett at python.org  Tue Mar  5 21:22:07 2013
From: brett at python.org (Brett Cannon)
Date: Tue, 5 Mar 2013 15:22:07 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <kh57mb$hbl$1@ger.gmane.org>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<kh57mb$hbl$1@ger.gmane.org>
Message-ID: <CAP1=2W73AtBUXmJpX7DnJd5RnN0424irXJZjp9HNKb0tDZ7UFA@mail.gmail.com>

On Tue, Mar 5, 2013 at 11:48 AM, Baptiste Carvello <
devel at baptiste-carvello.net> wrote:

> Le 05/03/2013 04:13, Stephen J. Turnbull a ?crit :
> > Mark Lawrence writes:
> >
> >  > People already use the bug tracker as an excuse not to contribute,
> >  > wouldn't this requirement make the situation worse?
> >
> > A failure to sign the CLA is already a decision not to contribute to
> > the distribution
>
> my 2 cents as an occasional contributor of minor patches: I understand
> that the scarce resource is reviewer time, so I would definitely accept
> to sign the CLA with my next contribution before a reviewer invests his
> time in it.
>
> However, please don't make the popup too pushy. I abhor websites which
> push people into entering legally binding agreements "with one click"
> without the opportunity to study them carefully (personnally, this would
> not be a problem as I already know what the CLA is about, but other
> contributors might not).
>
> Also, please keep the possibility to use the old paper-based signing
> procedure. I for one don't consider so-called "electronic signatures"
> based on email address verification (as opposed to real crypto) to be as
> good as a handwritten signature, and I don't want to legitimize them by
> using them.
>

At the bottom of the CLA page there are instructions on how to still use
the paper form.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130305/d1c702f5/attachment.html>

From donald at stufft.io  Tue Mar  5 21:33:14 2013
From: donald at stufft.io (Donald Stufft)
Date: Tue, 05 Mar 2013 15:33:14 -0500
Subject: [Python-Dev] built-in Python test runner
In-Reply-To: <20130305144900.5ecf90ba@resist.wooz.org>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
	<20130305144900.5ecf90ba@resist.wooz.org>
Message-ID: <5136568A.3090406@stufft.io>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Barry Warsaw wrote:
> On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote:
> 
>> Doesn't setuptools/distribute already have a setup.py test command?
>> That seems like the easiest way forward?
> 
> Yes, and in theory it can make `python setup.py test` work well.  But
> there are lots of little details (such as API differences for
> ensuring that doctests run, "additional tests" discovery, etc.) that
> make this often not work so well in practice.  Some of that is social
> and some of it is technical.  I still claim that including test suite
> information in a package's metadata would be a win, but maybe that's
> just too much to hope for right now.

For the "right now" solution you can easily override the test command
in setup.py. Long term this would be something that could be added to
the METADATA but I think it is (and rightly should be) lower priority
to actually getting it working for it's core purpose.

> 
> -Barry _______________________________________________ Python-Dev
> mailing list Python-Dev at python.org 
> http://mail.python.org/mailman/listinfo/python-dev Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/donald%40stufft.io
-----BEGIN PGP SIGNATURE-----
Version: GnuPG/MacGPG2 v2.0.18 (Darwin)

iQIcBAEBAgAGBQJRNlaJAAoJEG48vOkzctz6SvsP/2+MYkn1NgcHdmaQg09A3MDH
a++7+hGGsXgQwwJ3q3u6T6Hzd1pokJI7hqGHAXBnkODrZZ8f9Z0+OP9I8HPUmo0D
mJDsDxn2VZImBkNZJlBGNFKIz4EjS5llFapzdl58ZkIVZ7Rz3PTaPVSiXOc+ppp7
xYkWXGnx+2DTFaHywi9qGsHrbAXqgcwIhXO29NYl98xA9X98/XSRiXmHVfNURA3U
1GV1G9H1Qfvu8YjfYfBCCUn6db6eLGVPO7VcxRh6Cyzfk5SFSuziCVI8v3t3msjw
KSba+8Pe3RQ7RS17VEJqCMQjkhhGnAndgIL3Jho41qb3g+Rdk2OP+weWbYV92Q8F
HL6QtPgm5/QS5tKyl6nK97+9q+NdhGOEzKOL0pBiF4HKdT0mKyBxqttVIgUDAVMQ
XcjhBu1wnpQnhkeZ8F3yGNubmE2tRdVVfhTfVDaA3ICl7uVBlbtUUMTJRK7DQ4vW
gzDk5aKJB8OHimC1ijeTQm3M2lXkS5z0e5IuaION2WrG5A2BEvH12d/I7ekc1Ixa
lVhScABFwY0UcrMt4td65er/w4Z0S+BL87SZMH1mffoS6XmL3fxUuOtAB+MUWDk7
Rd4xXnUFvr3SMmMRjSogpO/HO5IBpuzUwu0wqSz8qPcex+l+lHyOEHBDvacEcv9b
zvGYjfQuk+2hWlAqw5o4
=6ELp
-----END PGP SIGNATURE-----

From andrew.svetlov at gmail.com  Tue Mar  5 21:40:38 2013
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 5 Mar 2013 22:40:38 +0200
Subject: [Python-Dev] tp_dictoffset and tp_weaklistoffset slots for Stable
	API
Message-ID: <CAL3CFcVVUWMO1E889nuJ-WWhMJtJ_RiqUvQCh1=whMe=v63RFw@mail.gmail.com>

Looking on PEP http://www.python.org/dev/peps/pep-0384/ and docs I
don't figure out how to specify this values.

Maybe I've missed something?

If not I like to solve that problem at us pycon sprints.
Hope, Martin von Loewis will visit the conference.

--
Thanks,
Andrew Svetlov

From rdmurray at bitdance.com  Tue Mar  5 22:06:17 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 05 Mar 2013 16:06:17 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAP1=2W73AtBUXmJpX7DnJd5RnN0424irXJZjp9HNKb0tDZ7UFA@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<87k3pmk07r.fsf@uwakimon.sk.tsukuba.ac.jp>
	<kh57mb$hbl$1@ger.gmane.org>
	<CAP1=2W73AtBUXmJpX7DnJd5RnN0424irXJZjp9HNKb0tDZ7UFA@mail.gmail.com>
Message-ID: <20130305210618.40C50250BCF@webabinitio.net>

On Tue, 05 Mar 2013 15:22:07 -0500, Brett Cannon <brett at python.org> wrote:
> On Tue, Mar 5, 2013 at 11:48 AM, Baptiste Carvello <
> devel at baptiste-carvello.net> wrote:
> 
> > Le 05/03/2013 04:13, Stephen J. Turnbull a ??crit :
> > > Mark Lawrence writes:
> > >
> > >  > People already use the bug tracker as an excuse not to contribute,
> > >  > wouldn't this requirement make the situation worse?
> > >
> > > A failure to sign the CLA is already a decision not to contribute to
> > > the distribution
> >
> > my 2 cents as an occasional contributor of minor patches: I understand
> > that the scarce resource is reviewer time, so I would definitely accept
> > to sign the CLA with my next contribution before a reviewer invests his
> > time in it.
> >
> > However, please don't make the popup too pushy. I abhor websites which
> > push people into entering legally binding agreements "with one click"
> > without the opportunity to study them carefully (personnally, this would
> > not be a problem as I already know what the CLA is about, but other
> > contributors might not).
> >
> > Also, please keep the possibility to use the old paper-based signing
> > procedure. I for one don't consider so-called "electronic signatures"
> > based on email address verification (as opposed to real crypto) to be as
> > good as a handwritten signature, and I don't want to legitimize them by
> > using them.
> >
> 
> At the bottom of the CLA page there are instructions on how to still use
> the paper form.

Then there also needs to be a way to ACK the popup so that it never
shows up again.  Which we should have anyway.  Ideally that would be
tied to the account and not to, say, a browser cookie.

--David

From ncoghlan at gmail.com  Tue Mar  5 23:44:34 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 6 Mar 2013 08:44:34 +1000
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <20130305144900.5ecf90ba@resist.wooz.org>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
	<20130305144900.5ecf90ba@resist.wooz.org>
Message-ID: <CADiSq7cs0ZqvboHku0jXb3FkGANwRMyh9kp59VtW0Qiozt=0YA@mail.gmail.com>

On 6 Mar 2013 05:51, "Barry Warsaw" <barry at python.org> wrote:
>
> On Mar 05, 2013, at 02:11 AM, Donald Stufft wrote:
>
> >Doesn't setuptools/distribute already have a setup.py test command? That
> >seems like the easiest way forward?
>
> Yes, and in theory it can make `python setup.py test` work well.  But
there
> are lots of little details (such as API differences for ensuring that
doctests
> run, "additional tests" discovery, etc.) that make this often not work so
well
> in practice.  Some of that is social and some of it is technical.  I still
> claim that including test suite information in a package's metadata would
be a
> win, but maybe that's just too much to hope for right now.

I think it's the right answer, too, but PEP 426 is already huge, so
metadata 2.1 is likely the earliest we will try to tackle the problem.

Cheers,
Nick.

>
> -Barry
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130306/58cd1621/attachment.html>

From regebro at gmail.com  Wed Mar  6 07:39:53 2013
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 6 Mar 2013 07:39:53 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<FED6DE3A5AE140FD884CDD899F20A043@gmail.com>
Message-ID: <CAL0kPAURGzOf=+94zXNg73b7943-9nqN7L8e6R_zZVhOvhLxig@mail.gmail.com>

On Tue, Mar 5, 2013 at 8:11 AM, Donald Stufft <donald.stufft at gmail.com> wrote:
> I don't care much what that mechanism is, but I think the easiest way
> to get there is to tell people to extend distutils with a test command
> (or use Distribute) and perhaps add such a command in 3.4 that will do
> the unittest discover thingy. I remember looking into zope.testrunner
> hooking into that mechanism as well, but I don't remember what the
> outcome was.
>
> Doesn't setuptools/distribute already have a setup.py test command?

Yes, but distutils do not.

> That seems like the easiest way forward?

Yup. Although I can understand people if they want something that is
independent of packaging/distribution.

//Lennart

From regebro at gmail.com  Wed Mar  6 07:41:41 2013
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 6 Mar 2013 07:41:41 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<CAJ3HoZ2Sfp6k+h-Zves8e-sM8YtuHZFnKO1mFnrJeAB0y-zEug@mail.gmail.com>
Message-ID: <CAL0kPAVk6_ch-tjMjX8FOB1bpSyMy2AW=8-3Xai4Jg8teaYR1Q@mail.gmail.com>

On Tue, Mar 5, 2013 at 8:13 AM, Robert Collins
<robertc at robertcollins.net> wrote:
> On 5 March 2013 20:02, Lennart Regebro <regebro at gmail.com> wrote:
>> What's needed here is not a tool that can run all unittests in
>> existence, but an official way for automated tools to run tests, with
>> the ability for any test and test framework to hook into that, so that
>> you can run any test suite automatically from an automated tool. The,
>> once that mechanism has been identified/implemented, we need to tell
>> everybody to do this.
>
> I think the command line is the right place to do that - declare as
> metadata the command line to run a packages tests.

Yeah, that's good and simple solution.

//Lennart

From regebro at gmail.com  Wed Mar  6 07:45:58 2013
From: regebro at gmail.com (Lennart Regebro)
Date: Wed, 6 Mar 2013 07:45:58 +0100
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CADiSq7fjfuqF2bnbZZfii1ru_aYuSub83vPSrY8vNYAsWx0SHw@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<CADiSq7fjfuqF2bnbZZfii1ru_aYuSub83vPSrY8vNYAsWx0SHw@mail.gmail.com>
Message-ID: <CAL0kPAW4G=r35-ax2arPPm1f90C56_pjH8HG_7AU7ReD=QEgzg@mail.gmail.com>

On Tue, Mar 5, 2013 at 9:25 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Tue, Mar 5, 2013 at 5:02 PM, Lennart Regebro <regebro at gmail.com> wrote:
>> I don't care much what that mechanism is, but I think the easiest way
>> to get there is to tell people to extend distutils with a test command
>> (or use Distribute) and perhaps add such a command in 3.4 that will do
>> the unittest discover thingy. I remember looking into zope.testrunner
>> hooking into that mechanism as well, but I don't remember what the
>> outcome was.
>
> There is no easy way forward at this point in time. There just isn't.
> If people want to dispute that claim, please feel free to solve all
> the other problems distutils-sig is trying to tackle, so we can pay
> attention to this one.

I have to admit that of all the packaging problems out there, this is
one of the easiest ones. ;-)
That said, it's not easy.

> We'll get to this eventually - there are just several other more
> important things ahead of it in the queue for packaging and
> distribution infrastructure enhancements (and python-dev is not the
> group that will solve them).

To be honest I'm not sure distutils-sig is the right place for this.
It's really only a packaging problem because Setuptools has a "test"
command. :-)
Perhaps we can solve this outside distutils-sig so that distutils-sig
can concentrate on the harder problems?

//Lennart

From tjreedy at udel.edu  Wed Mar  6 11:20:31 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 06 Mar 2013 05:20:31 -0500
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
Message-ID: <kh759h$v7i$1@ger.gmane.org>

Clicking this link
http://www.microsoft.com/en-us/download/details.aspx?id=14597
on this Developer Guide page
http://docs.python.org/devguide/setup.html#windows
now returns a
"We are sorry, the page you requested cannot be found."
page with search results.

The first search result
http://social.msdn.microsoft.com/Forums/nl/Vsexpressinstall/thread/2dc7ae6a-a0e7-436b-a1b3-3597ffac6a97
suggests that one must first go to http://profile.microsoft.com
which forwards to the live.com login page. Logging in with my un-expired 
non-developer account did not make the original link work.

The mdsn page http://msdn.microsoft.com/en-US/
has Visual Studio / Download trial, which leads to
https://www.microsoft.com/visualstudio/eng/downloads
which lists 2012 and 2010 but not 2008.

I suspect that an msdn account is required for most people to get 2008.

A later link leads to
https://www.dreamspark.com/Product/Product.aspx?productid=34#
which suggests that vc++2008 express is also available to verified 
degree students. I don't qualify so I will not try.

So it would appear that section "1.1.3.3. Windows" of "1. Getting 
Started" (setup.rst) needs further revision.

Or perhaps we could persuade Microsoft to let us distribute it ourselves 
so Windows versions of 2.7 do not become increasingly unusable.

-- 
Terry Jan Reedy


From mcepl at redhat.com  Wed Mar  6 14:09:54 2013
From: mcepl at redhat.com (=?UTF-8?Q?Mat=C4=9Bj?= Cepl)
Date: Wed, 06 Mar 2013 14:09:54 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
	Swartz memorial)
In-Reply-To: <mailman.2549.1361895946.2939.python-list@python.org>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
Message-ID: <1362575394.23949.2.camel@wycliff.ceplovi.cz>


On 2013-02-26, 16:25 GMT, Terry Reedy wrote:
> On 2/21/2013 4:22 PM, Matej Cepl wrote:
>> as my method to commemorate Aaron Swartz, I have decided to port his
>> html2text to work fully with the latest python 3.3. After some time
>> dealing with various bugs, I have now in my repo
>> https://github.com/mcepl/html2text (branch python3) working solution
>> which works all the way to python 3.2 (inclusive;
>> https://travis-ci.org/mcepl/html2text). However, the last problem
>> remains. This
>>
>> <li>Run this command:
>> <pre>ls -l *.html</pre></li>
>> <li>?</li>
>>
>> should lead to
>>
>>    * Run this command:
>>
>>          ls -l *.html
>>
>>    * ?
>>
>> but it doesn?t. It leads to this (with python 3.3 only)
>>
>>      * Run this command:
>>            ls -l *.html
>>
>>      * ?
>>
>> Does anybody know about something which changed in modules re or
>> http://docs.python.org/3.3/whatsnew/changelog.html between 3.2 and 
>> 3.3, which could influence this script?
>
> Search the changelob or 3.3 misc/News for items affecting those two 
> modules. There are at least 4.
> http://docs.python.org/3.3/whatsnew/changelog.html
>
> It is faintly possible that the switch from narrow/wide builds to 
> unified builds somehow affected that. Have you tested with 2.7/3.2 on 
> both narrow and wide unicode builds?

So, in the end, I have went the long way and bisected cpython to 
find the commit which broke my tests, and it seems that the 
culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is 
clearly something Unicode related.

Unfortunately, it really doesn't tell me what exactly is broken 
(is it a known regression) and if there is known workaround.  
Could anybody suggest a way how to find bugs on 
http://bugs.python.org related to some particular commit (plain 
search for 123f2dc0 didn?t find anything).

Any thoughts?

Mat?j

P.S.: Crossposting to python-devel in hope there would be 
somebody understanding more about that particular commit. For 
that I have also intentionally not trim the original messages to 
preserve context.

-- 
http://www.ceplovi.cz/matej/, Jabber: mcepl<at>ceplovi.cz
GPG Finger: 89EF 4BC6 288A BF43 1BAB  25C3 E09F EF25 D964 84AC
 
When you're happy that cut and paste actually works I think it's
a sign you've been using X-Windows for too long.
   -- from /. discussion on poor integration between KDE and
      GNOME
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 190 bytes
Desc: This is a digitally signed message part
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130306/e2b13a5e/attachment.pgp>

From ncoghlan at gmail.com  Wed Mar  6 14:50:34 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 6 Mar 2013 23:50:34 +1000
Subject: [Python-Dev] built-in Python test runner (was: Python Language
 Summit at PyCon: Agenda)
In-Reply-To: <CAL0kPAW4G=r35-ax2arPPm1f90C56_pjH8HG_7AU7ReD=QEgzg@mail.gmail.com>
References: <CAF-Rda_KYkTZQ8t+70NLfD78=YrYnjLBr7kuR=K3GCXGWXSGPw@mail.gmail.com>
	<CAJ3HoZ1K1fMLrN=sRp4p=Ty-EiLKBJgYk5Tmms7zUh78zLOLRQ@mail.gmail.com>
	<20130304171439.7c37c840@anarchist.wooz.org>
	<CAF-Rda-kJH6WGhjrXNt6Y8wVXqpCBRGTLS0dYE=3MKsz23+zog@mail.gmail.com>
	<CAJ3HoZ2rwFdizn1eGFPggkZU2oGWcU-35NZYoPK71s5_1pHbXQ@mail.gmail.com>
	<CAF-Rda_AvRf4GC7KcmQNJhpEk77e31m+vsK8SzZD9CNw+SMPZg@mail.gmail.com>
	<CAJ3HoZ0WN98M6MT+YAc31xGiu6KO=ySYyD3Wk85UxM52mSUL1Q@mail.gmail.com>
	<CAL0kPAWMrCgGMnaJC0r9ShUc17cM+=qWWQJ+jKwQz__a85ONxg@mail.gmail.com>
	<CADiSq7fjfuqF2bnbZZfii1ru_aYuSub83vPSrY8vNYAsWx0SHw@mail.gmail.com>
	<CAL0kPAW4G=r35-ax2arPPm1f90C56_pjH8HG_7AU7ReD=QEgzg@mail.gmail.com>
Message-ID: <CADiSq7drLoctvF5iyNVHJR2o01o6XUCnu9Kk7JD7YHMvEujh5w@mail.gmail.com>

On Wed, Mar 6, 2013 at 4:45 PM, Lennart Regebro <regebro at gmail.com> wrote:
> Perhaps we can solve this outside distutils-sig so that distutils-sig
> can concentrate on the harder problems?

It's a distutils-sig problem because you need a way to publish any new
testing related metadata, and because we're planning to evolve a hooks
system to cover the different steps in the build process in a
decoupled manner. "Run the tests" will be just another hook, but we're
not up to dealing with that yet (the only hook that will be in
metadata 2.0 is the post-install hook that will bring the wheel format
up to the point of being a near-total replacement for "./setup.py
install", and even that isn't written up formally yet - it's just a
post in a thread on distutils-sig).

You could, as Daniel suggested, work on defining a PEP 426 extension
as a prototype concept, but it won't help you much until PEP 426
support is widespread, and by then we'll probably be looking at the
meta-build system more broadly and figuring out the full set of
desired hooks (including test invocation).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rdmurray at bitdance.com  Wed Mar  6 14:51:20 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 06 Mar 2013 08:51:20 -0500
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
	Swartz memorial)
In-Reply-To: <1362575394.23949.2.camel@wycliff.ceplovi.cz>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
Message-ID: <20130306135121.65795250BD8@webabinitio.net>

On Wed, 06 Mar 2013 14:09:54 +0100, =?UTF-8?Q?Mat=C4=9Bj?= Cepl <mcepl at redhat.com> wrote:
> So, in the end, I have went the long way and bisected cpython to 
> find the commit which broke my tests, and it seems that the 
> culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is 
> clearly something Unicode related.
> 
> Unfortunately, it really doesn't tell me what exactly is broken 
> (is it a known regression) and if there is known workaround.  
> Could anybody suggest a way how to find bugs on 
> http://bugs.python.org related to some particular commit (plain 
> search for 123f2dc0 didn???t find anything).

If no issue number is mentioned in the commit message, then chances
are there's no specific issue in the tracker related to that particular
commit.  Normally there will be an issue, but sometimes things are done
without one (a practice we should maybe think about changing).

Most likely the commit's author, Victor Stinner, will see your message
or this one and respond.  That particular change recently came up (by
implication) in another context (unicode singletons...)

--David

From amauryfa at gmail.com  Wed Mar  6 15:18:30 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 6 Mar 2013 15:18:30 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
 Swartz memorial)
In-Reply-To: <1362575394.23949.2.camel@wycliff.ceplovi.cz>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
Message-ID: <CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>

Hi,

2013/3/6 Mat?j Cepl <mcepl at redhat.com>

>
> On 2013-02-26, 16:25 GMT, Terry Reedy wrote:
> > On 2/21/2013 4:22 PM, Matej Cepl wrote:
> >> as my method to commemorate Aaron Swartz, I have decided to port his
> >> html2text to work fully with the latest python 3.3. After some time
> >> dealing with various bugs, I have now in my repo
> >> https://github.com/mcepl/html2text (branch python3) working solution
> >> which works all the way to python 3.2 (inclusive;
> >> https://travis-ci.org/mcepl/html2text). However, the last problem
> >> remains. This
> >>
> >> <li>Run this command:
> >> <pre>ls -l *.html</pre></li>
> >> <li>?</li>
> >>
> >> should lead to
> >>
> >>    * Run this command:
> >>
> >>          ls -l *.html
> >>
> >>    * ?
> >>
> >> but it doesn?t. It leads to this (with python 3.3 only)
> >>
> >>      * Run this command:
> >>            ls -l *.html
> >>
> >>      * ?
> >>
> >> Does anybody know about something which changed in modules re or
> >> http://docs.python.org/3.3/whatsnew/changelog.html between 3.2 and
> >> 3.3, which could influence this script?
> >
> > Search the changelob or 3.3 misc/News for items affecting those two
> > modules. There are at least 4.
> > http://docs.python.org/3.3/whatsnew/changelog.html
> >
> > It is faintly possible that the switch from narrow/wide builds to
> > unified builds somehow affected that. Have you tested with 2.7/3.2 on
> > both narrow and wide unicode builds?
>
> So, in the end, I have went the long way and bisected cpython to
> find the commit which broke my tests, and it seems that the
> culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is
> clearly something Unicode related.
>
> Unfortunately, it really doesn't tell me what exactly is broken
> (is it a known regression) and if there is known workaround.
> Could anybody suggest a way how to find bugs on
> http://bugs.python.org related to some particular commit (plain
> search for 123f2dc0 didn?t find anything).
>

I strongly suspect an incorrect usage of the "is" operator:
https://github.com/mcepl/html2text/blob/master/html2text.py#L95
Identity of strings is not guaranteed...

Does it change something if you use "==" instead?

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130306/5377c32c/attachment.html>

From ezio.melotti at gmail.com  Wed Mar  6 15:40:42 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 6 Mar 2013 16:40:42 +0200
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh759h$v7i$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
Message-ID: <CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>

Hi,

On Wed, Mar 6, 2013 at 12:20 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Clicking this link
> http://www.microsoft.com/en-us/download/details.aspx?id=14597
> on this Developer Guide page
> http://docs.python.org/devguide/setup.html#windows
> now returns a
> "We are sorry, the page you requested cannot be found."
> page with search results.
>
> The first search result
> http://social.msdn.microsoft.com/Forums/nl/Vsexpressinstall/thread/2dc7ae6a-a0e7-436b-a1b3-3597ffac6a97
> suggests that one must first go to http://profile.microsoft.com
> which forwards to the live.com login page. Logging in with my un-expired
> non-developer account did not make the original link work.
>
> The mdsn page http://msdn.microsoft.com/en-US/
> has Visual Studio / Download trial, which leads to
> https://www.microsoft.com/visualstudio/eng/downloads
> which lists 2012 and 2010 but not 2008.
>
> I suspect that an msdn account is required for most people to get 2008.
>
> A later link leads to
> https://www.dreamspark.com/Product/Product.aspx?productid=34#
> which suggests that vc++2008 express is also available to verified degree
> students. I don't qualify so I will not try.
>

I did try a few weeks ago, when I had to download a copy of Windows
for a project.  Long story short, after 30+ minutes and a number of
confirmation emails I reached a point where I had a couple of new
accounts on MSDN/Dreamspark, a "purchased" free copy of Windows in my
e-cart, and some .exe I had to download in order to download and
verify the purchased copy.  That's where I gave up.

Best Regards,
Ezio Melotti


> So it would appear that section "1.1.3.3. Windows" of "1. Getting Started"
> (setup.rst) needs further revision.
>
> Or perhaps we could persuade Microsoft to let us distribute it ourselves so
> Windows versions of 2.7 do not become increasingly unusable.
>
> --
> Terry Jan Reedy
>

From python at mrabarnett.plus.com  Wed Mar  6 17:22:15 2013
From: python at mrabarnett.plus.com (MRAB)
Date: Wed, 06 Mar 2013 16:22:15 +0000
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
 Swartz memorial)
In-Reply-To: <CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
	<CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
Message-ID: <51376D37.4040209@mrabarnett.plus.com>

On 2013-03-06 14:18, Amaury Forgeot d'Arc wrote:
> Hi,
>
> 2013/3/6 Mat?j Cepl <mcepl at redhat.com <mailto:mcepl at redhat.com>>
>
>
>     On 2013-02-26, 16:25 GMT, Terry Reedy wrote:
>      > On 2/21/2013 4:22 PM, Matej Cepl wrote:
>      >> as my method to commemorate Aaron Swartz, I have decided to port his
>      >> html2text to work fully with the latest python 3.3. After some time
>      >> dealing with various bugs, I have now in my repo
>      >> https://github.com/mcepl/html2text (branch python3) working solution
>      >> which works all the way to python 3.2 (inclusive;
>      >> https://travis-ci.org/mcepl/html2text). However, the last problem
>      >> remains. This
>      >>
>      >> <li>Run this command:
>      >> <pre>ls -l *.html</pre></li>
>      >> <li>?</li>
>      >>
>      >> should lead to
>      >>
>      >>    * Run this command:
>      >>
>      >>          ls -l *.html
>      >>
>      >>    * ?
>      >>
>      >> but it doesn?t. It leads to this (with python 3.3 only)
>      >>
>      >>      * Run this command:
>      >>            ls -l *.html
>      >>
>      >>      * ?
>      >>
>      >> Does anybody know about something which changed in modules re or
>      >> http://docs.python.org/3.3/whatsnew/changelog.html between 3.2 and
>      >> 3.3, which could influence this script?
>      >
>      > Search the changelob or 3.3 misc/News for items affecting those two
>      > modules. There are at least 4.
>      > http://docs.python.org/3.3/whatsnew/changelog.html
>      >
>      > It is faintly possible that the switch from narrow/wide builds to
>      > unified builds somehow affected that. Have you tested with 2.7/3.2 on
>      > both narrow and wide unicode builds?
>
>     So, in the end, I have went the long way and bisected cpython to
>     find the commit which broke my tests, and it seems that the
>     culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is
>     clearly something Unicode related.
>
>     Unfortunately, it really doesn't tell me what exactly is broken
>     (is it a known regression) and if there is known workaround.
>     Could anybody suggest a way how to find bugs on
>     http://bugs.python.org related to some particular commit (plain
>     search for 123f2dc0 didn?t find anything).
>
>
> I strongly suspect an incorrect usage of the "is" operator:
> https://github.com/mcepl/html2text/blob/master/html2text.py#L95
> Identity of strings is not guaranteed...
>
> Does it change something if you use "==" instead?
>
That function looks a little odd to me. Maybe I just don't understand
what it's doing! :-)

From rosuav at gmail.com  Wed Mar  6 17:30:14 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 Mar 2013 03:30:14 +1100
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
Message-ID: <CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>

On Thu, Mar 7, 2013 at 1:40 AM, Ezio Melotti <ezio.melotti at gmail.com> wrote:
> I did try a few weeks ago, when I had to download a copy of Windows
> for a project.  Long story short, after 30+ minutes and a number of
> confirmation emails I reached a point where I had a couple of new
> accounts on MSDN/Dreamspark, a "purchased" free copy of Windows in my
> e-cart, and some .exe I had to download in order to download and
> verify the purchased copy.  That's where I gave up.

That's the point where I'd start looking at peer-to-peer downloads.
These sorts of things are often available on torrent sites; once the
original publisher starts making life harder, third-party sources
become more attractive.

ChrisA

From stefan_ml at behnel.de  Wed Mar  6 17:46:46 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 06 Mar 2013 17:46:46 +0100
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
	<CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
Message-ID: <kh7rti$5na$1@ger.gmane.org>

Chris Angelico, 06.03.2013 17:30:
> On Thu, Mar 7, 2013 at 1:40 AM, Ezio Melotti wrote:
>> I did try a few weeks ago, when I had to download a copy of Windows
>> for a project.  Long story short, after 30+ minutes and a number of
>> confirmation emails I reached a point where I had a couple of new
>> accounts on MSDN/Dreamspark, a "purchased" free copy of Windows in my
>> e-cart, and some .exe I had to download in order to download and
>> verify the purchased copy.  That's where I gave up.
> 
> That's the point where I'd start looking at peer-to-peer downloads.
> These sorts of things are often available on torrent sites; once the
> original publisher starts making life harder, third-party sources
> become more attractive.

May I express my doubts that the license allows a redistribution of the
software in this form?

Stefan



From rosuav at gmail.com  Wed Mar  6 17:55:40 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 Mar 2013 03:55:40 +1100
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh7rti$5na$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
	<CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
	<kh7rti$5na$1@ger.gmane.org>
Message-ID: <CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>

On Thu, Mar 7, 2013 at 3:46 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> Chris Angelico, 06.03.2013 17:30:
>> On Thu, Mar 7, 2013 at 1:40 AM, Ezio Melotti wrote:
>>> I did try a few weeks ago, when I had to download a copy of Windows
>>> for a project.  Long story short, after 30+ minutes and a number of
>>> confirmation emails I reached a point where I had a couple of new
>>> accounts on MSDN/Dreamspark, a "purchased" free copy of Windows in my
>>> e-cart, and some .exe I had to download in order to download and
>>> verify the purchased copy.  That's where I gave up.
>>
>> That's the point where I'd start looking at peer-to-peer downloads.
>> These sorts of things are often available on torrent sites; once the
>> original publisher starts making life harder, third-party sources
>> become more attractive.
>
> May I express my doubts that the license allows a redistribution of the
> software in this form?

Someone would have to check, but in most cases, software licenses
govern the use, more than the distribution. If you're allowed to
download it free of charge from microsoft.com, you should be able to
get hold of it in some other way and it be exactly the same. But yeah,
if you want to be legal you'd have to actually read the EULA.

Is there any plan for future Python versions to use a free compiler on
Windows? That would eliminate this issue, but presumably would create
others.

ChrisA

From brian at python.org  Wed Mar  6 18:03:11 2013
From: brian at python.org (Brian Curtin)
Date: Wed, 6 Mar 2013 11:03:11 -0600
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
	<CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
	<kh7rti$5na$1@ger.gmane.org>
	<CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
Message-ID: <CAD+XWwru7C+fzv8AT_tuk99zFVwz9fMcQahWWJQSSL+s=580TA@mail.gmail.com>

On Wed, Mar 6, 2013 at 10:55 AM, Chris Angelico <rosuav at gmail.com> wrote:
> Is there any plan for future Python versions to use a free compiler on
> Windows? That would eliminate this issue, but presumably would create
> others.

No plan, although there are at times patches/issues floating around to
add some level of support for MinGW (or something like it) in addition
to Microsoft's compiler.

From casevh at gmail.com  Wed Mar  6 18:00:04 2013
From: casevh at gmail.com (Case Van Horsen)
Date: Wed, 6 Mar 2013 09:00:04 -0800
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh759h$v7i$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
Message-ID: <CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>

On Wed, Mar 6, 2013 at 2:20 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Clicking this link
> http://www.microsoft.com/en-us/download/details.aspx?id=14597
> on this Developer Guide page
> http://docs.python.org/devguide/setup.html#windows
> now returns a
> "We are sorry, the page you requested cannot be found."
> page with search results.
>
> The first search result
> http://social.msdn.microsoft.com/Forums/nl/Vsexpressinstall/thread/2dc7ae6a-a0e7-436b-a1b3-3597ffac6a97
> suggests that one must first go to http://profile.microsoft.com
> which forwards to the live.com login page. Logging in with my un-expired
> non-developer account did not make the original link work.
>
> The mdsn page http://msdn.microsoft.com/en-US/
> has Visual Studio / Download trial, which leads to
> https://www.microsoft.com/visualstudio/eng/downloads
> which lists 2012 and 2010 but not 2008.
>
> I suspect that an msdn account is required for most people to get 2008.
>
> A later link leads to
> https://www.dreamspark.com/Product/Product.aspx?productid=34#
> which suggests that vc++2008 express is also available to verified degree
> students. I don't qualify so I will not try.
>
> So it would appear that section "1.1.3.3. Windows" of "1. Getting Started"
> (setup.rst) needs further revision.
>
> Or perhaps we could persuade Microsoft to let us distribute it ourselves so
> Windows versions of 2.7 do not become increasingly unusable.

The "Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1"
is still available for download. It includes the command line
compilers that are used with VS 2008. I have used to create extensions
for Python 2.6 to 3.2. There is a later version of the SDK (for .NET
4.x) that includes the compilers from VS 2010.

To use the SDK compiler, you need to do a few manual steps first.

After starting a command window, you need to run a batch file to
configure your environment. Choose the appropriate option from

C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat

or

C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat

Then set two environment variables:

set MSSdk=1
set DISTUTILS_USE_SDK=1


After these steps, the standard python setup.py install should work.

casevh
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com

From robert.kern at gmail.com  Wed Mar  6 18:16:28 2013
From: robert.kern at gmail.com (Robert Kern)
Date: Wed, 06 Mar 2013 17:16:28 +0000
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
	<CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
	<kh7rti$5na$1@ger.gmane.org>
	<CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
Message-ID: <kh7tl9$ps9$1@ger.gmane.org>

On 2013-03-06 16:55, Chris Angelico wrote:
> On Thu, Mar 7, 2013 at 3:46 AM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>> Chris Angelico, 06.03.2013 17:30:
>>> On Thu, Mar 7, 2013 at 1:40 AM, Ezio Melotti wrote:
>>>> I did try a few weeks ago, when I had to download a copy of Windows
>>>> for a project.  Long story short, after 30+ minutes and a number of
>>>> confirmation emails I reached a point where I had a couple of new
>>>> accounts on MSDN/Dreamspark, a "purchased" free copy of Windows in my
>>>> e-cart, and some .exe I had to download in order to download and
>>>> verify the purchased copy.  That's where I gave up.
>>>
>>> That's the point where I'd start looking at peer-to-peer downloads.
>>> These sorts of things are often available on torrent sites; once the
>>> original publisher starts making life harder, third-party sources
>>> become more attractive.
>>
>> May I express my doubts that the license allows a redistribution of the
>> software in this form?
>
> Someone would have to check, but in most cases, software licenses
> govern the use, more than the distribution. If you're allowed to
> download it free of charge from microsoft.com, you should be able to
> get hold of it in some other way and it be exactly the same.

Sorry, but that's not how copyright works. The owner of the copyright on a work 
has to give you permission to allow you to distribute their work (modulo certain 
statutorily-defined exceptions that don't apply here). Just because you got the 
work from them free of charge doesn't mean that they have given you permission 
to redistribute it. If the agreements that you have with the copyright owner do 
not mention redistribution, you do not have permission to redistribute it.

IANAL, TINLA.

-- 
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
  that is made terrible by our own mad attempt to interpret it as though it had
  an underlying truth."
   -- Umberto Eco


From Steve.Dower at microsoft.com  Wed Mar  6 18:13:49 2013
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Wed, 6 Mar 2013 17:13:49 +0000
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh759h$v7i$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
Message-ID: <47cc440a8ac74ef8b82dec8b1b961ed1@BLUPR03MB035.namprd03.prod.outlook.com>

From: Terry Reedy
> Clicking this link
> http://www.microsoft.com/en-us/download/details.aspx?id=14597
> on this Developer Guide page
> http://docs.python.org/devguide/setup.html#windows
> now returns a
> "We are sorry, the page you requested cannot be found."
> page with search results.
> 
> The first search result
> http://social.msdn.microsoft.com/Forums/nl/Vsexpressinstall/thread/2dc7a
> e6a-a0e7-436b-a1b3-3597ffac6a97
> suggests that one must first go to http://profile.microsoft.com which
> forwards to the live.com login page. Logging in with my un-expired non-
> developer account did not make the original link work.
> 
> The mdsn page http://msdn.microsoft.com/en-US/ has Visual Studio /
> Download trial, which leads to
> https://www.microsoft.com/visualstudio/eng/downloads
> which lists 2012 and 2010 but not 2008.
> 
> I suspect that an msdn account is required for most people to get 2008.

Worse than that, it looks like you need a subscription and then a download "helper", which will get you the web installer that then goes off and downloads it for you.

> A later link leads to
> https://www.dreamspark.com/Product/Product.aspx?productid=34#
> which suggests that vc++2008 express is also available to verified degree
> students. I don't qualify so I will not try.
>
> So it would appear that section "1.1.3.3. Windows" of "1. Getting Started"
> (setup.rst) needs further revision.
> 
> Or perhaps we could persuade Microsoft to let us distribute it ourselves so
> Windows versions of 2.7 do not become increasingly unusable.

I'll ask around and see what we can do. We clearly still have the download available, so it may just be a case of making the web installer publicly available again. Chances are if you have the installer then it will still work.

We may also make just the compilers available in some other way. It looks like the Windows Development Kits (previously Platform SDK) don't have it, but IIRC the driver kits occasionally ship with compilers.

I'll get back to the list when I get something.

Cheers,
Steve



From eliben at gmail.com  Wed Mar  6 18:19:23 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 6 Mar 2013 09:19:23 -0800
Subject: [Python-Dev] [docs] undocumented argtypes magic in ctypes?
In-Reply-To: <CAL3CFcUoKG=+vaTBMs6nSbgz_gJgxLX3_uK94JLSyEpOY7iy_Q@mail.gmail.com>
References: <CAF-Rda9T=LfviFKqT-qkNDPP9ba-Kg=M41RLac_miA99dXjOAA@mail.gmail.com>
	<CAL3CFcUoKG=+vaTBMs6nSbgz_gJgxLX3_uK94JLSyEpOY7iy_Q@mail.gmail.com>
Message-ID: <CAF-Rda_Gs0GT55+=HimkUVa5icS0X=vWBHug8iByq9=C1YEsmA@mail.gmail.com>

On Wed, Mar 6, 2013 at 8:33 AM, Andrew Svetlov <andrew.svetlov at gmail.com>wrote:

> Looks like bug for me.
>

ctypes seems to auto-convert arguments when argtypes is specified. This
fact is documented. However, I'm not sure whether this auto-conversion is
advanced enough to apply byref. Because otherwise, DIRENT is certainly not
convertible to DIRENT_p

Eli




>
> On Tue, Mar 5, 2013 at 4:26 PM, Eli Bendersky <eliben at gmail.com> wrote:
> > Hello,
> >
> > While playing with ctypes a bit, I noticed a feature that doesn't appear
> to
> > be documented. Suppose I import the readdir_r function (assuming DIRENT
> is a
> > correctly declared ctypes.Structure):
> >
> > DIR_p = c_void_p
> > DIRENT_p = POINTER(DIRENT)
> > DIRENT_pp = POINTER(DIRENT_p)
> >
> > readdir_r = lib.readdir_r
> > readdir_r.argtypes = [DIR_p, DIRENT_p, DIRENT_pp]
> > readdir_r.restype = c_int
> >
> > It seems that I can then call it as follows:
> >
> > dirent = DIRENT()
> > result = DIRENT_p()
> >
> > readdir_r(dir_fd, dirent, result)
> >
> > Note that while readdir_r takes DIRENT_p and DIRENT_pp as its second and
> > third args, I pass in just DIRENT and DIRENT_p, accordingly. What I
> should
> > have done is use byref() on both, but ctypes seems to have some magic
> > applied when argtypes declares pointer types. If I use byref, it still
> > works. However, if I keep the same call and comment out the argtypes
> > declaration, I get a segfault.
> >
> > So, is it a feature that should be documented, explicitly discouraged or
> is
> > it a bug?
> >
> > Eli
> >
> >
> > _______________________________________________
> > docs mailing list
> > docs at python.org
> > http://mail.python.org/mailman/listinfo/docs
> >
>
>
>
> --
> Thanks,
> Andrew Svetlov
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130306/d87ac147/attachment.html>

From Steve.Dower at microsoft.com  Wed Mar  6 18:29:11 2013
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Wed, 6 Mar 2013 17:29:11 +0000
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>
Message-ID: <fa3ed1f62dd44f4abaf6debbe9b5eabf@BLUPR03MB035.namprd03.prod.outlook.com>

From: Case Van Horsen
> On Wed, Mar 6, 2013 at 2:20 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> > Clicking this link
> > http://www.microsoft.com/en-us/download/details.aspx?id=14597
> > on this Developer Guide page
> > http://docs.python.org/devguide/setup.html#windows
> > now returns a
> > "We are sorry, the page you requested cannot be found."
> > page with search results.
> >
> > The first search result
> > http://social.msdn.microsoft.com/Forums/nl/Vsexpressinstall/thread/2dc
> > 7ae6a-a0e7-436b-a1b3-3597ffac6a97 suggests that one must first go to
> > http://profile.microsoft.com which forwards to the live.com login
> > page. Logging in with my un-expired non-developer account did not make
> > the original link work.
> >
> > The mdsn page http://msdn.microsoft.com/en-US/ has Visual Studio /
> > Download trial, which leads to
> > https://www.microsoft.com/visualstudio/eng/downloads
> > which lists 2012 and 2010 but not 2008.
> >
> > I suspect that an msdn account is required for most people to get 2008.
> >
> > A later link leads to
> > https://www.dreamspark.com/Product/Product.aspx?productid=34#
> > which suggests that vc++2008 express is also available to verified
> > degree students. I don't qualify so I will not try.
> >
> > So it would appear that section "1.1.3.3. Windows" of "1. Getting Started"
> > (setup.rst) needs further revision.
> >
> > Or perhaps we could persuade Microsoft to let us distribute it
> > ourselves so Windows versions of 2.7 do not become increasingly
> unusable.
> 
> The "Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1"
> is still available for download. It includes the command line compilers that are
> used with VS 2008. I have used to create extensions for Python 2.6 to 3.2.
> There is a later version of the SDK (for .NET
> 4.x) that includes the compilers from VS 2010.

This is the same response that I got internally.

The download link is http://www.microsoft.com/en-us/download/details.aspx?id=3138 and you can choose to only download and install the compilers.

Cheers,
Steve

> To use the SDK compiler, you need to do a few manual steps first.
> 
> After starting a command window, you need to run a batch file to configure
> your environment. Choose the appropriate option from
> 
> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat
> 
> or
> 
> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat
> 
> Then set two environment variables:
> 
> set MSSdk=1
> set DISTUTILS_USE_SDK=1
> 
> 
> After these steps, the standard python setup.py install should work.
> 
> casevh
> >
> > --
> > Terry Jan Reedy



From victor.stinner at gmail.com  Wed Mar  6 19:34:10 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Wed, 6 Mar 2013 19:34:10 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
 Swartz memorial)
In-Reply-To: <CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
	<CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
Message-ID: <CAMpsgwZ8ryt1XO-FHuVZHEtVO4uXsMtyRA1VjJEpOjBsAnXq6Q@mail.gmail.com>

Hi,

In short, Unicode was rewritten in Python 3.3 for the PEP 393. It's
not surprising that minor details like singleton differ. You should
not use "is" to compare strings in Python, or your program will fail
on other Python implementations (like PyPy, IronPython, or Jython) or
even on a different CPython version.

Anyway, you spotted a missed optimization: it's now "fixed" in Python
3.3 and 3.4 by the following commits. Copy/paste of the CIA IRC bot:

19:30 < irker555> cpython: Victor Stinner 3.3 * 82517:3dd2fa78fb89 /
Objects/unicodeobject.c:
                  _PyUnicode_Writer() now also reuses Unicode
singletons: empty string and latin1
                  single character http://hg.python.org/cpython/rev/3dd2fa78fb89
19:30 < irker032> cpython: Victor Stinner default * 82518:fa59a85b373f
/ Objects/unicodeobject.c:
                  (Merge 3.3) _PyUnicode_Writer() now also reuses
Unicode singletons: empty string and
                  latin1 single character
http://hg.python.org/cpython/rev/fa59a85b373f

Victor

2013/3/6 Amaury Forgeot d'Arc <amauryfa at gmail.com>:
>> So, in the end, I have went the long way and bisected cpython to
>> find the commit which broke my tests, and it seems that the
>> culprit is http://hg.python.org/cpython/rev/123f2dc08b3e so it is
>> clearly something Unicode related.
>>
>> Unfortunately, it really doesn't tell me what exactly is broken
>> (is it a known regression) and if there is known workaround.
>> Could anybody suggest a way how to find bugs on
>> http://bugs.python.org related to some particular commit (plain
>> search for 123f2dc0 didn?t find anything).
>
>
> I strongly suspect an incorrect usage of the "is" operator:
> https://github.com/mcepl/html2text/blob/master/html2text.py#L95
> Identity of strings is not guaranteed...
>
> Does it change something if you use "==" instead?
>
> --
> Amaury Forgeot d'Arc

From steve at pearwood.info  Wed Mar  6 21:06:15 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 07 Mar 2013 07:06:15 +1100
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
Message-ID: <5137A1B7.6020002@pearwood.info>

On 05/03/13 09:08, Brett Cannon wrote:

> Depends on your paranoia. If you're worried about accidentally lifting IP
> merely by reading someone's source code, then you wouldn't want to touch
> code without the CLA signed.
>
> Now I'm not that paranoid, but I'm still not about to commit someone's code
> now without the CLA signed to make sure we are legally covered for the
> patch. If someone chooses not to contribute because of the CLA that's fine,
> but since we have already told at least Anatoly that we won't accept
> patches from him until he signs the CLA I'm not going to start acting
> differently towards others. I view legally covering our ass by having
> someone fill in a form is worth the potential loss of some contribution in
> the grand scheme of things.

Pardon my ignorance, but how does a CLA protect us in the event of an IP
violation?




-- 
Steven

From brett at python.org  Wed Mar  6 21:28:29 2013
From: brett at python.org (Brett Cannon)
Date: Wed, 6 Mar 2013 15:28:29 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <5137A1B7.6020002@pearwood.info>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
	<5137A1B7.6020002@pearwood.info>
Message-ID: <CAP1=2W4_m4efv_h=jbBLpMe-UPFmihGjzxXQ37OX=jWbkvksig@mail.gmail.com>

On Wed, Mar 6, 2013 at 3:06 PM, Steven D'Aprano <steve at pearwood.info> wrote:

> On 05/03/13 09:08, Brett Cannon wrote:
>
>  Depends on your paranoia. If you're worried about accidentally lifting IP
>> merely by reading someone's source code, then you wouldn't want to touch
>> code without the CLA signed.
>>
>> Now I'm not that paranoid, but I'm still not about to commit someone's
>> code
>> now without the CLA signed to make sure we are legally covered for the
>> patch. If someone chooses not to contribute because of the CLA that's
>> fine,
>> but since we have already told at least Anatoly that we won't accept
>> patches from him until he signs the CLA I'm not going to start acting
>> differently towards others. I view legally covering our ass by having
>> someone fill in a form is worth the potential loss of some contribution in
>> the grand scheme of things.
>>
>
> Pardon my ignorance, but how does a CLA protect us in the event of an IP
> violation?


Maybe it doesn't. IANAL and I was just trying to think in as paranoid of a
fashion as I could.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130306/b4c7a121/attachment.html>

From stephen at xemacs.org  Wed Mar  6 23:06:10 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 07 Mar 2013 07:06:10 +0900
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <5137A1B7.6020002@pearwood.info>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
	<5137A1B7.6020002@pearwood.info>
Message-ID: <87r4jsi3ot.fsf@uwakimon.sk.tsukuba.ac.jp>

Steven D'Aprano writes:

 > Pardon my ignorance, but how does a CLA protect us in the event of an IP
 > violation?

By licensing the content to the PSF, the contributor implicitly claims
that he has the right to do so (I think the AFL even has an explicit
provenance clause).  This protects the PSF against criminal
infringement and statutory damages for copyright violation (which
require wilful infringement).

I don't know it if helps for patent infringement.

From tjreedy at udel.edu  Wed Mar  6 23:43:15 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 06 Mar 2013 17:43:15 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <5137A1B7.6020002@pearwood.info>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <kh33sr$b2k$1@ger.gmane.org>
	<CAP1=2W6B-7JZ=4jgqEdeJWfT2Q-iu8OP8R6a3AzZRptmLb2aKg@mail.gmail.com>
	<5137A1B7.6020002@pearwood.info>
Message-ID: <kh8gq7$osp$1@ger.gmane.org>

On 3/6/2013 3:06 PM, Steven D'Aprano wrote:
> On 05/03/13 09:08, Brett Cannon wrote:
>
>> Depends on your paranoia. If you're worried about accidentally lifting IP
>> merely by reading someone's source code, then you wouldn't want to touch
>> code without the CLA signed.
>>
>> Now I'm not that paranoid, but I'm still not about to commit someone's
>> code
>> now without the CLA signed to make sure we are legally covered for the
>> patch. If someone chooses not to contribute because of the CLA that's
>> fine,
>> but since we have already told at least Anatoly that we won't accept
>> patches from him until he signs the CLA I'm not going to start acting
>> differently towards others. I view legally covering our ass by having
>> someone fill in a form is worth the potential loss of some
>> contribution in
>> the grand scheme of things.
>
> Pardon my ignorance, but how does a CLA protect us in the event of an IP
> violation?

The penalty for willful copyright violation (possible punitive damages) 
is higher than for inadvertent violation (typically, remove the 
offending code). In the CLA, contributors affirm that they will only 
contribute code they have a legal right to contribute. This makes it 
clear that PSF only wants legal code. We do not grab 3rd party code 
without author participation even if the license would seem to make it 
legal to do so.

Good repository software, including svn and hg, can trace every line to 
a specific commit. Commit messages typically have an issue number and 
credit (blame) any patch author other than the one making the commit. So 
any line should be traceable to a specific person and we should have a 
CLA for that person.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Wed Mar  6 23:52:27 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 06 Mar 2013 17:52:27 -0500
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
	<CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
	<kh7rti$5na$1@ger.gmane.org>
	<CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
Message-ID: <kh8hbf$u7b$1@ger.gmane.org>

On 3/6/2013 11:55 AM, Chris Angelico wrote:

> Someone would have to check, but in most cases, software licenses
> govern the use, more than the distribution. If you're allowed to
> download it free of charge from microsoft.com, you should be able to
> get hold of it in some other way and it be exactly the same. But yeah,
> if you want to be legal you'd have to actually read the EULA.

As I remember, the 2008 vcexpress license specifically prohibits 
redistribtion even though MS gave it away for free. So we can not 
document other means of obtaining it. We went through the same issue 
with vc2005 when that was pulled from the MS site. I had the file but 
could not legally send it to anyone. As it is, my copy of 2008 file, 
which I meant to keep, seems gone (I believe the directory I had it in 
got corrupted).

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Mar  7 00:32:30 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 06 Mar 2013 18:32:30 -0500
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <fa3ed1f62dd44f4abaf6debbe9b5eabf@BLUPR03MB035.namprd03.prod.outlook.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>
	<fa3ed1f62dd44f4abaf6debbe9b5eabf@BLUPR03MB035.namprd03.prod.outlook.com>
Message-ID: <kh8jmi$j93$1@ger.gmane.org>

On 3/6/2013 12:29 PM, Steve Dower wrote:
> From: Case Van Horsen

>> The "Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1"
>> is still available for download. It includes the command line compilers that are
>> used with VS 2008. I have used to create extensions for Python 2.6 to 3.2.
>> There is a later version of the SDK (for .NET
>> 4.x) that includes the compilers from VS 2010.
>
> This is the same response that I got internally.
>
> The download link is
 > http://www.microsoft.com/en-us/download/details.aspx?id=3138
 > and you can choose to only download and install the compilers.

The C++ compiler appears to the the full compiler that will build both 
32 and 64 bits apps. Will downloading just the compiler(s) allow one to 
build Python with the project files in PCBuild or does something else 
need to be checked also?

>> To use the SDK compiler, you need to do a few manual steps first.
>>
>> After starting a command window, you need to run a batch file to configure
>> your environment. Choose the appropriate option from
>>
>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat
>>
>> or
>>
>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat
>>
>> Then set two environment variables:
>>
>> set MSSdk=1
>> set DISTUTILS_USE_SDK=1
>>
>> After these steps, the standard python setup.py install should work.

This may be fine for building extensions, but it appears that more 
instructions are needed for a novice to build python itself.

Following the instruction in the developer's guide,
http://docs.python.org/devguide/setup.html#windows
I was able to download and install vc express, double click on 
<branch_dir>/PCBuild/pcbuild.sln to bring up the VS GUI, and use the 
menu to build a debug version of that branch. The new python is put in 
the same directory and can be run with another menu selection. Any 
alternate path should be that easy too.

-- 
Terry Jan Reedy


From rosuav at gmail.com  Thu Mar  7 01:59:59 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Thu, 7 Mar 2013 11:59:59 +1100
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh8hbf$u7b$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
	<CACBhJdEbrF4WnCMfu46YoTns+Xry_gjp7iztk_+Z+5jMYbv1BQ@mail.gmail.com>
	<CAPTjJmrZDPE7tYaE99wrP-pXTz9Y4gkZhQ9t0oPFp2M8bEzacA@mail.gmail.com>
	<kh7rti$5na$1@ger.gmane.org>
	<CAPTjJmqkwFV6FOg9n9p7DKTV5FN5nwrO-VtPMh54XTtncQL06w@mail.gmail.com>
	<kh8hbf$u7b$1@ger.gmane.org>
Message-ID: <CAPTjJmp8ZgyAt-BzMySJm_2LXRdWw_Ws6ZVL3tKiG1wp6k1gMA@mail.gmail.com>

On Thu, Mar 7, 2013 at 9:52 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/6/2013 11:55 AM, Chris Angelico wrote:
>
>> Someone would have to check, but in most cases, software licenses
>> govern the use, more than the distribution. If you're allowed to
>> download it free of charge from microsoft.com, you should be able to
>> get hold of it in some other way and it be exactly the same. But yeah,
>> if you want to be legal you'd have to actually read the EULA.
>
>
> As I remember, the 2008 vcexpress license specifically prohibits
> redistribtion even though MS gave it away for free. So we can not document
> other means of obtaining it. We went through the same issue with vc2005 when
> that was pulled from the MS site. I had the file but could not legally send
> it to anyone. As it is, my copy of 2008 file, which I meant to keep, seems
> gone (I believe the directory I had it in got corrupted).

Blah. Okay, that settles that, then.

Of course, everything I said above is still possible, just not
something the PSF will officially condone.

ChrisA

From tjreedy at udel.edu  Thu Mar  7 02:08:03 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 06 Mar 2013 20:08:03 -0500
Subject: [Python-Dev] Introducing Electronic Contributor Agreements
In-Reply-To: <20130304214624.0f6dec8c@pitrou.net>
References: <CAD+XWwpdGs+niJDLJ=kjAa4o8aOpz5fa=0EM-0yZHwcEdYJu6Q@mail.gmail.com>
	<CAP1=2W7b07tMo7BThofVcy4F9L47WFPpSN9SHjBp5c+wrtXEpA@mail.gmail.com>
	<kh317n$ifo$1@ger.gmane.org> <20130304214624.0f6dec8c@pitrou.net>
Message-ID: <kh8p9n$o0$1@ger.gmane.org>

On 3/4/2013 3:46 PM, Antoine Pitrou wrote:
> On Mon, 04 Mar 2013 15:46:48 -0500
> Terry Reedy <tjreedy at udel.edu> wrote:

>> Either policy could be facilitated by tracker changes. In order to see
>> the file upload box, one must login and the tracker knows who has a CLA
>> on file (as indicated by a * suffix on the name). If a file is uploaded
>> by someone without, a box could popup with the link to the e-form and a
>> message that a CLA is required.
>
> And how about people who upload something else than a patch?

Restrict the popup to filenames ending in .diff or .patch.

-- 
Terry Jan Reedy


From casevh at gmail.com  Thu Mar  7 06:01:50 2013
From: casevh at gmail.com (Case Van Horsen)
Date: Wed, 6 Mar 2013 21:01:50 -0800
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh8jmi$j93$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
	<CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>
	<fa3ed1f62dd44f4abaf6debbe9b5eabf@BLUPR03MB035.namprd03.prod.outlook.com>
	<kh8jmi$j93$1@ger.gmane.org>
Message-ID: <CANerV6=_LZ3EEsamuTG9Gfy_E3rFzvrEqY2=E3H-_Ffnvv8+VQ@mail.gmail.com>

On Wed, Mar 6, 2013 at 3:32 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/6/2013 12:29 PM, Steve Dower wrote:
>>
>> From: Case Van Horsen
>
>
>>> The "Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1"
>>> is still available for download. It includes the command line compilers
>>> that are
>>> used with VS 2008. I have used to create extensions for Python 2.6 to
>>> 3.2.
>>> There is a later version of the SDK (for .NET
>>> 4.x) that includes the compilers from VS 2010.
>>
>>
>> This is the same response that I got internally.
>>
>> The download link is
>
>> http://www.microsoft.com/en-us/download/details.aspx?id=3138
>> and you can choose to only download and install the compilers.
>
> The C++ compiler appears to the the full compiler that will build both 32
> and 64 bits apps. Will downloading just the compiler(s) allow one to build
> Python with the project files in PCBuild or does something else need to be
> checked also?
>
>
>>> To use the SDK compiler, you need to do a few manual steps first.
>>>
>>> After starting a command window, you need to run a batch file to
>>> configure
>>> your environment. Choose the appropriate option from
>>>
>>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars64.bat
>>>
>>> or
>>>
>>> C:\Program Files (x86)\Microsoft Visual Studio 9.0\VC\bin\vcvars32.bat
>>>
>>> Then set two environment variables:
>>>
>>> set MSSdk=1
>>> set DISTUTILS_USE_SDK=1
>>>
>>> After these steps, the standard python setup.py install should work.
>
>
> This may be fine for building extensions, but it appears that more
> instructions are needed for a novice to build python itself.

There is a build.bat file in the PCbuild directory that will rebuild
Python from a command prompt. After entering the commands listed above
at a command prompt, I was able to build a debug version of Python
2.7.3 by moving to <branch_dir>\PCbuild and entering "build -d" (the
-d indicates a debug build).

>
> Following the instruction in the developer's guide,
> http://docs.python.org/devguide/setup.html#windows
> I was able to download and install vc express, double click on
> <branch_dir>/PCBuild/pcbuild.sln to bring up the VS GUI, and use the menu to
> build a debug version of that branch. The new python is put in the same
> directory and can be run with another menu selection. Any alternate path
> should be that easy too.

casevh

>
>
> --
> Terry Jan Reedy
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/casevh%40gmail.com

From stefan_ml at behnel.de  Thu Mar  7 08:32:59 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Thu, 07 Mar 2013 08:32:59 +0100
Subject: [Python-Dev] Add PyDict_GetItemSetDefault() as C-API for
	dict.setdefault()
Message-ID: <kh9fr7$g38$1@ger.gmane.org>

Hi,

I've written a patch that adds a new C-API call for dict.setdefault(). The
reason is that there is currently no way to test for a key and insert a
fallback value for it without either evaluating the hash function twice or
calling through the Python function. Both may involve considerable overhead
and the double hash may have side-effects.

http://bugs.python.org/issue17327

It does not include an explicit test because it does not add any code over
the normal dict.setdefault() implementation, which is already tested at the
Python level. If you prefer having a dummy test that checks that the
function is there, I don't mind adding one.

Please comment and/or apply the patch.

Thanks,

Stefan


From mcepl at redhat.com  Thu Mar  7 11:08:40 2013
From: mcepl at redhat.com (Matej Cepl)
Date: Thu, 7 Mar 2013 11:08:40 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
	Swartz memorial)
In-Reply-To: <CAMpsgwZ8ryt1XO-FHuVZHEtVO4uXsMtyRA1VjJEpOjBsAnXq6Q@mail.gmail.com>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
	<CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
	<CAMpsgwZ8ryt1XO-FHuVZHEtVO4uXsMtyRA1VjJEpOjBsAnXq6Q@mail.gmail.com>
Message-ID: <20130307100840.GA24941@wycliff.ceplovi.cz>

On 2013-03-06, 18:34 GMT, Victor Stinner wrote:
> In short, Unicode was rewritten in Python 3.3 for the PEP 393. It's
> not surprising that minor details like singleton differ. You should
> not use "is" to compare strings in Python, or your program will fail
> on other Python implementations (like PyPy, IronPython, or Jython) or
> even on a different CPython version.

I am sorry, I don't understand what you are saying. Even though 
this has been changed to 
https://github.com/mcepl/html2text/blob/fix_tests/html2text.py#L90 
the tests still fail.

But, Amaury is right: the function doesn't make much sense. 
However, ...

when I have ?fixed it? from
https://github.com/mcepl/html2text/blob/master/html2text.py#L95

def onlywhite(line):
     """Return true if the line does only consist of whitespace characters."""
     for c in line:
         if c is not ' ' and c is not '  ':
             return c is ' '
     return line

to 
https://github.com/mcepl/html2text/blob/fix_tests/html2text.py#L90

def onlywhite(line):
     """Return true if the line does only consist of whitespace 
     characters."""
         for c in line:
            if c != ' ' and c != ' ':
               return c == ' '
         return line

tests on ALL versions of Python are suddenly failing ...  
https://travis-ci.org/mcepl/html2text/builds/5288190

Curiouser and curiouser! At least, I seem to have the point,
where things are breaking, but I have to admit that condition
really doesn?t make any sense to me.

> Anyway, you spotted a missed optimization: it's now "fixed" in
> Python 3.3 and 3.4 by the following commits.

Well, whatever is the problem, it is not fixed in python 3.3.0
(as you can see in
https://travis-ci.org/mcepl/html2text/builds/4969045) as I can
see on my computer. Actually, good news is that it seems to be
fixed in the master branch of cpython (or the tip, as they say in
the Mercurial world).

Any thoughts?

Mat?j

From catch-all at masklinn.net  Thu Mar  7 11:31:03 2013
From: catch-all at masklinn.net (Xavier Morel)
Date: Thu, 7 Mar 2013 11:31:03 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
	Swartz memorial)
In-Reply-To: <20130307100840.GA24941@wycliff.ceplovi.cz>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
	<CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
	<CAMpsgwZ8ryt1XO-FHuVZHEtVO4uXsMtyRA1VjJEpOjBsAnXq6Q@mail.gmail.com>
	<20130307100840.GA24941@wycliff.ceplovi.cz>
Message-ID: <3EAD780A-61B6-4460-87C6-C2C7BF6F9CA1@masklinn.net>


On 2013-03-07, at 11:08 , Matej Cepl wrote:

> On 2013-03-06, 18:34 GMT, Victor Stinner wrote:
>> In short, Unicode was rewritten in Python 3.3 for the PEP 393. It's
>> not surprising that minor details like singleton differ. You should
>> not use "is" to compare strings in Python, or your program will fail
>> on other Python implementations (like PyPy, IronPython, or Jython) or
>> even on a different CPython version.
> 
> I am sorry, I don't understand what you are saying. Even though 
> this has been changed to 
> https://github.com/mcepl/html2text/blob/fix_tests/html2text.py#L90 
> the tests still fail.
> 
> But, Amaury is right: the function doesn't make much sense. 
> However, ...
> 
> when I have ?fixed it? from
> https://github.com/mcepl/html2text/blob/master/html2text.py#L95
> 
> def onlywhite(line):
>     """Return true if the line does only consist of whitespace characters."""
>     for c in line:
>         if c is not ' ' and c is not '  ':
>             return c is ' '
>     return line
> 
> to 
> https://github.com/mcepl/html2text/blob/fix_tests/html2text.py#L90
> 
> def onlywhite(line):
>     """Return true if the line does only consist of whitespace 
>     characters."""
>         for c in line:
>            if c != ' ' and c != ' ':
>               return c == ' '
>         return line

The second test looks like some kind of corruption, it's supposedly
iterating on the characters of a line yet testing for two spaces? Is it
possible that the original was a literal tab embedded in the source code
(instead of '\t') and that got broken at some point?

According to its name + docstring, the implementation of this method
should really be replaced by `return line and line.isspace()` (the first
part being to handle the case of an empty line: in the current
implementation the line will be returned directly if no whitespace is
found, which will be "negative" for an empty line, and ''.isspace() ->
false). Does that fix the failing tests?

From theller at ctypes.org  Thu Mar  7 12:37:14 2013
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 07 Mar 2013 12:37:14 +0100
Subject: [Python-Dev] [docs] undocumented argtypes magic in ctypes?
In-Reply-To: <CAF-Rda_Gs0GT55+=HimkUVa5icS0X=vWBHug8iByq9=C1YEsmA@mail.gmail.com>
References: <CAF-Rda9T=LfviFKqT-qkNDPP9ba-Kg=M41RLac_miA99dXjOAA@mail.gmail.com>
	<CAL3CFcUoKG=+vaTBMs6nSbgz_gJgxLX3_uK94JLSyEpOY7iy_Q@mail.gmail.com>
	<CAF-Rda_Gs0GT55+=HimkUVa5icS0X=vWBHug8iByq9=C1YEsmA@mail.gmail.com>
Message-ID: <kh9u40$hdr$1@ger.gmane.org>

Am 06.03.2013 18:19, schrieb Eli Bendersky:
>
>
>
> On Wed, Mar 6, 2013 at 8:33 AM, Andrew Svetlov <andrew.svetlov at gmail.com
> <mailto:andrew.svetlov at gmail.com>> wrote:
>
>     Looks like bug for me.
>
>
> ctypes seems to auto-convert arguments when argtypes is specified. This
> fact is documented. However, I'm not sure whether this auto-conversion
> is advanced enough to apply byref. Because otherwise, DIRENT is
> certainly not convertible to DIRENT_p

If argtypes specify a 'POINTER(X)' type as an argument, then ctypes
automatically applies byref() if an 'X' instance is passed to the
actual call.  This is by design, but I'm not sure if it is documented
or not.

However, if argtypes is not given, this does (and of course cannot) work.

Thomas



From eliben at gmail.com  Thu Mar  7 14:25:18 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 7 Mar 2013 05:25:18 -0800
Subject: [Python-Dev] [docs] undocumented argtypes magic in ctypes?
In-Reply-To: <kh9u40$hdr$1@ger.gmane.org>
References: <CAF-Rda9T=LfviFKqT-qkNDPP9ba-Kg=M41RLac_miA99dXjOAA@mail.gmail.com>
	<CAL3CFcUoKG=+vaTBMs6nSbgz_gJgxLX3_uK94JLSyEpOY7iy_Q@mail.gmail.com>
	<CAF-Rda_Gs0GT55+=HimkUVa5icS0X=vWBHug8iByq9=C1YEsmA@mail.gmail.com>
	<kh9u40$hdr$1@ger.gmane.org>
Message-ID: <CAF-Rda9oR592_K1dURGziriCC7ACDmJOQAqFhtPPm3tUghs63g@mail.gmail.com>

On Thu, Mar 7, 2013 at 3:37 AM, Thomas Heller <theller at ctypes.org> wrote:

> Am 06.03.2013 18:19, schrieb Eli Bendersky:
>
>>
>>
>>
>> On Wed, Mar 6, 2013 at 8:33 AM, Andrew Svetlov <andrew.svetlov at gmail.com
>> <mailto:andrew.svetlov at gmail.**com <andrew.svetlov at gmail.com>>> wrote:
>>
>>     Looks like bug for me.
>>
>>
>> ctypes seems to auto-convert arguments when argtypes is specified. This
>> fact is documented. However, I'm not sure whether this auto-conversion
>> is advanced enough to apply byref. Because otherwise, DIRENT is
>> certainly not convertible to DIRENT_p
>>
>
> If argtypes specify a 'POINTER(X)' type as an argument, then ctypes
> automatically applies byref() if an 'X' instance is passed to the
> actual call.  This is by design, but I'm not sure if it is documented
> or not.
>
> However, if argtypes is not given, this does (and of course cannot) work.
>

Great, thanks for confirming this, Thomas. I had the feeling it's a
documentation issue (hence I sent it to the docs@ list first), because the
behavior seems very deliberate and looking at the code of ctypes I did see
conversions going on.

Have I missed that this is documented somewhere, or should I open a docs
issue?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130307/826417ea/attachment.html>

From victor.stinner at gmail.com  Thu Mar  7 14:34:25 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Thu, 7 Mar 2013 14:34:25 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
 Swartz memorial)
In-Reply-To: <20130307100840.GA24941@wycliff.ceplovi.cz>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
	<CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
	<CAMpsgwZ8ryt1XO-FHuVZHEtVO4uXsMtyRA1VjJEpOjBsAnXq6Q@mail.gmail.com>
	<20130307100840.GA24941@wycliff.ceplovi.cz>
Message-ID: <CAMpsgwZ1q4LZ9OYd3uji8iXBDsLVqQ5rihV79HvPjamg9kYmWg@mail.gmail.com>

You should try to write a simple test not using your library (just
copy/paste code) reproducing the issue. If you can do that, please
fill an issue on bugs.python.org.

Victor

2013/3/7 Matej Cepl <mcepl at redhat.com>:
> On 2013-03-06, 18:34 GMT, Victor Stinner wrote:
>> In short, Unicode was rewritten in Python 3.3 for the PEP 393. It's
>> not surprising that minor details like singleton differ. You should
>> not use "is" to compare strings in Python, or your program will fail
>> on other Python implementations (like PyPy, IronPython, or Jython) or
>> even on a different CPython version.
>
> I am sorry, I don't understand what you are saying. Even though
> this has been changed to
> https://github.com/mcepl/html2text/blob/fix_tests/html2text.py#L90
> the tests still fail.
>
> But, Amaury is right: the function doesn't make much sense.
> However, ...
>
> when I have ?fixed it? from
> https://github.com/mcepl/html2text/blob/master/html2text.py#L95
>
> def onlywhite(line):
>      """Return true if the line does only consist of whitespace characters."""
>      for c in line:
>          if c is not ' ' and c is not '  ':
>              return c is ' '
>      return line
>
> to
> https://github.com/mcepl/html2text/blob/fix_tests/html2text.py#L90
>
> def onlywhite(line):
>      """Return true if the line does only consist of whitespace
>      characters."""
>          for c in line:
>             if c != ' ' and c != ' ':
>                return c == ' '
>          return line
>
> tests on ALL versions of Python are suddenly failing ...
> https://travis-ci.org/mcepl/html2text/builds/5288190
>
> Curiouser and curiouser! At least, I seem to have the point,
> where things are breaking, but I have to admit that condition
> really doesn?t make any sense to me.
>
>> Anyway, you spotted a missed optimization: it's now "fixed" in
>> Python 3.3 and 3.4 by the following commits.
>
> Well, whatever is the problem, it is not fixed in python 3.3.0
> (as you can see in
> https://travis-ci.org/mcepl/html2text/builds/4969045) as I can
> see on my computer. Actually, good news is that it seems to be
> fixed in the master branch of cpython (or the tip, as they say in
> the Mercurial world).
>
> Any thoughts?
>
> Mat?j
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/victor.stinner%40gmail.com

From theller at ctypes.org  Thu Mar  7 14:53:14 2013
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 07 Mar 2013 14:53:14 +0100
Subject: [Python-Dev] [docs] undocumented argtypes magic in ctypes?
In-Reply-To: <CAF-Rda9oR592_K1dURGziriCC7ACDmJOQAqFhtPPm3tUghs63g@mail.gmail.com>
References: <CAF-Rda9T=LfviFKqT-qkNDPP9ba-Kg=M41RLac_miA99dXjOAA@mail.gmail.com>
	<CAL3CFcUoKG=+vaTBMs6nSbgz_gJgxLX3_uK94JLSyEpOY7iy_Q@mail.gmail.com>
	<CAF-Rda_Gs0GT55+=HimkUVa5icS0X=vWBHug8iByq9=C1YEsmA@mail.gmail.com>
	<kh9u40$hdr$1@ger.gmane.org>
	<CAF-Rda9oR592_K1dURGziriCC7ACDmJOQAqFhtPPm3tUghs63g@mail.gmail.com>
Message-ID: <kha62v$1f4$1@ger.gmane.org>

>         ctypes seems to auto-convert arguments when argtypes is
>         specified. This
>         fact is documented. However, I'm not sure whether this
>         auto-conversion
>         is advanced enough to apply byref. Because otherwise, DIRENT is
>         certainly not convertible to DIRENT_p
>
>     If argtypes specify a 'POINTER(X)' type as an argument, then ctypes
>     automatically applies byref() if an 'X' instance is passed to the
>     actual call.  This is by design, but I'm not sure if it is documented
>     or not.
>
>     However, if argtypes is not given, this does (and of course cannot)
>     work.
>
> Great, thanks for confirming this, Thomas. I had the feeling it's a
> documentation issue (hence I sent it to the docs@ list first), because
> the behavior seems very deliberate and looking at the code of ctypes I
> did see conversions going on.
>
> Have I missed that this is documented somewhere, or should I open a docs
> issue?

I didn't find anything in the docs (in the two minutes I spent for 
that), so please open a docs issue, or, better, fix it.

Thomas



From eliben at gmail.com  Thu Mar  7 15:09:58 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 7 Mar 2013 06:09:58 -0800
Subject: [Python-Dev] [docs] undocumented argtypes magic in ctypes?
In-Reply-To: <kha62v$1f4$1@ger.gmane.org>
References: <CAF-Rda9T=LfviFKqT-qkNDPP9ba-Kg=M41RLac_miA99dXjOAA@mail.gmail.com>
	<CAL3CFcUoKG=+vaTBMs6nSbgz_gJgxLX3_uK94JLSyEpOY7iy_Q@mail.gmail.com>
	<CAF-Rda_Gs0GT55+=HimkUVa5icS0X=vWBHug8iByq9=C1YEsmA@mail.gmail.com>
	<kh9u40$hdr$1@ger.gmane.org>
	<CAF-Rda9oR592_K1dURGziriCC7ACDmJOQAqFhtPPm3tUghs63g@mail.gmail.com>
	<kha62v$1f4$1@ger.gmane.org>
Message-ID: <CAF-Rda8FZZCVSJO90PpqWoZe4kq-q0qHKZ5YsgwVhJOjmzOz5w@mail.gmail.com>

On Thu, Mar 7, 2013 at 5:53 AM, Thomas Heller <theller at ctypes.org> wrote:

>         ctypes seems to auto-convert arguments when argtypes is
>>         specified. This
>>         fact is documented. However, I'm not sure whether this
>>         auto-conversion
>>         is advanced enough to apply byref. Because otherwise, DIRENT is
>>         certainly not convertible to DIRENT_p
>>
>>     If argtypes specify a 'POINTER(X)' type as an argument, then ctypes
>>     automatically applies byref() if an 'X' instance is passed to the
>>     actual call.  This is by design, but I'm not sure if it is documented
>>     or not.
>>
>>     However, if argtypes is not given, this does (and of course cannot)
>>     work.
>>
>> Great, thanks for confirming this, Thomas. I had the feeling it's a
>> documentation issue (hence I sent it to the docs@ list first), because
>> the behavior seems very deliberate and looking at the code of ctypes I
>> did see conversions going on.
>>
>> Have I missed that this is documented somewhere, or should I open a docs
>> issue?
>>
>
> I didn't find anything in the docs (in the two minutes I spent for that),
> so please open a docs issue, or, better, fix it.


http://bugs.python.org/issue17378, has a patch already. Please take a look.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130307/397c67fc/attachment.html>

From Steve.Dower at microsoft.com  Thu Mar  7 18:53:56 2013
From: Steve.Dower at microsoft.com (Steve Dower)
Date: Thu, 7 Mar 2013 17:53:56 +0000
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <kh8jmi$j93$1@ger.gmane.org>
References: <kh759h$v7i$1@ger.gmane.org>
	<CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>
	<fa3ed1f62dd44f4abaf6debbe9b5eabf@BLUPR03MB035.namprd03.prod.outlook.com>
	<kh8jmi$j93$1@ger.gmane.org>
Message-ID: <ab143ce5fce64f609bd32c2d133f390c@BLUPR03MB035.namprd03.prod.outlook.com>

> From: Terry Reedy
> On 3/6/2013 12:29 PM, Steve Dower wrote:
> > From: Case Van Horsen
> 
> >> The "Microsoft Windows SDK for Windows 7 and .NET Framework 3.5 SP1"
> >> is still available for download. It includes the command line
> >> compilers that are used with VS 2008. I have used to create extensions for
> Python 2.6 to 3.2.
> >> There is a later version of the SDK (for .NET
> >> 4.x) that includes the compilers from VS 2010.
> >
> > This is the same response that I got internally.
> >
> > The download link is
>  > http://www.microsoft.com/en-us/download/details.aspx?id=3138
>  > and you can choose to only download and install the compilers.
> 
> The C++ compiler appears to the the full compiler that will build both
> 32 and 64 bits apps. Will downloading just the compiler(s) allow one to build
> Python with the project files in PCBuild or does something else need to be
> checked also?

Just testing this now, but 

Any version of Visual Studio (Professional or higher), OR
	Visual Studio 2012 Express for Desktop (http://www.microsoft.com/visualstudio/eng/products/visual-studio-express-for-windows-desktop) OR
	Visual C++ 2010 Express (http://www.microsoft.com/visualstudio/en-us/express-cpp/overview) (maybe - haven't confirmed this yet)

For Python 3.3: the compilers and headers from the "Windows SDK for Windows 7 and .NET Framework 4" (http://www.microsoft.com/en-us/download/details.aspx?id=8279)
For earlier versions: the compilers and headers from the "Windows SDK for Windows 7 and .NET Framework 3.5" (http://www.microsoft.com/en-us/download/details.aspx?id=3138)
(You can install both compilers on the same machine.)

Once these compilers have been installed, VS will let you choose which one your project will use. In Project Properties there is a "Platform Toolset" list that will include all of the installed compilers. For Python 3.3, you'll want VC100, and earlier versions will want VC90.

If you open an existing project (including PCBuild.sln), VS will offer to update it. If you don't update it, and you have the earlier compilers installed, it will use them. Right now, I've only tested this with 3.3, which used a different project format to earlier versions (.vcxproj, rather than .vcproj). I assume we know how to upgrade the project files without changing the platform target, but I haven't confirmed that yet.

> >> To use the SDK compiler, you need to do a few manual steps first.
> >>
> >> After starting a command window, you need to run a batch file to
> >> configure your environment. Choose the appropriate option from
> >>
> >> C:\Program Files (x86)\Microsoft Visual Studio
> >> 9.0\VC\bin\vcvars64.bat
> >>
> >> or
> >>
> >> C:\Program Files (x86)\Microsoft Visual Studio
> >> 9.0\VC\bin\vcvars32.bat
> >>
> >> Then set two environment variables:
> >>
> >> set MSSdk=1
> >> set DISTUTILS_USE_SDK=1
> >>
> >> After these steps, the standard python setup.py install should work.
> 
> This may be fine for building extensions, but it appears that more instructions
> are needed for a novice to build python itself.

I'm not even sure that these variables are necessary - certainly without the compilers installed setup.py looks in the right place for them. I'll try this as well.

> Following the instruction in the developer's guide,
> http://docs.python.org/devguide/setup.html#windows
> I was able to download and install vc express, double click on
> <branch_dir>/PCBuild/pcbuild.sln to bring up the VS GUI, and use the menu
> to build a debug version of that branch. The new python is put in the same
> directory and can be run with another menu selection. Any alternate path
> should be that easy too.

I'll admit I'm not a huge fan of the current Windows build setup, but since so few people seem to use it I understand why it hasn't changed.

As for the documentation, I'd be happy to provide an update for this section once I've checked out that everything works.


Cheers,
Steve

> --
> Terry Jan Reedy



From g.brandl at gmx.net  Thu Mar  7 21:20:57 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 07 Mar 2013 21:20:57 +0100
Subject: [Python-Dev] Difference in RE between 3.2 and 3.3 (or Aaron
	Swartz memorial)
In-Reply-To: <20130307100840.GA24941@wycliff.ceplovi.cz>
References: <slrnkid41e.u21.mcepl@wycliff.ceplovi.cz>
	<mailman.2549.1361895946.2939.python-list@python.org>
	<1362575394.23949.2.camel@wycliff.ceplovi.cz>
	<CAGmFidaC3Wu=Vrp6xkWyU0z8KyMWqtUPfWRGMbTYH9BPb8S4zw@mail.gmail.com>
	<CAMpsgwZ8ryt1XO-FHuVZHEtVO4uXsMtyRA1VjJEpOjBsAnXq6Q@mail.gmail.com>
	<20130307100840.GA24941@wycliff.ceplovi.cz>
Message-ID: <khasqs$bei$1@ger.gmane.org>

Am 07.03.2013 11:08, schrieb Matej Cepl:

>> Anyway, you spotted a missed optimization: it's now "fixed" in
>> Python 3.3 and 3.4 by the following commits.
> 
> Well, whatever is the problem, it is not fixed in python 3.3.0
> (as you can see in
> https://travis-ci.org/mcepl/html2text/builds/4969045) as I can
> see on my computer. Actually, good news is that it seems to be
> fixed in the master branch of cpython (or the tip, as they say in
> the Mercurial world).

It's not a "fix", it's an optimization.  Please understand that
using the "is" operator on strings is entirely wrong.

Georg


From tjreedy at udel.edu  Thu Mar  7 23:21:35 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 07 Mar 2013 17:21:35 -0500
Subject: [Python-Dev] PEP 434:  IDLE Enhancement Exception
Message-ID: <khb3tk$k96$1@ger.gmane.org>

This re-write of Todd's draft focuses better on the specific proposal 
and motivation. It tries to take into account comments posted both here 
and on python-ideas

--------------------------------------------------------------------

PEP: 434
Title: IDLE Enhancement Exception for All Branches
Version: $Revision$
Last-Modified: $Date$
Author: Todd Rovito <rovitotv at gmail.com>,
         Terry Reedy <tjreedy at udel.edu>
BDFL-Delegate: Nick Coghlan
Status: Draft
Type: Informational
Content-Type: text/x-rst
Created: 16-Feb-2013
Post-History: 16-Feb-2013


Abstract
========

Most CPython tracker issues are classified as behavior or
enhancement.  Most behavior patches are backported to branches for
existing versions.  Enhancement patches are restricted to the default
branch that becomes the next Python version.

This PEP proposes that the restriction on applying enhancements be
relaxed for IDLE code, residing in .../Lib/idlelib/.  In practice,
this would mean that IDLE developers would not have to classify or
agree on the classification of a patch but could instead focus on
what is best for IDLE users and future IDLE developement. It would
also mean that IDLE patches would not necessarily have to be split
into 'bugfix' changes and enhancement changes.

The PEP would apply to changes in existing features and addition of
small features, such as would require a new menu entry, but not
necessarily to possible major re-writes such as switching to themed
widgets or tabbed windows.


Motivation
==========

This PEP was prompted by controversy on both the tracker and pydev
list over adding Cut, Copy, and Paste to right-click context menus
(Issue 1207589, opened in 2005 [1]_; pydev thread [2]_).  The
features were available as keyboard shortcuts but not on the context
menu. It is standard, at least on Windows, that they should be when
applicable (a read-only window would only have Copy), so users do not
have to shift to the keyboard after selecting text for cutting or
copying or a slice point for pasting.  The context menu was not
documented until 10 days before the new options were added (Issue
10405 [3]_).

Normally, behavior is called a bug if it conflicts with documentation
judged to be correct. But if there is no doc, what is the standard?
If the code is its own documentation, most IDLE issues on the tracker
are enhancement issues.  If we substitute reasonable user expectation,
(which can, of course, be its own subject of disagreement), many more
issues are behavior issues.

For context menus, people disagreed on the status of the additions --
bugfix or enhancement. Even people who called it an enhancement
disagreed as to whether the patch should be backported.  This PEP
proposes to make the status disagreement irrelevant by explicitly
allowing more liberal backporting than for other stdlib modules.


Rationale
=========

People primarily use IDLE by running the gui application, rather than
by directly importing the effectively private (undocumented)
implementation modules in idlelib. Whether they use the shell, the
editor, or both, we believe they will benefit more from consistency
across the latest releases of current Python versions than from
consistency within the bugfix releases for one Python version. This
is especially true when existing behavior is clearly unsatisfactory.

When people use the standard interpreter, the OS-provided frame works
pretty much the same for all Python versions. If, for instance,
Microsoft were to upgrade the Command Prompt gui, the improvements
would be present regardless of which Python were running within it.
Similarly, if one edits Python code with editor X, behaviors such as
the right-click context menu and the search-replace box do not depend
on the version of Python being edited or even the language being
edited.

The benefit for IDLE developers is mixed. On the one hand, testing
more versions and possibly having to adjust a patch, especially for
2.7, is more work. (There is, of course, the option on not
backporting everything. For issue 12510, some changes to calltips for
classes were not included in the 2.7 patch because of issues with
old-style classes [4]_.)  On the other hand, bike-shedding can be an
energy drain. If the obvious fix for a bug looks like an enhancement,
writing a separate bugfix-only patch is more work. And making the
code diverge between versions makes future multi-version patches more
difficult.

These issue are illustrated by the search-and-replace dialog box.
It used to raise an exception for certain user entries [5]_. The
uncaught exception caused IDLE to exit.  At least on Windows, the
exit was silent (no visible traceback) and looked like a crash if
IDLE was started normally, from an icon.

Was this a bug?  IDLE Help (on the current Help submenu) just says
"Replace... Open a search-and-replace dialog box", and a box *was*
opened. It is not, in general, a bug for a library method to raise an
exception.  And it is not, in general, a bug for a library method to
ignore an exception raised by functions it calls. So if we were to
adopt the 'code = doc' philosopy in the absence of detailed docs, one
might say 'No'.

However, IDLE exiting when it does not need to is definitely
obnoxious.  So four of us agreed that it should be prevented. But
there was still the question of what to do instead?  Catch the
exception?  Just not raise the exception?  Beep?  Display an error
message box?  Or try to do something useful with the user's entry?
Would replacing a 'crash' with useful behavior be an enhancement,
limited to future Python releases?  Should IDLE developers have to
ask that?


Backwards Compatibility
=======================

For IDLE, there are three types of users who might be concerned about
back compatibility. First are people who run IDLE as an application.
We have already discussed them above.

Second are people who import one of the idlelib modules.  As far as
we know, this is only done to start the IDLE application, and we do
not propose breaking such use.  Otherwise, the modules are
undocumented and effectively private implementations.  If an IDLE
module were defined as public, documented, and perhaps moved to the
tkinter package, it would then follow the normal rules. (Documenting
the private interfaces for the benefit of people working on the IDLE
code is a separate issue.)

Third are people who write IDLE extensions. The guaranteed extension
interface is given in idlelib/extension.txt. This should be respected
at least in existing versions, and not frivolously changed in future
versions.  But there is a warning that "The extension cannot assume
much about this [EditorWindow] argument."  This guarantee should
rarely be an issue with patches, and the issue is not specific to
'enhancement' versus 'bugfix' patches.

As is happens, after the context menu patch was applied, it came up
that extensions that added items to the context menu (rare) would be
broken because the patch a) added a new item to standard rmenu_specs
and b) expected every rmenu_spec to be lengthened. It is not clear
whether this violates the guarantee, but there is a second patch that
fixes assumption b).  It should be applied when it is clear that the
first patch will not have to be reverted.


References
==========

.. [1] IDLE: Right Click Context Menu, Foord, Michael
    (http://bugs.python.org/issue1207589)

.. [2] Cut/Copy/Paste items in IDLE right click context menu
    (http://mail.python.org/pipermail/python-dev/2012-November/122514.html)

.. [3] IDLE breakpoint facility undocumented, Daily, Ned
    (http://bugs.python.org/issue10405)

.. [4] IDLE: calltips mishandle raw strings and other examples,
     Reedy, Terry
    (http://bugs.python.org/issue12510)

.. [5] IDLE: replace ending with '\' causes crash, Reedy, Terry
    (http://bugs.python.org/issue13052)


Copyright
=========

This document has been placed in the public domain.

-- 
Terry Jan Reedy


From victor.stinner at gmail.com  Fri Mar  8 00:39:39 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Fri, 8 Mar 2013 00:39:39 +0100
Subject: [Python-Dev] pytracemalloc 0.7: new tool to track memory leaks in
	Python
Message-ID: <CAMpsgwbLBk3xjyHRR1sAMwK9dFgL+mg3A7W-qUzEJM52X1_DVQ@mail.gmail.com>

Hi,

See below for a copy of my email posted to python-list and
python-announce mailing lists. pytracemalloc tool requires a patch on
Python to hook memory allocation functions. I posted the patch there:
http://bugs.python.org/issue3329

Thanks to this patch, it would also be possible to enable or disable
debug memory allocators (ex: _PyMem_DebugMalloc vs PyMem_Malloc) at
runtime, instead of having to decide at compile time.

Ezio proposed a similar idea for the "[X refs, Y blocks]" message
display at Python (compiled in debug mode) exit. He proposed to
disable the message by default, and add a (command line) option to
show the message:
http://bugs.python.org/issue17323

--

Wyplay is proud to announce the release of a new tool to track Python
memory allocations: "pytracemalloc".

https://pypi.python.org/pypi/pytracemalloc
https://github.com/wyplay/pytracemalloc

pytracemalloc provides the following information:

- Allocated size and number of allocations per file,
  or optionally per file and line number
- Compute the average size of memory allocations
- Compute delta between two "snapshots"
- Get the source of a memory allocation: filename and line number

It helps to track memory leaks: show directly in which Python files
the memory increases.

Example of pytracemalloc output (compact):

    2013-02-28 23:40:18: Top 5 allocations per file
    #1: .../Lib/test/regrtest.py: 3998 KB
    #2: .../Lib/unittest/case.py: 2343 KB
    #3: .../ctypes/test/__init__.py: 513 KB
    #4: .../Lib/encodings/__init__.py: 525 KB
    #5: .../Lib/compiler/transformer.py: 438 KB
    other: 32119 KB
    Total allocated size: 39939 KB

Example of pytracemalloc output (full):

    2013-03-04 01:01:55: Top 10 allocations per file and line
    #1: .../2.7/Lib/linecache.py:128: size=408 KiB (+408 KiB),
count=5379 (+5379), average=77 B
    #2: .../unittest/test/__init__.py:14: size=401 KiB (+401 KiB),
count=6668 (+6668), average=61 B
    #3: .../2.7/Lib/doctest.py:506: size=319 KiB (+319 KiB), count=197
(+197), average=1 KiB
    #4: .../Lib/test/regrtest.py:918: size=429 KiB (+301 KiB),
count=5806 (+3633), average=75 B
    #5: .../Lib/unittest/case.py:332: size=162 KiB (+136 KiB),
count=452 (+380), average=367 B
    #6: .../Lib/test/test_doctest.py:8: size=105 KiB (+105 KiB),
count=1125 (+1125), average=96 B
    #7: .../Lib/unittest/main.py:163: size=77 KiB (+77 KiB),
count=1149 (+1149), average=69 B
    #8: .../Lib/test/test_types.py:7: size=75 KiB (+75 KiB),
count=1644 (+1644), average=46 B
    #9: .../2.7/Lib/doctest.py:99: size=64 KiB (+64 KiB), count=1000
(+1000), average=66 B
    #10: .../Lib/test/test_exceptions.py:6: size=56 KiB (+56 KiB),
count=932 (+932), average=61 B
    3023 more: size=1580 KiB (+1138 KiB), count=12635 (+7801), average=128 B
    Total: size=3682 KiB (+3086 KiB), count=36987 (+29908), average=101 B

To install pytracemalloc, you need to patch and recompile your own
version of Python to be able to hook all Python memory allocations.

--

Wyplay was created in March 2006 in the south of France. Independent,
Europe-based, and internationally recognized, Wyplay?s TV-centric
software solutions power the world?s most popular operator and
consumer electronic brand names. Targeted products includes:
Connected-HDTVs, Media Center CE devices, HD IPTV boxes, DVB-S/C/T HD
STBs, and in-home media-HDD products.
http://www.wyplay.com/

Victor

From status at bugs.python.org  Fri Mar  8 18:07:03 2013
From: status at bugs.python.org (Python tracker)
Date: Fri,  8 Mar 2013 18:07:03 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20130308170703.5F554568ED@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2013-03-01 - 2013-03-08)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3895 (+18)
  closed 25265 (+38)
  total  29160 (+56)

Open issues with patches: 1703 


Issues opened (42)
==================

#10886: Unhelpful backtrace for multiprocessing.Queue
http://bugs.python.org/issue10886  reopened by neologix

#14489: repr() function link on the built-in function documentation is
http://bugs.python.org/issue14489  reopened by r.david.murray

#17330: Stop checking for directory cache invalidation in importlib
http://bugs.python.org/issue17330  opened by erik.bray

#17332: typo in json docs - "convered" should be "converted"
http://bugs.python.org/issue17332  opened by ernest

#17335: FieldStorageClass is messed up
http://bugs.python.org/issue17335  opened by Neal.Norwitz

#17337: input() and raw_input() do not work correctly with colored pro
http://bugs.python.org/issue17337  opened by mic_e

#17338: Add length_hint parameter to list, dict,	set constructors to a
http://bugs.python.org/issue17338  opened by alex

#17339: bytes() TypeError message is misleadingly narrow
http://bugs.python.org/issue17339  opened by terry.reedy

#17340: Handle malformed cookie
http://bugs.python.org/issue17340  opened by keakon

#17341: Poor error message when compiling invalid regex
http://bugs.python.org/issue17341  opened by roysmith

#17342: datetime.strptime does not implement %z
http://bugs.python.org/issue17342  opened by zwn

#17343: Add a version of str.split which returns an iterator
http://bugs.python.org/issue17343  opened by alex

#17344: checking size of size_t... configure: error:
http://bugs.python.org/issue17344  opened by shilpi

#17345: Portable and extended type specifiers for array module
http://bugs.python.org/issue17345  opened by nnemkin

#17348: Unicode - encoding seems to be lost for inputs of unicode char
http://bugs.python.org/issue17348  opened by Pradyun.Gedam

#17349: wsgiref.simple_server.demo_app is not PEP-3333 compatible
http://bugs.python.org/issue17349  opened by kedder

#17350: Use STAF call python script will case 1124861 issue in 2.7.2 v
http://bugs.python.org/issue17350  opened by gwtking

#17351: Fixed python3 descriptor documentation example + removal of ex
http://bugs.python.org/issue17351  opened by pelson

#17352: Be clear that __prepare__ must be declared as a class method
http://bugs.python.org/issue17352  opened by ncoghlan

#17353: Plistlib outputs empty data tags when deeply nested
http://bugs.python.org/issue17353  opened by jfortier

#17354: TypeError when running setup.py upload --show-response
http://bugs.python.org/issue17354  opened by mitya57

#17357: Add missing verbosity message to importlib
http://bugs.python.org/issue17357  opened by brett.cannon

#17358: imp.load_module() leads to the improper caching of the 'file' 
http://bugs.python.org/issue17358  opened by brett.cannon

#17359: python modules.zip is not documented
http://bugs.python.org/issue17359  opened by lemburg

#17360: Regular expressions on mmap'd files can overflow
http://bugs.python.org/issue17360  opened by fatlotus

#17362: enable-new-dtags only for GNU ELF linker
http://bugs.python.org/issue17362  opened by rpetrov

#17365: Remove Python 2 code from test_print
http://bugs.python.org/issue17365  opened by berker.peksag

#17368: Python version of JSON decoder does not work with object_pairs
http://bugs.python.org/issue17368  opened by Kuukunen

#17369: Message.get_filename produces exception if the RFC2231 encodin
http://bugs.python.org/issue17369  opened by r.david.murray

#17370: PEP should note if it has been superseded
http://bugs.python.org/issue17370  opened by brandon-rhodes

#17371: Mismatch between Python 3.3 build environment and distutils co
http://bugs.python.org/issue17371  opened by mayaa

#17372: provide pretty printer for xml.etree.ElementTree
http://bugs.python.org/issue17372  opened by eric.snow

#17373: Add inspect.Signature.from_callable()
http://bugs.python.org/issue17373  opened by eric.snow

#17374: Remove restriction against Semaphore having a negative value
http://bugs.python.org/issue17374  opened by rhettinger

#17375: Add docstrings to methods in the threading module
http://bugs.python.org/issue17375  opened by rhettinger

#17376: TimedRotatingFileHandler documentation regarding 'Week day' la
http://bugs.python.org/issue17376  opened by tshepang

#17380: initproc return value is unclear
http://bugs.python.org/issue17380  opened by zbysz

#17381: IGNORECASE breaks unicode literal range matching
http://bugs.python.org/issue17381  opened by acdha

#17382: debugging with idle: current line not highlighted
http://bugs.python.org/issue17382  opened by dzabel

#17383: Possibly ambiguous phrasing in tutorial/modules#more-on-module
http://bugs.python.org/issue17383  opened by Piotr.Kuchta

#17384: test_logging failures on Windows
http://bugs.python.org/issue17384  opened by ezio.melotti

#17385: Use deque instead of list the threading.Condition waiter queue
http://bugs.python.org/issue17385  opened by rhettinger



Most recent 15 issues with no replies (15)
==========================================

#17385: Use deque instead of list the threading.Condition waiter queue
http://bugs.python.org/issue17385

#17373: Add inspect.Signature.from_callable()
http://bugs.python.org/issue17373

#17372: provide pretty printer for xml.etree.ElementTree
http://bugs.python.org/issue17372

#17365: Remove Python 2 code from test_print
http://bugs.python.org/issue17365

#17362: enable-new-dtags only for GNU ELF linker
http://bugs.python.org/issue17362

#17354: TypeError when running setup.py upload --show-response
http://bugs.python.org/issue17354

#17351: Fixed python3 descriptor documentation example + removal of ex
http://bugs.python.org/issue17351

#17350: Use STAF call python script will case 1124861 issue in 2.7.2 v
http://bugs.python.org/issue17350

#17349: wsgiref.simple_server.demo_app is not PEP-3333 compatible
http://bugs.python.org/issue17349

#17348: Unicode - encoding seems to be lost for inputs of unicode char
http://bugs.python.org/issue17348

#17345: Portable and extended type specifiers for array module
http://bugs.python.org/issue17345

#17342: datetime.strptime does not implement %z
http://bugs.python.org/issue17342

#17340: Handle malformed cookie
http://bugs.python.org/issue17340

#17335: FieldStorageClass is messed up
http://bugs.python.org/issue17335

#17332: typo in json docs - "convered" should be "converted"
http://bugs.python.org/issue17332



Most recent 15 issues waiting for review (15)
=============================================

#17385: Use deque instead of list the threading.Condition waiter queue
http://bugs.python.org/issue17385

#17376: TimedRotatingFileHandler documentation regarding 'Week day' la
http://bugs.python.org/issue17376

#17375: Add docstrings to methods in the threading module
http://bugs.python.org/issue17375

#17373: Add inspect.Signature.from_callable()
http://bugs.python.org/issue17373

#17369: Message.get_filename produces exception if the RFC2231 encodin
http://bugs.python.org/issue17369

#17365: Remove Python 2 code from test_print
http://bugs.python.org/issue17365

#17362: enable-new-dtags only for GNU ELF linker
http://bugs.python.org/issue17362

#17354: TypeError when running setup.py upload --show-response
http://bugs.python.org/issue17354

#17351: Fixed python3 descriptor documentation example + removal of ex
http://bugs.python.org/issue17351

#17338: Add length_hint parameter to list, dict,	set constructors to a
http://bugs.python.org/issue17338

#17330: Stop checking for directory cache invalidation in importlib
http://bugs.python.org/issue17330

#17329: Document unittest.SkipTest
http://bugs.python.org/issue17329

#17325: improve organization of the PyPI distutils docs
http://bugs.python.org/issue17325

#17324: SimpleHTTPServer serves files even if the URL has a trailing s
http://bugs.python.org/issue17324

#17323: Disable [X refs, Y blocks] ouput in debug builds
http://bugs.python.org/issue17323



Top 10 most discussed issues (10)
=================================

#17322: urllib.request add_header() currently allows trailing spaces (
http://bugs.python.org/issue17322  17 msgs

#13564: ftplib and sendfile()
http://bugs.python.org/issue13564  16 msgs

#17330: Stop checking for directory cache invalidation in importlib
http://bugs.python.org/issue17330  15 msgs

#17338: Add length_hint parameter to list, dict,	set constructors to a
http://bugs.python.org/issue17338  15 msgs

#13477: tarfile module should have a command line
http://bugs.python.org/issue13477  14 msgs

#12921: http.server.BaseHTTPRequestHandler.send_error and trailing new
http://bugs.python.org/issue12921  13 msgs

#10967: move regrtest over to using more unittest infrastructure
http://bugs.python.org/issue10967  10 msgs

#12768: docstrings for the threading module
http://bugs.python.org/issue12768   8 msgs

#17383: Possibly ambiguous phrasing in tutorial/modules#more-on-module
http://bugs.python.org/issue17383   8 msgs

#16997: subtests
http://bugs.python.org/issue16997   7 msgs



Issues closed (38)
==================

#11732: Skip decorator for tests requiring manual intervention on Wind
http://bugs.python.org/issue11732  closed by ezio.melotti

#11787: File handle leak in TarFile lib
http://bugs.python.org/issue11787  closed by ezio.melotti

#13747: ssl_version documentation error
http://bugs.python.org/issue13747  closed by pitrou

#13898: Ignored exception in test_ssl
http://bugs.python.org/issue13898  closed by nadeem.vawda

#14123: Indicate that there are no current plans to deprecate printf-s
http://bugs.python.org/issue14123  closed by ezio.melotti

#14645: Generator does not translate linesep characters in certain cir
http://bugs.python.org/issue14645  closed by r.david.murray

#15448: utimes() functions fail with ENOSYS even when detected by conf
http://bugs.python.org/issue15448  closed by neologix

#15465: Improved documentation for C API version info
http://bugs.python.org/issue15465  closed by python-dev

#16669: Docstrings for namedtuple
http://bugs.python.org/issue16669  closed by rhettinger

#16848: Mac OS X: python-config --ldflags and location of Python.frame
http://bugs.python.org/issue16848  closed by ned.deily

#16860: Use O_CLOEXEC in the tempfile module
http://bugs.python.org/issue16860  closed by neologix

#16962: _posixsubprocess module uses outdated getdents system call
http://bugs.python.org/issue16962  closed by gregory.p.smith

#17032: Misleading error message: global name 'X' is not defined
http://bugs.python.org/issue17032  closed by ezio.melotti

#17146: Improve test.support.import_fresh_module()
http://bugs.python.org/issue17146  closed by eric.snow

#17278: SIGSEGV in _heapqmodule.c
http://bugs.python.org/issue17278  closed by pitrou

#17298: Twisted test failure triggered by change in 2.7 branch
http://bugs.python.org/issue17298  closed by ezio.melotti

#17302: HTTP/2.0 - Implementations/Testing efforts
http://bugs.python.org/issue17302  closed by terry.reedy

#17309: __bytes__ doesn't work in subclass of int and str
http://bugs.python.org/issue17309  closed by benjamin.peterson

#17312: test_aifc doesn't clean up after itself
http://bugs.python.org/issue17312  closed by ezio.melotti

#17315: test_posixpath doesn't clean up after itself
http://bugs.python.org/issue17315  closed by ezio.melotti

#17327: Add PyDict_GetItemSetDefault() as C-API for dict.setdefault()
http://bugs.python.org/issue17327  closed by python-dev

#17328: Fix reference leak in dict_setdefault() in case of resize fail
http://bugs.python.org/issue17328  closed by python-dev

#17331: namedtuple raises a SyntaxError instead of ValueError on inval
http://bugs.python.org/issue17331  closed by rhettinger

#17333: Fix test discovery for test_imaplib.py
http://bugs.python.org/issue17333  closed by ezio.melotti

#17334: Fix test discovery for test_index.py
http://bugs.python.org/issue17334  closed by ezio.melotti

#17336: Complex number representation round-trip doesn't work with sig
http://bugs.python.org/issue17336  closed by mark.dickinson

#17346: Pickle tests do not test protocols 0, 1, and 2 for bytes
http://bugs.python.org/issue17346  closed by ezio.melotti

#17347: bsddb._openDBEnv() should not touch current directory
http://bugs.python.org/issue17347  closed by jcea

#17355: http tests testing more than the error code are fragile
http://bugs.python.org/issue17355  closed by r.david.murray

#17356: Invalid link to repr() built-in function description
http://bugs.python.org/issue17356  closed by r.david.murray

#17361: use CC to test compiler flags in setup.py
http://bugs.python.org/issue17361  closed by skrah

#17363: Argument Mixup in PyState_AddModule
http://bugs.python.org/issue17363  closed by ezio.melotti

#17364: Multiprocessing documentation mentions function that doesn't e
http://bugs.python.org/issue17364  closed by ezio.melotti

#17366: os.chdir win32
http://bugs.python.org/issue17366  closed by amaury.forgeotdarc

#17367: subprocess deadlock when read() is interrupted
http://bugs.python.org/issue17367  closed by sbt

#17377: JSON module in standard library behaves incorrectly on input l
http://bugs.python.org/issue17377  closed by r.david.murray

#17378: Document that ctypes automatically applies byref() when argtyp
http://bugs.python.org/issue17378  closed by python-dev

#17379: Zen amendment
http://bugs.python.org/issue17379  closed by brett.cannon

From steve at pearwood.info  Sat Mar  9 02:13:54 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Sat, 09 Mar 2013 12:13:54 +1100
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
Message-ID: <513A8CD2.8090105@pearwood.info>

On 02/03/13 02:43, Demian Brecht wrote:
> Cross-posting from python-ideas due to no response there. Perhaps it's
> due to a general lack of usage/caring for cookiejar, but figured
> /someone/'s got to have an opinion about my proposal ;)

Apparently not :-(


> TL;DR: CookieJar > FileCookieJar > *CookieJar are architecturally
> broken and this is an attempt to rectify that (and fix a couple bugs
> along the way).
[...]
> This will obviously break backwards compatibility, so I'm not entirely
> sure what best practice is around that: leave well enough alone even
> though it might not make sense, keep the old implementations around
> and deprecate them to be eventually replaced by the processors, or
> other ideas?

I don't have an opinion on cookiejars per se, but I think that the first
thing to do is get an idea of just how major a backward-compatibility
breakage this would be. If you change the cookiejar architecture, then
run the Python test suite, what happens? The number of failures will give
you an idea of how bad it will be.

If there are no failures, you could consider just making the change. You
probably should make an attempt to find out what third party apps use the
cookiejars and see what they do.

If there are failures, then you need to add a second cookiejar
implementation, and deprecate the old one.

Oh, and please don't call the new cookier jar anything like "NewCookieJar".
Because in a few years, it won't be.

Actually, I lied, I do have an opinion on cookiejars. I agree with Terry
that it is a bit weird to have an ABC inherit from a concrete class. Not
just weird, but a violation of the Liskov Substitution Principle that
an instance of a subclass should be usable anywhere an instance of the
parent class is. If you can't even instantiate the subclass, that's a
pretty major violation for no apparent benefit :-)


-- 
Steven

From rdmurray at bitdance.com  Sun Mar 10 21:59:40 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Sun, 10 Mar 2013 16:59:40 -0400
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <513A8CD2.8090105@pearwood.info>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
	<513A8CD2.8090105@pearwood.info>
Message-ID: <20130310205940.D93E8250BC3@webabinitio.net>

On Sat, 09 Mar 2013 12:13:54 +1100, Steven D'Aprano <steve at pearwood.info> wrote:
> On 02/03/13 02:43, Demian Brecht wrote:
> > Cross-posting from python-ideas due to no response there. Perhaps it's
> > due to a general lack of usage/caring for cookiejar, but figured
> > /someone/'s got to have an opinion about my proposal ;)
> 
> Apparently not :-(
> 
> 
> > TL;DR: CookieJar > FileCookieJar > *CookieJar are architecturally
> > broken and this is an attempt to rectify that (and fix a couple bugs
> > along the way).
> [...]
> > This will obviously break backwards compatibility, so I'm not entirely
> > sure what best practice is around that: leave well enough alone even
> > though it might not make sense, keep the old implementations around
> > and deprecate them to be eventually replaced by the processors, or
> > other ideas?
> 
> I don't have an opinion on cookiejars per se, but I think that the first
> thing to do is get an idea of just how major a backward-compatibility
> breakage this would be. If you change the cookiejar architecture, then
> run the Python test suite, what happens? The number of failures will give
> you an idea of how bad it will be.
> 
> If there are no failures, you could consider just making the change. You
> probably should make an attempt to find out what third party apps use the
> cookiejars and see what they do.

To be clear, just passing the stdlib tests is *not* sufficient to think
that backward compatibility is not likely to be broken.  Deciding about
the likelihood of breakage is a hard problem, to which we generally
employ gut-level heuristics :) (And code search, as Steven suggests).

Since you say that it will "obviously" break backward compatibility, I'd
say that if we are going to do anything we'd have to think about how best
to introduce a more sane implementation and deprecate the old...and if we
are going to do that, we probably ought to spend a bit of time seeing if
there are any other open cookiejar issues we can tackle at the same time.

If, that is, you are interested enough to continue to be the point person
for this, which probably won't be a short process :)

The problem here is getting people interested, apparently :(

Since I start my Pycon diversion-from-work next week, maybe I can find
some time to take at least a preliminary look.

--David

From tjreedy at udel.edu  Sun Mar 10 22:36:59 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 10 Mar 2013 17:36:59 -0400
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <20130310205940.D93E8250BC3@webabinitio.net>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
	<513A8CD2.8090105@pearwood.info>
	<20130310205940.D93E8250BC3@webabinitio.net>
Message-ID: <khiue6$ltc$1@ger.gmane.org>

On 3/10/2013 4:59 PM, R. David Murray wrote:

> To be clear, just passing the stdlib tests is *not* sufficient to think
> that backward compatibility is not likely to be broken.  Deciding about
> the likelihood of breakage is a hard problem, to which we generally
> employ gut-level heuristics :) (And code search, as Steven suggests).
>
> Since you say that it will "obviously" break backward compatibility, I'd
> say that if we are going to do anything we'd have to think about how best
> to introduce a more sane implementation and deprecate the old...and if we
> are going to do that, we probably ought to spend a bit of time seeing if
> there are any other open cookiejar issues we can tackle at the same time.

A) For similar reasons, I consider the proposal a first draft, and 
probably not the exact right thing to do.
B) I have had similar thoughts about taking a broader look. Searching 
open issues for cookie gets 24 hits and I think at least half are about 
cookie.py or cookiejar.py.

> If, that is, you are interested enough to continue to be the point person
> for this, which probably won't be a short process :)
>
> The problem here is getting people interested, apparently :(

The number of relatively recent problem reports indicates that people 
are using the two modules, so fixing them is worthwhile in that sense.

On the other no, it does not seem that any *current* developers are 
working with cookies.

Messages on http://bugs.python.org/issue17340 suggest that cookie.py 
should be based on http://tools.ietf.org/html/rfc6265
I added you as nosy to get your opinion.

> Since I start my Pycon diversion-from-work next week, maybe I can find
> some time to take at least a preliminary look.

I am willing to learn and help, but my only experience with them is as a 
browser user defending against the onslaught of cookies. (I once sped up 
IExplorer by deleting a massive cookie cache.)

-- 
Terry Jan Reedy


From ezio.melotti at gmail.com  Mon Mar 11 02:14:37 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Mon, 11 Mar 2013 03:14:37 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #17385: Fix
 quadratic behavior in threading.Condition
In-Reply-To: <3ZPLXL0jVYzS7C@mail.python.org>
References: <3ZPLXL0jVYzS7C@mail.python.org>
Message-ID: <CACBhJdE+HBf1ik9EF7rFFLupMxhad24kkXi1a6Gu4RfAmSKsjg@mail.gmail.com>

Hi,

On Mon, Mar 11, 2013 at 2:58 AM, raymond.hettinger
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/0f86b51f8f8b
> changeset:   82592:0f86b51f8f8b
> user:        Raymond Hettinger <python at rcn.com>
> date:        Sun Mar 10 17:57:28 2013 -0700
> summary:
>   Issue #17385: Fix quadratic behavior in threading.Condition
>
> files:
>   Lib/threading.py |  10 ++++++++--
>   Misc/NEWS        |   3 +++
>   2 files changed, 11 insertions(+), 2 deletions(-)
>
>
> diff --git a/Lib/threading.py b/Lib/threading.py
> --- a/Lib/threading.py
> +++ b/Lib/threading.py
> @@ -10,6 +10,12 @@
>      from time import time as _time
>  from traceback import format_exc as _format_exc
>  from _weakrefset import WeakSet
> +try:
> +    from _itertools import islice as _slice
> +    from _collections import deque as _deque
> +except ImportError:
> +    from itertools import islice as _islice
> +    from collections import deque as _deque
>

Shouldn't the one in the 'try' be _islice too?

Best Regards,
Ezio Melotti

From demianbrecht at gmail.com  Mon Mar 11 06:46:26 2013
From: demianbrecht at gmail.com (Demian Brecht)
Date: Sun, 10 Mar 2013 22:46:26 -0700
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <20130310205940.D93E8250BC3@webabinitio.net>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
	<513A8CD2.8090105@pearwood.info>
	<20130310205940.D93E8250BC3@webabinitio.net>
Message-ID: <513D6FB2.4020105@gmail.com>

On 2013-03-10 1:59 PM, R. David Murray wrote:
> To be clear, just passing the stdlib tests is*not*  sufficient to think
> that backward compatibility is not likely to be broken.  Deciding about
> the likelihood of breakage is a hard problem, to which we generally
> employ gut-level heuristics:)  (And code search, as Steven suggests).

I figured that this would be a hard problem, which is also why I didn't 
delve into a patch further than a proposed first stab at a more sane 
implementation, coupled with changes to the unit tests.

> Since you say that it will "obviously" break backward compatibility, I'd
> say that if we are going to do anything we'd have to think about how best
> to introduce a more sane implementation and deprecate the old...and if we
> are going to do that, we probably ought to spend a bit of time seeing if
> there are any other open cookiejar issues we can tackle at the same time.

I was hoping that there would be a little more interest (and potentially 
some further historical context on why the module was implemented as it 
was) from those in the group.

> If, that is, you are interested enough to continue to be the point person
> for this, which probably won't be a short process:)

I'm not sure who this was directed to (me or Steven), but I was looking 
for an area in the stdlib that I could really sink my teeth into and get 
my hands dirty with and this definitely seems to be just that. I figured 
that it wouldn't be a short process and the more that I read up on RFC 
6265 (and 2965) and compare them to the implementation in cookie and 
cookiejar, the more I'm thinking that this will be a relatively complex 
and lengthy process. (Definitely interested in that btw :)).

>
> The problem here is getting people interested, apparently:(
>
> Since I start my Pycon diversion-from-work next week, maybe I can find
> some time to take at least a preliminary look.

In case you haven't already seen it, I had posted a second patch (that 
doesn't break the Liskov substitution principle as Terry pointed out 
after reviewing my overzealous initial patch) here: 
http://bugs.python.org/issue16901. I think the design is much more sane 
than what's currently there and aligns with how HTTP cookies are 
processed in urllib.request.

Now having said all that, the more I think about it and the more I read, 
the more I wonder why there are even specialized implementations (LWP 
and Mozilla) in the stdlib to begin with. I would assume that the only 
thing that the stdlib /should/ be covering is the RFC (6265, but still 
allowing 2965).

If there are deviations (and some are eluded to throughout the code), 
then I would think that those should be handled by packages external to 
the stdlib. It seems that the Mozilla implementation covers 2965, but 
LWP is based on the Perl library (which isn't known to be supported by 
any browser environment). Why is this even there to begin with? To 
paraphrase the comments that I read in the code: "This isn't supported 
by any browser, but they're easy to parse". In my mind, this shouldn't 
be reason enough for inclusion in the stdlib.

I'd also go as far to say that if cookies are implemented as 
consistently as, say, OAuth 2.0 providers (meaning very little to no 
consistency), then there really shouldn't be a cookie implementation in 
the stdlib anyway.

So to sum it up, yes I'm very much interested in doing what I can to 
help the development of the stdlib (more so interested in parts that 
don't currently have experts listed, such as http and imaplib), but will 
definitely need to be shown the ropes a bit as my professional life has 
revolved around closed source games.

From demianbrecht at gmail.com  Mon Mar 11 06:48:33 2013
From: demianbrecht at gmail.com (Demian Brecht)
Date: Sun, 10 Mar 2013 22:48:33 -0700
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <khiue6$ltc$1@ger.gmane.org>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
	<513A8CD2.8090105@pearwood.info>
	<20130310205940.D93E8250BC3@webabinitio.net>
	<khiue6$ltc$1@ger.gmane.org>
Message-ID: <513D7031.1000603@gmail.com>

On 2013-03-10 2:36 PM, Terry Reedy wrote:
> A) For similar reasons, I consider the proposal a first draft, and
> probably not the exact right thing to do.

That is correct. The more I think about it, the more I'm convincing 
myself that even though the proposal is more sane than what's there 
right now, it's definitely not the exact correct thing to do.

From rdmurray at bitdance.com  Mon Mar 11 13:44:01 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Mon, 11 Mar 2013 08:44:01 -0400
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <513D6FB2.4020105@gmail.com>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
	<513A8CD2.8090105@pearwood.info>
	<20130310205940.D93E8250BC3@webabinitio.net>
	<513D6FB2.4020105@gmail.com>
Message-ID: <20130311124402.F0AD3250BCD@webabinitio.net>

On Sun, 10 Mar 2013 22:46:26 -0700, Demian Brecht <demianbrecht at gmail.com> wrote:
> On 2013-03-10 1:59 PM, R. David Murray wrote:
> I was hoping that there would be a little more interest (and potentially 
> some further historical context on why the module was implemented as it 
> was) from those in the group.

It isn't clear who wrote the original code.  It looks like Martin von
L??wis checked it in, so he may be the author (or not).  It is pretty
old code, checked in May 31 18:22:40 2004, according to the repo
history.

> > If, that is, you are interested enough to continue to be the point person
> > for this, which probably won't be a short process:)
> 
> I'm not sure who this was directed to (me or Steven), but I was looking 
> for an area in the stdlib that I could really sink my teeth into and get 
> my hands dirty with and this definitely seems to be just that. I figured 
> that it wouldn't be a short process and the more that I read up on RFC 
> 6265 (and 2965) and compare them to the implementation in cookie and 
> cookiejar, the more I'm thinking that this will be a relatively complex 
> and lengthy process. (Definitely interested in that btw :)).

It was directed to you.  We love having people pick up maintenance of
modules that don't currently have someone specifically interested in
them, so it is great that you are interested.  If you produce code and
proposals and keep asking, people *will* respond, though some patience
and persistence may be required.

> > The problem here is getting people interested, apparently:(
> >
> > Since I start my Pycon diversion-from-work next week, maybe I can find
> > some time to take at least a preliminary look.
> 
> In case you haven't already seen it, I had posted a second patch (that 
> doesn't break the Liskov substitution principle as Terry pointed out 
> after reviewing my overzealous initial patch) here: 
> http://bugs.python.org/issue16901. I think the design is much more sane 
> than what's currently there and aligns with how HTTP cookies are 
> processed in urllib.request.

I haven't looked it over yet, but I put it on my todo list.

> Now having said all that, the more I think about it and the more I read, 
> the more I wonder why there are even specialized implementations (LWP 
> and Mozilla) in the stdlib to begin with. I would assume that the only 
> thing that the stdlib /should/ be covering is the RFC (6265, but still 
> allowing 2965).

Because reality.

Take a look at http://bugs.python.org/issue2193 (for example), and see
if you still want to tackle this topic :) (I hope you do).

> If there are deviations (and some are eluded to throughout the code), 
> then I would think that those should be handled by packages external to 
> the stdlib. It seems that the Mozilla implementation covers 2965, but 
> LWP is based on the Perl library (which isn't known to be supported by 
> any browser environment). Why is this even there to begin with? To 
> paraphrase the comments that I read in the code: "This isn't supported 
> by any browser, but they're easy to parse". In my mind, this shouldn't 
> be reason enough for inclusion in the stdlib.

Well, at the time it probably was.  And given that it is there, *someone*
is probably depending on it.  But, we can probably pay less attention
to that variant, and perhaps not carry it forward if we do decide
to go through a deprecation of some sort (*).

The other reality is that our cookie support won't be very useful if
it adheres strictly to the RFCs, since the servers and browsers don't.
What we need is something practical...which may differ to a greater or
lesser degree from what we currently have.

> I'd also go as far to say that if cookies are implemented as 
> consistently as, say, OAuth 2.0 providers (meaning very little to no 
> consistency), then there really shouldn't be a cookie implementation in 
> the stdlib anyway.

But there is, and in fact it *is* useful and used by many people, so
IMO it is worth maintaining.

> So to sum it up, yes I'm very much interested in doing what I can to 
> help the development of the stdlib (more so interested in parts that 
> don't currently have experts listed, such as http and imaplib), but will 
> definitely need to be shown the ropes a bit as my professional life has 
> revolved around closed source games.

Excellent.  If you aren't already on the core-mentorship mailing list, you
might want to sign up.  Your approach (adopting modules without current
maintainers) is a good one.

--David

(*) Our deprecation for stuff like this tends to be that we pretty much
stop maintaining it, document it as deprecated, but don't delete it.

From brett at python.org  Mon Mar 11 14:22:38 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 11 Mar 2013 09:22:38 -0400
Subject: [Python-Dev] [Python-checkins] cpython (2.7): #16004: Add `make
	touch`.
In-Reply-To: <3ZPVtS3mkkzSHQ@mail.python.org>
References: <3ZPVtS3mkkzSHQ@mail.python.org>
Message-ID: <CAP1=2W64=2SKA7ZVwK-VwLfr-phKCtrxPqUKBp4MF5AqRkqZoA@mail.gmail.com>

Should this also touch Python/importlib.h?


On Mon, Mar 11, 2013 at 3:14 AM, ezio.melotti <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/da3f4774b939
> changeset:   82600:da3f4774b939
> branch:      2.7
> parent:      82593:3e14aafeca04
> user:        Ezio Melotti <ezio.melotti at gmail.com>
> date:        Mon Mar 11 09:14:09 2013 +0200
> summary:
>   #16004: Add `make touch`.
>
> files:
>   Makefile.pre.in |  6 +++++-
>   Misc/NEWS       |  2 ++
>   2 files changed, 7 insertions(+), 1 deletions(-)
>
>
> diff --git a/Makefile.pre.in b/Makefile.pre.in
> --- a/Makefile.pre.in
> +++ b/Makefile.pre.in
> @@ -1250,6 +1250,10 @@
>         etags Include/*.h; \
>         for i in $(SRCDIRS); do etags -a $$i/*.[ch]; done
>
> +# Touch generated files
> +touch:
> +       touch Include/Python-ast.h Python/Python-ast.c
> +
>  # Sanitation targets -- clean leaves libraries, executables and tags
>  # files, which clobber removes as well
>  pycremoval:
> @@ -1339,7 +1343,7 @@
>  .PHONY: frameworkinstall frameworkinstallframework
> frameworkinstallstructure
>  .PHONY: frameworkinstallmaclib frameworkinstallapps
> frameworkinstallunixtools
>  .PHONY: frameworkaltinstallunixtools recheck autoconf clean clobber
> distclean
> -.PHONY: smelly funny patchcheck altmaninstall
> +.PHONY: smelly funny patchcheck touch altmaninstall
>  .PHONY: gdbhooks
>
>  # IF YOU PUT ANYTHING HERE IT WILL GO AWAY
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -874,6 +874,8 @@
>  Build
>  -----
>
> +- Issue #16004: Add `make touch`.
> +
>  - Issue #5033: Fix building of the sqlite3 extension module when the
>    SQLite library version has "beta" in it. Patch by Andreas Pelme.
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130311/330859af/attachment.html>

From brett at python.org  Mon Mar 11 14:23:07 2013
From: brett at python.org (Brett Cannon)
Date: Mon, 11 Mar 2013 09:23:07 -0400
Subject: [Python-Dev] [Python-checkins] cpython (2.7): #16004: Add `make
	touch`.
In-Reply-To: <CAP1=2W64=2SKA7ZVwK-VwLfr-phKCtrxPqUKBp4MF5AqRkqZoA@mail.gmail.com>
References: <3ZPVtS3mkkzSHQ@mail.python.org>
	<CAP1=2W64=2SKA7ZVwK-VwLfr-phKCtrxPqUKBp4MF5AqRkqZoA@mail.gmail.com>
Message-ID: <CAP1=2W4MimB=xKMLELgRYu5knpEGjqkCjCAbFocatZHmxB4kHA@mail.gmail.com>

On Mon, Mar 11, 2013 at 9:22 AM, Brett Cannon <brett at python.org> wrote:

> Should this also touch Python/importlib.h?
>
>
nm, noticed this was added on 2.7 and not default.


>
> On Mon, Mar 11, 2013 at 3:14 AM, ezio.melotti <python-checkins at python.org>wrote:
>
>> http://hg.python.org/cpython/rev/da3f4774b939
>> changeset:   82600:da3f4774b939
>> branch:      2.7
>> parent:      82593:3e14aafeca04
>> user:        Ezio Melotti <ezio.melotti at gmail.com>
>> date:        Mon Mar 11 09:14:09 2013 +0200
>> summary:
>>   #16004: Add `make touch`.
>>
>> files:
>>   Makefile.pre.in |  6 +++++-
>>   Misc/NEWS       |  2 ++
>>   2 files changed, 7 insertions(+), 1 deletions(-)
>>
>>
>> diff --git a/Makefile.pre.in b/Makefile.pre.in
>> --- a/Makefile.pre.in
>> +++ b/Makefile.pre.in
>> @@ -1250,6 +1250,10 @@
>>         etags Include/*.h; \
>>         for i in $(SRCDIRS); do etags -a $$i/*.[ch]; done
>>
>> +# Touch generated files
>> +touch:
>> +       touch Include/Python-ast.h Python/Python-ast.c
>> +
>>  # Sanitation targets -- clean leaves libraries, executables and tags
>>  # files, which clobber removes as well
>>  pycremoval:
>> @@ -1339,7 +1343,7 @@
>>  .PHONY: frameworkinstall frameworkinstallframework
>> frameworkinstallstructure
>>  .PHONY: frameworkinstallmaclib frameworkinstallapps
>> frameworkinstallunixtools
>>  .PHONY: frameworkaltinstallunixtools recheck autoconf clean clobber
>> distclean
>> -.PHONY: smelly funny patchcheck altmaninstall
>> +.PHONY: smelly funny patchcheck touch altmaninstall
>>  .PHONY: gdbhooks
>>
>>  # IF YOU PUT ANYTHING HERE IT WILL GO AWAY
>> diff --git a/Misc/NEWS b/Misc/NEWS
>> --- a/Misc/NEWS
>> +++ b/Misc/NEWS
>> @@ -874,6 +874,8 @@
>>  Build
>>  -----
>>
>> +- Issue #16004: Add `make touch`.
>> +
>>  - Issue #5033: Fix building of the sqlite3 extension module when the
>>    SQLite library version has "beta" in it. Patch by Andreas Pelme.
>>
>>
>> --
>> Repository URL: http://hg.python.org/cpython
>>
>> _______________________________________________
>> Python-checkins mailing list
>> Python-checkins at python.org
>> http://mail.python.org/mailman/listinfo/python-checkins
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130311/0ab3fb28/attachment.html>

From solipsis at pitrou.net  Mon Mar 11 14:28:40 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Mon, 11 Mar 2013 14:28:40 +0100
Subject: [Python-Dev] cpython (2.7): #16004: Add `make touch`.
References: <3ZPVtS3mkkzSHQ@mail.python.org>
Message-ID: <20130311142840.09935555@pitrou.net>

On Mon, 11 Mar 2013 08:14:24 +0100 (CET)
ezio.melotti <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/da3f4774b939
> changeset:   82600:da3f4774b939
> branch:      2.7
> parent:      82593:3e14aafeca04
> user:        Ezio Melotti <ezio.melotti at gmail.com>
> date:        Mon Mar 11 09:14:09 2013 +0200
> summary:
>   #16004: Add `make touch`.

Shouldn't that be mentioned / explained / documented somewhere?
It doesn't look obvious in which circumstances it could be useful.

Regards

Antoine.



From demianbrecht at gmail.com  Mon Mar 11 16:35:17 2013
From: demianbrecht at gmail.com (Demian Brecht)
Date: Mon, 11 Mar 2013 08:35:17 -0700
Subject: [Python-Dev] FileCookieJars
In-Reply-To: <20130311124402.F0AD3250BCD@webabinitio.net>
References: <CAE+T62ZB2RN7EN48e7pWQyPyqqDBpAeWwY2QaajmiqRh8Hh25w@mail.gmail.com>
	<513A8CD2.8090105@pearwood.info>
	<20130310205940.D93E8250BC3@webabinitio.net>
	<513D6FB2.4020105@gmail.com>
	<20130311124402.F0AD3250BCD@webabinitio.net>
Message-ID: <513DF9B5.6040503@gmail.com>

On 2013-03-11 5:44 AM, R. David Murray wrote:

> though some patience
> and persistence may be required.

I have a wife and kids. This, I've become quite good at ;)

> Take a look at http://bugs.python.org/issue2193 (for example), and see
> if you still want to tackle this topic :) (I hope you do).

Egad. I knew that cookies were quite the can of worms prior to digging 
into this as much as I have, but I didn't realize that the RFC had been 
written /after/ cookie implementations had already surfaced in the wild 
(I guess I shouldn't have actually been surprised either). Just makes 
this more challenging and therefore interesting to work on imo :)

> The other reality is that our cookie support won't be very useful if
> it adheres strictly to the RFCs, since the servers and browsers don't.
> What we need is something practical...which may differ to a greater or
> lesser degree from what we currently have.

Yes, I wasn't sure of the general standpoint of Python stdlibs in terms 
of practicality versus strict adherence. While adhering to Postel's law 
in cases such as cookies can definitely make an implementation much more 
tricky, it increases its practical usage (I didn't realize just how 
deviant servers and browsers were for this particular topic until after 
reading through issue 2193).

> But there is, and in fact it *is* useful and used by many people, so
> IMO it is worth maintaining.

I see your point here and agree. It's much different when changes can be 
dictated in closed source packages (what I'm most accustomed to) than 
dealing with an open source project at the scale of Python and the stdlib.

> Excellent.  If you aren't already on the core-mentorship mailing list, you
> might want to sign up.  Your approach (adopting modules without current
> maintainers) is a good one.

Thanks, I wasn't aware of the core-mentorship list. I'll be signing up 
shortly. Good to know my approach is sane :)


From ezio.melotti at gmail.com  Mon Mar 11 21:11:28 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Mon, 11 Mar 2013 22:11:28 +0200
Subject: [Python-Dev] cpython (2.7): #16004: Add `make touch`.
In-Reply-To: <20130311142840.09935555@pitrou.net>
References: <3ZPVtS3mkkzSHQ@mail.python.org>
	<20130311142840.09935555@pitrou.net>
Message-ID: <CACBhJdHyhpcribpr0YLr=dDoHZ2nscRdD+uto06BWxfBEcBEjg@mail.gmail.com>

Hi,

On Mon, Mar 11, 2013 at 3:28 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Mon, 11 Mar 2013 08:14:24 +0100 (CET)
> ezio.melotti <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/da3f4774b939
>> changeset:   82600:da3f4774b939
>> branch:      2.7
>> parent:      82593:3e14aafeca04
>> user:        Ezio Melotti <ezio.melotti at gmail.com>
>> date:        Mon Mar 11 09:14:09 2013 +0200
>> summary:
>>   #16004: Add `make touch`.
>
> Shouldn't that be mentioned / explained / documented somewhere?
> It doesn't look obvious in which circumstances it could be useful.
>

It will be documented in http://bugs.python.org/issue15964
(SyntaxError in asdl when building 2.7 with system Python 3).

Best Regards,
Ezio Melotti

> Regards
>
> Antoine.
>

From ezio.melotti at gmail.com  Tue Mar 12 07:52:18 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Tue, 12 Mar 2013 08:52:18 +0200
Subject: [Python-Dev] [Python-checkins] CANNOT Patch 3.x NEWS [was
 cpython (2.7): Issue #14707: add news entry\
In-Reply-To: <513EC7E2.90400@udel.edu>
References: <3ZQ4z44QVLzRXr@mail.python.org>
	<513EC7E2.90400@udel.edu>
Message-ID: <CACBhJdGeJom5gkuJRc96m7MZ77HN3Fmz4QZ9QCLUXisGwyp7vw@mail.gmail.com>

Hi,

On Tue, Mar 12, 2013 at 8:14 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/12/2013 1:50 AM, terry.reedy wrote:
>>
>> http://hg.python.org/cpython/rev/c162e2ff15bd
>> changeset:   82624:c162e2ff15bd
>> branch:      2.7
>> parent:      82617:cd0191a9b5c9
>> user:        Terry Jan Reedy <tjreedy at udel.edu>
>> date:        Tue Mar 12 01:26:28 2013 -0400
>> summary:
>>    Issue #14707: add news entry
>>
>> files:
>>    Misc/NEWS |  3 +++
>>    1 files changed, 3 insertions(+), 0 deletions(-)
>>
>>
>> diff --git a/Misc/NEWS b/Misc/NEWS
>> --- a/Misc/NEWS
>> +++ b/Misc/NEWS
>> @@ -944,6 +944,9 @@
>>   Documentation
>>   -------------
>>
>> +- Issue #14707: remove doubled words in docs and docstrings
>> +  reported by Serhiy Storchaka and Matthew Barnett.
>> +
>>   - Issue #16406: combine the pages for uploading and registering to PyPI.
>>
>>   - Issue #16403: Document how distutils uses the maintainer field in
>
>
> The above was easy. When I tried to transplant this patch to 3.2, export and
> import, or directly edit 3.2 NEWS with Notepad++ or IDLE, hg makes a 319kb
> patch that deletes and add the entire file in chunks. I did not think I
> should commit and push that.
>

What are the exact commands you used?
Are your clones up to date (i.e. did you do "hg pull" and "hg up"
before "hg graft")?
  If not, you should pull/update.
Does "hg heads ." show you more than one head?
  If so you should do "hg merge".
Is your clone "clean" (i.e. does "hg status" show anything as 'M')?
  If not, you should do "hg revert -ar 3.2" or "hg up -C 3.2".
Once your clone is clean you can just edit Misc/NEWS manually since
it's easier than trying to graft the 2 changesets you made on 2.7 to
add and edit the Misc/NEWS entry.
You can also check with "hg in" and "hg out" if there's something you
haven't pulled/pushed yet, but that shouldn't be a problem.

> The failure of transplant and import are perhaps understandable because 3.2
> has a gratuitous case difference with /combine/Combine/.
>
> - Issue #16406: Combine the pages for uploading and registering to PyPI.
>
> But the inability to make a proper diff from direct edit is something else.
> If I add just a single blank line, even that generates a mega patch. Same
> with 3.3 NEWS. I also tried deleting the file to make hg regenerate from the
> repository database.
>
> Anyone have any idea what the problem is? Has anything changed with hg,
> windows, line endings and this text file in the last few months? I just
> pushed patches for about 20 scattered files in Docs, Lib, Modules, and Tools
> earlier today, so the problem seems to be specific to NEWS.
>

Not sure about this, but in the meanwhile you could try what I
suggested above -- if that doesn't work we can find some other
solution.
(If you prefer you can come on #python-dev too.)

Best Regards,
Ezio Melotti

> tjr
>

From ncoghlan at gmail.com  Tue Mar 12 16:26:07 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Mar 2013 01:26:07 +1000
Subject: [Python-Dev] [Python-checkins] cpython (2.7): #16004: Add `make
	touch`.
In-Reply-To: <CAP1=2W4MimB=xKMLELgRYu5knpEGjqkCjCAbFocatZHmxB4kHA@mail.gmail.com>
References: <3ZPVtS3mkkzSHQ@mail.python.org>
	<CAP1=2W64=2SKA7ZVwK-VwLfr-phKCtrxPqUKBp4MF5AqRkqZoA@mail.gmail.com>
	<CAP1=2W4MimB=xKMLELgRYu5knpEGjqkCjCAbFocatZHmxB4kHA@mail.gmail.com>
Message-ID: <CADiSq7dGWKujBkvp7UJ_porbw6P4ZjrpmqoCUv8-u93nGSUSbg@mail.gmail.com>

On 11 Mar 2013 06:23, "Brett Cannon" <brett at python.org> wrote:
>
>
>
>
> On Mon, Mar 11, 2013 at 9:22 AM, Brett Cannon <brett at python.org> wrote:
>>
>> Should this also touch Python/importlib.h?
>>
>
> nm, noticed this was added on 2.7 and not default.

Default already had it, this was a back port so that "make touch" could be
given as a consistent fix for certain build problems in the devguide.
(Specifically, make trying to rebuild those files when you don't yet have
the necessary pieces available to do so)

Cheers,
Nick.

>
>>
>>
>> On Mon, Mar 11, 2013 at 3:14 AM, ezio.melotti <python-checkins at python.org>
wrote:
>>>
>>> http://hg.python.org/cpython/rev/da3f4774b939
>>> changeset:   82600:da3f4774b939
>>> branch:      2.7
>>> parent:      82593:3e14aafeca04
>>> user:        Ezio Melotti <ezio.melotti at gmail.com>
>>> date:        Mon Mar 11 09:14:09 2013 +0200
>>> summary:
>>>   #16004: Add `make touch`.
>>>
>>> files:
>>>   Makefile.pre.in |  6 +++++-
>>>   Misc/NEWS       |  2 ++
>>>   2 files changed, 7 insertions(+), 1 deletions(-)
>>>
>>>
>>> diff --git a/Makefile.pre.in b/Makefile.pre.in
>>> --- a/Makefile.pre.in
>>> +++ b/Makefile.pre.in
>>> @@ -1250,6 +1250,10 @@
>>>         etags Include/*.h; \
>>>         for i in $(SRCDIRS); do etags -a $$i/*.[ch]; done
>>>
>>> +# Touch generated files
>>> +touch:
>>> +       touch Include/Python-ast.h Python/Python-ast.c
>>> +
>>>  # Sanitation targets -- clean leaves libraries, executables and tags
>>>  # files, which clobber removes as well
>>>  pycremoval:
>>> @@ -1339,7 +1343,7 @@
>>>  .PHONY: frameworkinstall frameworkinstallframework
frameworkinstallstructure
>>>  .PHONY: frameworkinstallmaclib frameworkinstallapps
frameworkinstallunixtools
>>>  .PHONY: frameworkaltinstallunixtools recheck autoconf clean clobber
distclean
>>> -.PHONY: smelly funny patchcheck altmaninstall
>>> +.PHONY: smelly funny patchcheck touch altmaninstall
>>>  .PHONY: gdbhooks
>>>
>>>  # IF YOU PUT ANYTHING HERE IT WILL GO AWAY
>>> diff --git a/Misc/NEWS b/Misc/NEWS
>>> --- a/Misc/NEWS
>>> +++ b/Misc/NEWS
>>> @@ -874,6 +874,8 @@
>>>  Build
>>>  -----
>>>
>>> +- Issue #16004: Add `make touch`.
>>> +
>>>  - Issue #5033: Fix building of the sqlite3 extension module when the
>>>    SQLite library version has "beta" in it. Patch by Andreas Pelme.
>>>
>>>
>>> --
>>> Repository URL: http://hg.python.org/cpython
>>>
>>> _______________________________________________
>>> Python-checkins mailing list
>>> Python-checkins at python.org
>>> http://mail.python.org/mailman/listinfo/python-checkins
>>>
>>
>
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130313/62e2ee9b/attachment.html>

From nad at acm.org  Tue Mar 12 20:23:30 2013
From: nad at acm.org (Ned Deily)
Date: Tue, 12 Mar 2013 12:23:30 -0700
Subject: [Python-Dev] CANNOT Patch 3.x NEWS [was cpython (2.7): Issue
	#14707: add news entry\
References: <3ZQ4z44QVLzRXr@mail.python.org> <513EC7E2.90400@udel.edu>
	<CACBhJdGeJom5gkuJRc96m7MZ77HN3Fmz4QZ9QCLUXisGwyp7vw@mail.gmail.com>
	<513F63BB.8060805@udel.edu>
Message-ID: <nad-4ABC76.12232912032013@news.gmane.org>

In article <513F63BB.8060805 at udel.edu>, Terry Reedy <tjreedy at udel.edu> 
wrote:

> I have tried deleting the NEWS file and reverting the deletion.
> hg update does not restore the file as it apparently thinks I actually 
> want the uncommitted deletion.

I'm not sure exactly the sequence of events here but chances are you ran 
into the "normal" problem of trying to merge a change that modifies 
Misc/NEWS.  Unless you are very lucky or careful, merging Misc/NEWS 
changes from, say, 3.2 to 3.3 or 3.3 to default seldom works cleanly.  
After merging but before committing, you can just revert Misc/NEWS and 
then manually re-insert the changes.   The auto-merge of Misc/NEWS is 
more often than not useless.

-- 
 Ned Deily,
 nad at acm.org


From anuj at codesyrup.com  Tue Mar 12 21:41:54 2013
From: anuj at codesyrup.com (Anuj Gupta)
Date: Wed, 13 Mar 2013 02:11:54 +0530
Subject: [Python-Dev] Namaste, Python-Dev
Message-ID: <513F9312.6070909@codesyrup.com>

Hello,

I've just joined Python-Dev and I intend to spend an increasing amount 
of time contributing to Python. A couple of days back I contributed my 
first (very) few lines, to the benchmark suite. Incidentally, it was my 
first contribution ever towards Free Software, and the feeling is 
incredible!

I am truly honoured to be a part of this amazing community, and I hope 
to serve it well.

Regards,

Anuj Gupta

From solipsis at pitrou.net  Tue Mar 12 21:48:02 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 12 Mar 2013 21:48:02 +0100
Subject: [Python-Dev] Namaste, Python-Dev
References: <513F9312.6070909@codesyrup.com>
Message-ID: <20130312214802.3bff927b@pitrou.net>


Hello Anuj,

On Wed, 13 Mar 2013 02:11:54 +0530
Anuj Gupta <anuj at codesyrup.com> wrote:
> 
> I've just joined Python-Dev and I intend to spend an increasing amount 
> of time contributing to Python. A couple of days back I contributed my 
> first (very) few lines, to the benchmark suite. Incidentally, it was my 
> first contribution ever towards Free Software, and the feeling is 
> incredible!

Thanks for joining us, and congratulations :-)

If you haven't already done so, we have a developers' guide which will
help you make further contributions:
http://docs.python.org/devguide/

Regards

Antoine.



From ezio.melotti at gmail.com  Wed Mar 13 00:34:58 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 13 Mar 2013 01:34:58 +0200
Subject: [Python-Dev] [Python-checkins] CANNOT Patch 3.x NEWS [was
 cpython (2.7): Issue #14707: add news entry\
In-Reply-To: <513F63BB.8060805@udel.edu>
References: <3ZQ4z44QVLzRXr@mail.python.org> <513EC7E2.90400@udel.edu>
	<CACBhJdGeJom5gkuJRc96m7MZ77HN3Fmz4QZ9QCLUXisGwyp7vw@mail.gmail.com>
	<513F63BB.8060805@udel.edu>
Message-ID: <CACBhJdHkTjVJqULGQG9aHfiA0PD8dAnr_ydkwwXzpHqh5RKm1w@mail.gmail.com>

Hi,

On Tue, Mar 12, 2013 at 7:19 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/12/2013 2:52 AM, Ezio Melotti wrote:
>> What are the exact commands you used?
>
> Clicks on TortoiseHg HgWorkbench GUI ;-).
>

I wonder if TortoiseHg is doing something wrong here.  Maybe you could
try from cmd too.

>> Are your clones up to date (i.e. did you do "hg pull" and "hg up"
>> before "hg graft")?
>
> There were no other pushes between my last de-double patch and this, and I
> am sure I ran my pull + 3*update .bat first. I have run it multiple times
> since.
>

Around the time you pushed on 2.7 I also pushed something, so that
might have created some conflict.
How does your .bat look like?  One gotcha of the share extension is
that if you use "hg pull -u" and there's nothing to pull because you
already pulled in one of the shared clones, the update won't be
executed (this is actually normal behaviour of "hg pull", but the
consequences are especially noticeable while using shared clones).

>> Does "hg heads ." show you more than one head?
>
> The DAG window shows the normal one head per branch as appropriate for the
> particular branch display. At the moment, hg heads shows the four commits
> from Eli, 82628 to 82631 as heads plus old 2.6 and 3.1 heads.
>
>> Is your clone "clean" (i.e. does "hg status" show anything as 'M')?
>
> The status window is empty until I edit NEWS and click Refresh, at which
> point M Misc/News shows up with the megadiff.
> Right click/ Revert/yes and the file is reverted.
>
>> Once your clone is clean you can just edit Misc/NEWS manually
>
> Since the graft and import failed (producing no diff), I have been editing
> manually and that is when I get the megadiff. I added a couple of blank
> lines to ACKS and got a normal diff. Now, adding a blank line to 2.7 NEWS
> also gives a blank line.
>
> Could the failed graft have messed up the master copy in my cpython
> repository.
>

That's possible.  From "hg help graft":
    If a graft merge results in conflicts, the graft process is interrupted so
    that the current merge can be manually resolved. Once all conflicts are
    addressed, the graft process can be continued with the -c/--continue
    option.

This doesn't mean that you copy is messed up though.  "hg up -C 3.2"
should restore it.
When I graft/merge and there are conflicts I use kdiff3, and it takes
just a few seconds to solve the conflicts usually (for Misc/NEWS is
ctrl+2, ctrl+3, ctrl+s, alt+f4, that roughly translates too "include
both the conflicting news, save and quit).

> I have tried deleting the NEWS file and reverting the deletion.
> hg update does not restore the file as it apparently thinks I actually want
> the uncommitted deletion.
>

How did you delete it?  I assume that if you do it from the TortoiseHG
GUI, it will mark it as "deleted" ('D' in "hg status").
If you do it from cmd/file manager hg should see it as missing ('!' in
"hg status") and you can use "hg revert Misc/NEWS" to restore it.

>> it's easier than trying to graft the 2 changesets you made on 2.7 to
>> add and edit the Misc/NEWS entry.
>
> There was only one 2.7 changeset with only the NEWS patch.
>

I was referring to the one that added the news + the one that fixed
the issue id.

>> You can also check with "hg in" and "hg out" if there's something you
>> haven't pulled/pushed yet, but that shouldn't be a problem.
>
> I tried both and got 'no changes'.
>
>> (If you prefer you can come on #python-dev too.)
>
> I may try that, but I suspect that my registration/nick has expired again
> and last time is was obnoxiously hard to get re-established.
>

There's no need to register your nick for #python-dev (there is for
#python though).  You can just fire up your favourite IRC client (or
even http://webchat.freenode.net/) and join.
(Registering the nick shouldn't be difficult though.)

> Terry
>

From fwierzbicki at gmail.com  Wed Mar 13 04:09:25 2013
From: fwierzbicki at gmail.com (fwierzbicki at gmail.com)
Date: Tue, 12 Mar 2013 20:09:25 -0700
Subject: [Python-Dev] Python Language Summit at PyCon: Agenda
In-Reply-To: <CADrh4zKQ=a7Z4mApQ5XM2zOp67uwo8WvLgw3Cjk6AOrSS4gG+Q@mail.gmail.com>
References: <13FB490E-3548-4959-9C2B-2880B8ACA6F5@voidspace.org.uk>
	<CADrh4zJxzN4EuM8OOJevD6UFB7Y-idS0wdsE6e9ejjM_+syfJA@mail.gmail.com>
	<CAP1=2W70L2+qw4gHU1QJcWdS0JgTe6P3uFJh_Q+JVHpqSzkPrQ@mail.gmail.com>
	<CADrh4z+CFD_HmgzCrYUTHguiCJt6wqU7ZOY+tD95PDsrA_3LSw@mail.gmail.com>
	<20130227223749.2f06a328@anarchist.wooz.org>
	<CADrh4zLiaPMPyQcveXkBUwgo61iot+jG86hYma07KPSw1xfeUg@mail.gmail.com>
	<20130301093223.743a04c8@anarchist.wooz.org>
	<20130301193803.4156607c@pitrou.net>
	<D35375BC-3A8D-4D0C-8C73-0DA26B024A33@voidspace.org.uk>
	<CAF7AXFGoyJX1N95uUokutt1hxQ7sWiMMGMvK+M3zHnEaW-QwVg@mail.gmail.com>
	<CADrh4zKhSheCsfmm=R_rZTFzwxvVks5V_SznEFt1Q-i6mJCQOw@mail.gmail.com>
	<CADrh4zKQ=a7Z4mApQ5XM2zOp67uwo8WvLgw3Cjk6AOrSS4gG+Q@mail.gmail.com>
Message-ID: <CADrh4zKzU_UtfKv76ELUzLV68UePT41gKatOYa80CMVXH6-wsg@mail.gmail.com>

Hi all,

I won't be able to make it to the summit and probably not the
conference. I have a raging 104F fever (40C for many of you =)

The doctor says I have influenza and am highly contagious, so I
shouldn't be going anywhere near conference full of people for five
days - looks like I'm missing this one :( I may make it down for a
couple of sprint days.

-Frank

From ncoghlan at gmail.com  Wed Mar 13 08:01:23 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 13 Mar 2013 00:01:23 -0700
Subject: [Python-Dev] [Python-checkins] CANNOT Patch 3.x NEWS [was
 cpython (2.7): Issue #14707: add news entry\
In-Reply-To: <51402D99.3020507@udel.edu>
References: <3ZQ4z44QVLzRXr@mail.python.org> <513EC7E2.90400@udel.edu>
	<CACBhJdGeJom5gkuJRc96m7MZ77HN3Fmz4QZ9QCLUXisGwyp7vw@mail.gmail.com>
	<513F63BB.8060805@udel.edu>
	<CACBhJdHkTjVJqULGQG9aHfiA0PD8dAnr_ydkwwXzpHqh5RKm1w@mail.gmail.com>
	<51402D99.3020507@udel.edu>
Message-ID: <CADiSq7dMQVuMiwi6WPYaA9MNdAfCW6NL4JB2BugMpUGs2rvAAg@mail.gmail.com>

On Wed, Mar 13, 2013 at 12:41 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Bottom line: I decided to restart from scratch. I am still not sure if the
> glitch was hg, disk 1, disk 2, or Windows, or some combination.
>
> After making and posting a patch to the tracker today, I tried to annotate a
> file and got an error something like 'cannot find revision -1'. I then
> noticed that there was no dag in the workbench dag window, as if there were
> no revisions. When I looked in .hg/store, the big file seemed to be missing.
> So I wiped, defragmented and compacted, and reloaded TortoiseHg. Tomorrow I
> will re-clone and share the repository. Since this is the second time I have
> re-cloned from python.org, I will follow the advice I read somewhere to make
> a _backup clone that I leave alone until I need it, so I only have to pull
> from now until then when I do.

I still keep a pristine clone around so "nuke it from orbit" remains
an option. "hg histedit" lets me deal with most of my screw-ups these
days, though.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ezio.melotti at gmail.com  Wed Mar 13 09:06:55 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 13 Mar 2013 10:06:55 +0200
Subject: [Python-Dev] [Python-checkins] CANNOT Patch 3.x NEWS [was
 cpython (2.7): Issue #14707: add news entry\
In-Reply-To: <51402D99.3020507@udel.edu>
References: <3ZQ4z44QVLzRXr@mail.python.org> <513EC7E2.90400@udel.edu>
	<CACBhJdGeJom5gkuJRc96m7MZ77HN3Fmz4QZ9QCLUXisGwyp7vw@mail.gmail.com>
	<513F63BB.8060805@udel.edu>
	<CACBhJdHkTjVJqULGQG9aHfiA0PD8dAnr_ydkwwXzpHqh5RKm1w@mail.gmail.com>
	<51402D99.3020507@udel.edu>
Message-ID: <CACBhJdG4gVqm4xMD=5-xVhZL=HVLFN=zEEnNeGZvtok2+qY11Q@mail.gmail.com>

On Wed, Mar 13, 2013 at 9:41 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> Bottom line: I decided to restart from scratch. I am still not sure if the
> glitch was hg, disk 1, disk 2, or Windows, or some combination.
>
> After making and posting a patch to the tracker today, I tried to annotate a
> file and got an error something like 'cannot find revision -1'. I then
> noticed that there was no dag in the workbench dag window, as if there were
> no revisions. When I looked in .hg/store, the big file seemed to be missing.

Note that with the share extension, the "big file" (which I assume is
the store/ directory) only exists in the "main" clone.
In the shared clones you'll find an .hg/sharedpath file that contains
the path to the original .hg/ that contains the store/ dir with all
the changesets.

> So I wiped, defragmented and compacted, and reloaded TortoiseHg. Tomorrow I
> will re-clone and share the repository. Since this is the second time I have
> re-cloned from python.org, I will follow the advice I read somewhere to make
> a _backup clone that I leave alone until I need it, so I only have to pull
> from now until then when I do.
>

Good idea :)

>> On 3/12/2013 7:34 PM, Ezio Melotti wrote:
>> I wonder if TortoiseHg is doing something wrong here.  Maybe you could
>> try from cmd too.
>
> Workbench has a 'command' window for typing hg commands which it should pass
> as is to Windows much as Command Prompt does. I tried some of the things you
> suggested there.
>
>> Around the time you pushed on 2.7 I also pushed something, so that
>> might have created some conflict.
>
> I do not remember seeing that.
>

I pushed on 3.3/default about half an hour after you pushed on 2.7, so
that might have caused a push race, if during that time you were doing
the merges and eventually tried to push after me without having
pulled/updated in the meanwhile.  The problem you described doesn't
seem to be related to push races though.

>> How does your .bat look like?
>
> pull -u to cpython + update of each of the three shares, much like written
> in the devguide.
>

It's better to avoid using "hg pull -u", because if there's nothing to
pull the "update" won't be executed.  Here it shouldn't be a big
problem, but you could break it if you manually pull something in one
of the shared clones, and then run the .bat.  Unless you also have an
explicit "hg up" in the clone where you do "hg pull -u", that clone
won't be updated by the script.

>> That's possible.  From "hg help graft":
>>      If a graft merge results in conflicts, the graft process is interrupted so
>>      that the current merge can be manually resolved. Once all conflicts are
>>      addressed, the graft process can be continued with the -c/--continue
>>      option.
>
> When merge produces a conflict, a window appears offering options including
> using kdiff3 to resolve. When I tried the graft, the message in the command
> window was just 'aborted', and I do not remember getting the resolve window.
>

What version of HG are you using?

>> When I graft/merge and there are conflicts I use kdiff3, and it takes
>> just a few seconds to solve the conflicts usually (for Misc/NEWS is
>> ctrl+2, ctrl+3, ctrl+s, alt+f4, that roughly translates too "include
>> both the conflicting news, save and quit).
>
> Since I have perhaps never gotten that sequence right, that info will be
> helpful.
>

Glad to help, however I got it the other way around.
The 1st pane is the parent and you can just ignore it; the 2nd pane is
the local copy and the 3rd pane is the one from the previous branch
that you are merging.
The bottom pane will be the resulting file.

For Misc/NEWS (the file that usually conflicts), you want the newest
NEWS entry first, so you do ctrl+3 to get the one you just added, and
ctrl+2 to get the one that was there already.  Note that for other
files you usually want to get only one of the versions, usually the
one you have in the 3rd pane, so that sequence only applies to
Misc/NEWS.
Another tip is to use ctrl+q instead of alt+f4.

>> If you do it from cmd/file manager hg should see it as missing ('!' in
>> "hg status") and you can use "hg revert Misc/NEWS" to restore it.
>
> This.
>
> Thanks for trying to help. I will let you know if there are any more
> problems after the re-clone.
>

Sure, and if you find part of the devguide that are not clear let me
know (I also just uploaded a new patch to
http://bugs.python.org/issue14468 to add a few new Mercurial FAQs to
the devguide).

Best Regards,
Ezio Melotti

> I still need to comment on the tcl/tk.dll and tkinter situation, but will
> just mention now that I ran the four test_txxxx files on 3.3a0 (on Windows)
> and they seemed to finish and be ok other than altering the environment.
>
>
> Terry

From trent at snakebite.org  Thu Mar 14 03:05:41 2013
From: trent at snakebite.org (Trent Nelson)
Date: Wed, 13 Mar 2013 19:05:41 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
Message-ID: <20130314020540.GB22505@snakebite.org>

    Just posted the slides for those that didn't have the benefit of
    attending the language summit today:

        https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async


        Trent.

From christian at python.org  Thu Mar 14 13:21:09 2013
From: christian at python.org (Christian Heimes)
Date: Thu, 14 Mar 2013 13:21:09 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314020540.GB22505@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
Message-ID: <5141C0B5.6060904@python.org>

Am 14.03.2013 03:05, schrieb Trent Nelson:
>     Just posted the slides for those that didn't have the benefit of
>     attending the language summit today:
> 
>         https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async

Wow, neat! Your idea with Py_PXCTC is ingenious.

As far as I remember the FS and GS segment registers are used by most
modern operating systems on x86 and x86_64 platforms nowadays to
distinguish threads. TLS is implemented with FS and GS registers. I
guess the __read[gf]sdword() intrinsics do exactly the same. Reading
registers is super fast and should have a negligible effect on code.

ARM CPUs don't have segment registers because they have a simpler
addressing model. The register CP15 came up after a couple of Google
searches.

IMHO you should target x86, x86_64, ARMv6 and ARMv7. ARMv7 is going to
be more important than x86 in the future. We are going to see more ARM
based servers.

Christian

From solipsis at pitrou.net  Thu Mar 14 13:38:47 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 14 Mar 2013 13:38:47 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
Message-ID: <20130314133847.5c1ae965@pitrou.net>

Le Thu, 14 Mar 2013 13:21:09 +0100,
Christian Heimes <christian at python.org> a ?crit :
> 
> IMHO you should target x86, x86_64, ARMv6 and ARMv7. ARMv7 is going to
> be more important than x86 in the future.  We are going to see more
> ARM based servers.

Well we can't really see less of them, since there are hardly any ;-)

Related reading:
http://www.anandtech.com/show/6757/calxedas-arm-server-tested

Regards

Antoine.



From a.cavallo at cavallinux.eu  Thu Mar 14 13:45:59 2013
From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu)
Date: Thu, 14 Mar 2013 13:45:59 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314133847.5c1ae965@pitrou.net>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org> <20130314133847.5c1ae965@pitrou.net>
Message-ID: <3dfe6492f42e332d837aa32b1b34ef90@cavallinux.eu>

By the way on the arm (and any platform that can do cross-compiling) 
I've created a Makefile based build of the python 2.7.x:

   https://bitbucket.org/cavallo71/android

Please don't be fooled by the Android name, it really can take any 
crosscompiler (provided it follows the gcc synatx).

It was born out of the frustration with trying to adapt ./configure to 
do cross compiling. It is a sliglty different update to the problem as 
tried by the kiwy project for example.

I hope this helps,
Antonio


On 2013-03-14 13:38, Antoine Pitrou wrote:
> Le Thu, 14 Mar 2013 13:21:09 +0100,
> Christian Heimes <christian at python.org> a ?crit :
>>
>> IMHO you should target x86, x86_64, ARMv6 and ARMv7. ARMv7 is going 
>> to
>> be more important than x86 in the future.  We are going to see more
>> ARM based servers.
>
> Well we can't really see less of them, since there are hardly any ;-)
>
> Related reading:
> http://www.anandtech.com/show/6757/calxedas-arm-server-tested
>


From martin at v.loewis.de  Thu Mar 14 18:46:22 2013
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Mar 2013 10:46:22 -0700
Subject: [Python-Dev] VC++ 2008 Express Edition now locked away?
In-Reply-To: <ab143ce5fce64f609bd32c2d133f390c@BLUPR03MB035.namprd03.prod.outlook.com>
References: <kh759h$v7i$1@ger.gmane.org>
	<CANerV6kn3vD0b7HPBFrp6j4=q31aeAxGxkBV9xTMP5A-ezy_2w@mail.gmail.com>
	<fa3ed1f62dd44f4abaf6debbe9b5eabf@BLUPR03MB035.namprd03.prod.outlook.com>
	<kh8jmi$j93$1@ger.gmane.org>
	<ab143ce5fce64f609bd32c2d133f390c@BLUPR03MB035.namprd03.prod.outlook.com>
Message-ID: <51420CEE.9000905@v.loewis.de>

Am 07.03.13 09:53, schrieb Steve Dower:
>>>> To use the SDK compiler, you need to do a few manual steps
>>>> first.
>>>>
>>>> After starting a command window, you need to run a batch file
>>>> to configure your environment. Choose the appropriate option
>>>> from
>>>>
>>>> C:\Program Files (x86)\Microsoft Visual Studio
>>>> 9.0\VC\bin\vcvars64.bat
>>>>
>>>> or
>>>>
>>>> C:\Program Files (x86)\Microsoft Visual Studio
>>>> 9.0\VC\bin\vcvars32.bat
>>>>
>>>> Then set two environment variables:
>>>>
>>>> set MSSdk=1 set DISTUTILS_USE_SDK=1
>>>>
>>>> After these steps, the standard python setup.py install should
>>>> work.
>>
>> This may be fine for building extensions, but it appears that more
>> instructions are needed for a novice to build python itself.
>
> I'm not even sure that these variables are necessary - certainly
> without the compilers installed setup.py looks in the right place for
> them. I'll try this as well.

Setting MSSdk shouldn't be necessary, as vcvars should already have set it
(unless that changed in recent SDKs). Setting DISTUTILS_USE_SDK is 
necessary as a protection to avoid unintionally picking up the wrong
build tools.

As for distutils finding them automatically: this only works for finding
VS installations. It is (AFAICT) not possible to automatically locate
SDK installations (other than by exhaustive search of the disk).

> As for the documentation, I'd be happy to provide an update for this
> section once I've checked out that everything works.

I think it should explain to to invoke msbuild, in addition to 
explaining how to plug old compilers into new IDEs.

Regards,
Martin

From trent at snakebite.org  Thu Mar 14 19:23:53 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 11:23:53 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <5141C0B5.6060904@python.org>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
Message-ID: <20130314182352.GC24307@snakebite.org>

On Thu, Mar 14, 2013 at 05:21:09AM -0700, Christian Heimes wrote:
> Am 14.03.2013 03:05, schrieb Trent Nelson:
> >     Just posted the slides for those that didn't have the benefit of
> >     attending the language summit today:
> > 
> >         https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async
> 
> Wow, neat! Your idea with Py_PXCTC is ingenious.

    Yeah, it's funny how the viability and performance of the whole
    approach comes down to a quirky little trick for quickly detecting
    if we're in a parallel thread ;-)  I was very chuffed when it all
    fell into place.  (And I hope the quirkiness of it doesn't detract
    from the overall approach.)

> As far as I remember the FS and GS segment registers are used by most
> modern operating systems on x86 and x86_64 platforms nowadays to
> distinguish threads. TLS is implemented with FS and GS registers. I
> guess the __read[gf]sdword() intrinsics do exactly the same.

    Yup, in fact, if I hadn't come up with the __read[gf]sword() trick,
    my only other option would have been TLS (or the GetCurrentThreadId
    /pthread_self() approach in the presentation).  TLS is fantastic,
    and it's definitely an intrinsic part of the solution (the "Y" part
    of "if we're a parallel thread, do Y"), but it definitely more
    costly than a simple FS/GS register read.

> Reading
> registers is super fast and should have a negligible effect on code.

    Yeah the actual instruction is practically free; the main thing you
    pay for is the extra branch.  However, most of the code looks like
    this:

        if (Py_PXCTX)
            something_small_and_inlineable();
        else
            Py_INCREF(op); /* also small and inlineable */

    In the majority of the cases, all the code for both branches is
    going to be in the same cache line, so a mispredicted branch is
    only going to result in a pipeline stall, which is better than a
    cache miss.

> ARM CPUs don't have segment registers because they have a simpler
> addressing model. The register CP15 came up after a couple of Google
> searches.

    Noted, thanks!

> IMHO you should target x86, x86_64, ARMv6 and ARMv7. ARMv7 is going to
> be more important than x86 in the future. We are going to see more ARM
> based servers.

    Yeah that's my general sentiment too.  I'm definitely curious to see
    if other ISAs offer similar facilities (Sparc, IA64, POWER etc), but
    the hierarchy will be x86/x64 > ARM > * for the foreseeable future.

    Porting the Py_PXCTX part is trivial compared to the work that is
    going to be required to get this stuff working on POSIX where none
    of the sublime Windows concurrency, synchronisation and async IO
    primitives exist.

> Christian

        Trent.


From trent at snakebite.org  Thu Mar 14 19:45:20 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 11:45:20 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314020540.GB22505@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
Message-ID: <20130314184519.GD24307@snakebite.org>

On Wed, Mar 13, 2013 at 07:05:41PM -0700, Trent Nelson wrote:
>     Just posted the slides for those that didn't have the benefit of
>     attending the language summit today:
> 
>         https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async

    Someone on /r/python asked if I could elaborate on the "do Y" part
    of "if we're in a parallel thread, do Y, if not, do X", which I
    (inadvertently) ended up replying to in detail.  I've included the
    response below.  (I'll work on converting this into a TL;DR set of
    slides soon.)

> Can you go into a bit of depth about "X" here?

That's a huge topic that I'm hoping to tackle ASAP.  The basic premise
is that parallel 'Context' objects (well, structs) are allocated for
each parallel thread callback.  The context persists for the lifetime of
the "parallel work".

The "lifetime of the parallel work" depends on what you're doing.  For a
simple ``async.submit_work(foo)``, the context is considered complete
once ``foo()`` has been called (presuming no exceptions were raised).

For an async client/server, the context will persist for the entirety of
the connection.

The context is responsible for encapsulating all resources related to
the parallel thread.  So, it has its own heap, and all memory
allocations are taken from that heap.

For any given parallel thread, only one context can be executing at a
time, and this can be accessed via the ``__declspec(thread) Context
*ctx`` global (which is primed by some glue code as soon as the parallel
thread starts executing a callback).

No reference counting or garbage collection is done during parallel
thread execution.  Instead, once the context is finished, it is
scheduled to be released, which means it'll be "processed" by the main
thread as part of its housekeeping work (during ``async.run()``
(technically, ``async.run_once()``).

The main thread simply destroys the entire heap in one fell swoop,
releasing all memory that was associated with that context.

There are a few side effects to this.  First, the heap allocator
(basically, the thing that answers ``malloc()`` calls) is incredibly
simple.  It allocates LARGE_PAGE_SIZE chunks of memory at a time (2MB on
x64), and simply returns pointers to that chunk for each memory request
(adjusting h->next and allocation stats as it goes along, obviously).
Once the 2MB has been exhausted, another 2MB is allocated.

That approach is fine for the ``submit_(work|timer|wait)`` callbacks,
which basically provide a way to run a presumably-finite-length function
in a parallel thread (and invoking callbacks/errbacks as required).

However, it breaks down when dealing with client/server stuff.  Each
invocation of a callback (say, ``data_received(...)``) may only consume,
say, 500 bytes, but it might be called a million times before the
connection is terminated.  You can't have cumulative memory usage with
possibly-infinite-length client/server-callbacks like you can with the
once-off ``submit_(work|wait|timer)`` stuff.

So, enter heap snapshots.  The logic that handles all client/server
connections is instrumented such that it takes a snapshot of the heap
(and all associated stats) prior to invoking a Python method (via
``PyObject_Call()``, for example, i.e. the invocation of
``data_received``).

When the method completes, we can simply roll back the snapshot.  The
heap's stats and next pointers et al all get reset back to what they
were before the callback was invoked.

That's how the chargen server is able to pump out endless streams of
data for every client whilst keeping memory usage static.  (Well, every
new client currently consumes at least a minimum of 2MB (but down the
track that can be tweaked back down to SMALL_PAGE_SIZE, 4096, for
servers that need to handle hundreds of thousands of clients
simultaneously).

The only issue with this approach is detecting when the callback has
done the unthinkable (from a shared-nothing perspective) and persisted
some random object it created outside of the parallel context it was
created in.

That's actually a huge separate technical issue to tackle -- and it
applies just as much to the normal ``submit_(wait|work|timer)``
callbacks as well.  I've got a somewhat-temporary solution in place for
that currently:

    d = async.dict()
    def foo():
        # async.rdtsc() is a helper method
        # that basically wraps the result of
        # the assembly RDTSC (read time-
        # stamp counter) instruction into a
        # PyLong object.  So, it's handy when
        # I need to test the very functionality
        # being demonstrated here (creating
        # an object within a parallel context
        # and persisting it elsewhere).
        d['foo'] = async.rdtsc()

    def bar():
        d['bar'] = async.rdtsc()

    async.submit_work(foo)
    async.submit_work(bar)

That'll result in two contexts being created, one for each callback
invocation.  ``async.dict()`` is a "parallel safe" wrapper around a
normal PyDict.  This is referred to as "protection".

In fact, the code above could have been written as follows:

    d = async.protect(dict())

What ``protect()`` does is instrument the object such that we intercept
``__getitem__``, ``__setitem__``, ``__getattr__`` and ``__setattr__``.
We replace these methods with counterparts that serve two purposes:

 1. The read-only methods are wrapped in a read-lock, the write
    methods are wrapped in a write lock (using underlying system slim
    read/write locks, which are uber fast).  (Basically, you can have
    unlimited readers holding the read lock, but only one writer can
    hold the write lock (excluding all the readers and other writers).)

 2. Detecting when parallel objects (objects created from within a
    parallel thread, and thus, backed by the parallel context's heap)
    have been assigned outside the context (in this case, to a
    "protected" dict object that was created from the main thread).

The first point is important as it ensures concurrent access doesn't
corrupt the data structure.

The second point is important because it allows us to prevent the
persisted object's context from automatically transitioning into the
complete->release->heapdestroy lifecycle when the callback completes.

This is known as "persistence", as in, a context has been persisted.
All sorts of things happen to the object when we detect that it's been
persisted.  The biggest thing is that reference counting is enabled
again for the object (from the perspective of the main thread; ref
counting is still a no-op within the parallel thread) -- however, once
the refcount hits zero, instead of free()ing the memory like we'd
normally do in the main thread (or garbage collecting it), we decref
the reference count of the owning context.

Once the owning context's refcount goes to zero, we know that no more
references exist to objects created from that parallel thread's
execution, and we're free to release the context (and thus, destroy the
heap -> free the memory).

That's currently implemented and works very well.  There are a few
drawbacks: one, the user must only assign to an "async protected"
object.  Use a normal dict and you're going to segfault or corrupt
things (or worse) pretty quickly.

Second, we're persisting the entire context potentially for a single
object.  The context may be huge; think of some data processing callback
that ran for ages, racked up a 100MB footprint, but only generated a
PyLong with the value 42 at the end, which consumes, like, 50 bytes (or
whatever the size of a PyLong is these days).

It's crazy keeping a 100MB context around indefinitely until that PyLong
object goes away, so, we need another option.  The idea I have for that
is "promotion".  Rather than persist the context, the object is
"promoted"; basically, the parallel thread palms it off to the main
thread, which proceeds to deep-copy the object, and take over ownership.
This removes the need for the context to be persisted.

Now, I probably shouldn't have said "deep-copy" there.  Promotion is a
terrible option for anything other than simple objects (scalars).  If
you've got a huge list that consumes 98% of your 100MB heap footprint,
well, persistence is perfect.  If it's a 50 byte scalar, promotion is
perfect.  (Also, deep-copy implies collection interrogation, which has
all sorts of complexities, so, err, I'll probably end up supporting
promotion if the object is a scalar that can be shallow-copied.  Any
form of collection or non-scalar type will get persisted by default.)

I haven't implemented promotion yet (persistence works well enough for
now).  And none of this is integrated into the heap snapshot/rollback
logic -- i.e. we don't detect if a client/server callback assigned an
object created in the parallel context to a main-thread object -- we
just roll back blindly as soon as the callback completes.

Before this ever has a chance of being eligible for adoption into
CPython, those problems will need to be addressed.  As much as I'd like
to ignore those corner cases that violate the shared-nothing approach --
it's inevitable someone, somewhere, will be assigning parallel objects
outside of the context, maybe for good reason, maybe by accident, maybe
because they don't know any better.  Whatever the reason, the result
shouldn't be corruption.

So, the remaining challenge is preventing the use case alluded to
earlier where someone tries to modify an object that hasn't been "async
protected".  That's a bit harder.  The idea I've got in mind is to
instrument the main CPython ceval loop, such that we do these checks as
part of opcode processing.  That allows us to keep all the logic in the
one spot and not have to go hacking the internals of every single
object's C backend to ensure correctness.

Now, that'll probably work to an extent.  I mean, after all, there are
opcodes for all the things we'd be interested in instrumenting,
LOAD_GLOBAL, STORE_GLOBAL, SETITEM etc.  What becomes challenging is
detecting arbitrary mutations via object calls, i.e. how do we know,
during the ceval loop, that foo.append(x) needs to be treated specially
if foo is a main-thread object and x is a parallel thread object?

There may be no way to handle that *other* than hacking the internals of
each object, unfortunately.  So, the viability of this whole approach
may rest on whether or that's deemed as an acceptable tradeoff (a
necessary evil, even) to the Python developer community.

If it's not, then it's unlikely this approach will ever see the light of
day in CPython.  If that turns out to be the case, then I see this
project taking the path that Stackless took (forking off and becoming a
separate interpreter).

There's nothing wrong with that; I am really excited about the
possibilities afforded by this approach, and I'm sure it will pique the
interest of commercial entities out there that have problems perfectly
suited to where this pattern excels (shared-nothing, highly concurrent),
much like the relationship that developed between Stackless and Eve
Online.

So, it'd be great if it eventually saw the light of day in CPython, but
that'll be a long way down the track (at least 4.x I'd say), and all
these issues that allow you to instantly segfault or corrupt the
interpreter will need to be addressed before it's even eligible for
*discussion* about inclusion.
</snip>

    Regards,

        Trent.

From stefanrin at gmail.com  Thu Mar 14 20:59:57 2013
From: stefanrin at gmail.com (Stefan Ring)
Date: Thu, 14 Mar 2013 20:59:57 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314182352.GC24307@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
Message-ID: <CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>

>     Yup, in fact, if I hadn't come up with the __read[gf]sword() trick,
>     my only other option would have been TLS (or the GetCurrentThreadId
>     /pthread_self() approach in the presentation).  TLS is fantastic,
>     and it's definitely an intrinsic part of the solution (the "Y" part
>     of "if we're a parallel thread, do Y"), but it definitely more
>     costly than a simple FS/GS register read.

I think you should be able to just take the address of a static
__thread variable to achieve the same thing in a more portable way.

From trent at snakebite.org  Thu Mar 14 22:30:14 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 14:30:14 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314184519.GD24307@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<20130314184519.GD24307@snakebite.org>
Message-ID: <20130314212949.GE24307@snakebite.org>

    Cross-referenced to relevant bits of code where appropriate.

    (And just a quick reminder regarding the code quality disclaimer:
    I've been hacking away on this stuff relentlessly for a few months;
    the aim has been to make continual forward progress without getting
    bogged down in non-value-add busy work.  Lots of wildly inconsistent
    naming conventions and dead code that'll be cleaned up down the
    track.  And the relevance of any given struct will tend to be
    proportional to how many unused members it has (homeless hoarder +
    shopping cart analogy).)

On Thu, Mar 14, 2013 at 11:45:20AM -0700, Trent Nelson wrote:
> The basic premise is that parallel 'Context' objects (well, structs)
> are allocated for each parallel thread callback.

    The 'Context' struct:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel_private.h#l546

    Allocated via new_context():

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l4211

    ....also relevant, new_context_for_socket() (encapsulates a
    client/server instance within a context).

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l4300

    Primary role of the context is to isolate the memory management.
    This is achieved via 'Heap':

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel_private.h#l281

    (Which I sort of half started refactoring to use the _HEAD_EXTRA
     approach when I thought I'd need to have a separate heap type for
     some TLS avenue I explored -- turns out that wasn't necessary).

> The context persists for the lifetime of the "parallel work".
> 
> The "lifetime of the parallel work" depends on what you're doing.  For
> a simple ``async.submit_work(foo)``, the context is considered
> complete once ``foo()`` has been called (presuming no exceptions were
> raised).

    Managing context lifetime is one of the main responsibilities of
    async.run_once():

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3841

> For an async client/server, the context will persist for the entirety
> of the connection.

    Marking a socket context as 'finished' for servers is the job of
    PxServerSocket_ClientClosed():

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l6885

> The context is responsible for encapsulating all resources related to
> the parallel thread.  So, it has its own heap, and all memory
> allocations are taken from that heap.

    The heap is initialized in two steps during new_context().  First,
    a handle is allocated for the underlying system heap (via
    HeapCreate):

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l4224

    The first "heap" is then initialized for use with our context via
    the Heap_Init(Context *c, size_t n, int page_size) call:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l1921

    Heaps are actually linked together via a doubly-linked list.  The
    first heap is a value member (not a pointer) of Context; however,
    the active heap is always accessed via the '*h' pointer which is
    updated as necessary.

        struct Heap {
            Heap *prev;
            Heap *next;
            void *base;
            void *next;
            int   allocated;
            int   remaining;
            ...

        struct Context {
            Heap  heap;
            Heap *h;
            ...

> For any given parallel thread, only one context can be executing at a
> time, and this can be accessed via the ``__declspec(thread) Context
> *ctx`` global (which is primed by some glue code as soon as the
> parallel thread starts executing a callback).

    Glue entry point for all callbacks is _PyParallel_EnteredCallback:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3047

    On the topic of callbacks, the main workhorse for the
    submit_(wait|work) callbacks is _PyParallel_WorkCallback:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3120

    The interesting logic starts at start:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3251

    The interesting part is the error handling.  If the callback raises
    an exception, we check to see if an errback has been provided.  If
    so, we call the errback with the error details.

    If the callback completes successfully (or it fails, but the errback
    completes successfully), that is treated as successful callback or
    errback completion, respectively:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3270

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3294

    If the errback fails, or no errback was provided, the exception
    percolates back to the main thread.  This is handled at error:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3300

    This should make the behavior of async.run_once() clearer.  The
    first thing it does is check to see if any errors have been posted.

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3917

    Errors are returned back to calling code on a first-error-wins
    basis.  (This involves fiddling with the context's lifetime, as
    we're essentially propagating an object created in a parallel
    context (the (exception, value, traceback) tuple) back to a main
    thread context -- so, we can't blow away that context until the
    exception has had a chance to properly bubble back up and be dealt
    with.)

    If there are no errors, we then check to see if any "call from main
    thread" requests have been made:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3936

    I added support for this in order to ease unit testing, but it has
    general usefulness.  It's exposed via two decorators:

    @async.call_from_main_thread
    def foo(arg):
        ...

    def callback():
        foo('abcd')

    async.submit_work(callback)

    That creates a parallel thread, invokes callback(), which then
    results in foo(arg) eventually being called from the main thread.
    This would be useful for synchronising access to a database or
    something like that.

    There's also @async.call_from_main_thread_and_wait, which I probably
    should have mentioned first:

    @async.call_from_main_thread_and_wait
    def update_login_details(login, details)
        db.update(login, details)

    def foo():
        ...
        update_login_details(x, y)
        # execution will resume when the main thread finishes
        # update_login_details()
        ...

    async.submit_work(foo)

    Once all "main thread work requests" have been processed, completed
    callbacks and errbacks are processed.  This basically just involves
    transitioning the associated context onto the "path to freedom" (the
    lifecycle that eventually results in the context being free()'d and
    the heap being destroyed).

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l4032

> No reference counting or garbage collection is done during parallel
> thread execution.  Instead, once the context is finished, it is
> scheduled to be released, which means it'll be "processed" by the main
> thread as part of its housekeeping work (during ``async.run()``
> (technically, ``async.run_once()``).
> 
> The main thread simply destroys the entire heap in one fell swoop,
> releasing all memory that was associated with that context.

    The "path to freedom" lifecycle is a bit complicated at the moment
    and could definitely use a review.  But, basically, the main methods
    are _PxState_PurgeContexts() and _PxState_FreeContext(); the former
    checks that the context is ready to be freed, the latter does the
    actual freeing.

    _PxState_PurgeContexts:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3789

    _PxState_FreeContext:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l3700

    The reason for the separation is to maintain bubbling effect; a
    context only makes one transition per run_once() invocation.
    Putting this in place was a key step to stop wild crashes in the
    early days when unittest would keep hold of exceptions longer than I
    was expecting -- it should probably be reviewed in light of the new
    persistence support I implemented (much later).

> There are a few side effects to this.  First, the heap allocator
> (basically, the thing that answers ``malloc()`` calls) is incredibly
> simple.  It allocates LARGE_PAGE_SIZE chunks of memory at a time (2MB
> on x64), and simply returns pointers to that chunk for each memory
> request (adjusting h->next and allocation stats as it goes along,
> obviously).  Once the 2MB has been exhausted, another 2MB is
> allocated.

    _PyHeap_Malloc is the workhorse here:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l2183

    Very simple, just keeps nudging along the h->next pointer for each
    request, allocating another heap when necessary.  Nice side effect
    is that it's ridiculously fast and very cache friendly.  Python code
    running within parallel contexts runs faster than normal main-thread
    code because of this (plus the boost from not doing any ref
    counting).

    The simplicity of this approach made the heap snapshot logic really
    simple to implement too; taking a snapshot and then rolling back is
    just a couple of memcpy's and some pointer fiddling.

> That approach is fine for the ``submit_(work|timer|wait)`` callbacks,
> which basically provide a way to run a presumably-finite-length
> function in a parallel thread (and invoking callbacks/errbacks as
> required).
> 
> However, it breaks down when dealing with client/server stuff.  Each
> invocation of a callback (say, ``data_received(...)``) may only
> consume, say, 500 bytes, but it might be called a million times before
> the connection is terminated.  You can't have cumulative memory usage
> with possibly-infinite-length client/server-callbacks like you can
> with the once-off ``submit_(work|wait|timer)`` stuff.
> 
> So, enter heap snapshots.  The logic that handles all client/server
> connections is instrumented such that it takes a snapshot of the heap
> (and all associated stats) prior to invoking a Python method (via
> ``PyObject_Call()``, for example, i.e. the invocation of
> ``data_received``).

    I came up with the heap snapshot stuff in a really perverse way.
    The first cut introduced a new 'TLS heap' concept; the idea was that
    before you'd call PyObject_CallObject(), you'd enable the TLS heap,
    then roll it back when you were done.  i.e. the socket IO loop code
    had a lot of stuff like this:

    snapshot = ENABLE_TLS_HEAP();
    if (!PyObject_CallObject(...)) {
        DISABLE_TLS_HEAP_AND_ROLLBACK(snapshot);
        ...
    }
    DISABLE_TLS_HEAP();

    ...
    /* do stuff */
    ROLLBACK_TLS_HEAP(snapshot);

    That was fine initially, until I had to deal with the (pretty
    common) case of allocating memory from the TLS heap (say, for an
    async send), and then having the callback picked up by a different
    thread.  That thread then had to return the other thread's snapshot
    and, well, it just fell apart conceptually.

    Then it dawned on me to just add the snapshot/rollback stuff to
    normal Context objects.  In retrospect, it's silly I didn't think of
    this in the first place -- the biggest advantage of the Context
    abstraction is that it's thread-local, but not bindingly so (as in,
    it'll only ever run on one thread at a time, but it doesn't matter
    which one, which is essential, because the ).

    Once I switched out all the TLS heap cruft for Context-specific heap
    snapshots, everything "Just Worked".  (I haven't removed the TLS
    heap stuff yet as I'm still using it elsewhere (where it doesn't
    have the issue above).  It's an xxx todo.)

    The main consumer of this heap snapshot stuff (at the moment) is the
    socket IO loop logic:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l5632

    Typical usage now looks like this:

       snapshot = PxContext_HeapSnapshot(c, NULL);
       if (!PxSocket_LoadInitialBytes(s)) {
           PxContext_RollbackHeap(c, &snapshot);
           PxSocket_EXCEPTION();
       }

       /* at some later point... */
       PxContext_RollbackHeap(c, &snapshot);

> When the method completes, we can simply roll back the snapshot.  The
> heap's stats and next pointers et al all get reset back to what they
> were before the callback was invoked.
>
> The only issue with this approach is detecting when the callback has
> done the unthinkable (from a shared-nothing perspective) and persisted
> some random object it created outside of the parallel context it was
> created in.
> 
> That's actually a huge separate technical issue to tackle -- and it
> applies just as much to the normal ``submit_(wait|work|timer)``
> callbacks as well.  I've got a somewhat-temporary solution in place
> for that currently:
> 
> That'll result in two contexts being created, one for each callback
> invocation.  ``async.dict()`` is a "parallel safe" wrapper around a
> normal PyDict.  This is referred to as "protection".
> 
> In fact, the code above could have been written as follows:
> 
>     d = async.protect(dict())
> 
> What ``protect()`` does is instrument the object such that we
> intercept ``__getitem__``, ``__setitem__``, ``__getattr__`` and
> ``__setattr__``.

    The 'protect' details are pretty hairy.  _protect does a few checks:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l1368

    ....and then palms things off to _PyObject_PrepOrigType:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l1054

    That method is where the magic happens.  We basically clone the type
    object for the object we're protecting, then replace the setitem,
    getitem etc methods with our counterparts (described next):

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l1100

    Note the voodoo involved in 'protecting' heap objects versus normal
    C-type objects, GC objects versus non-GC, etc.

> We replace these methods with counterparts that
> serve two purposes:
> 
>  1. The read-only methods are wrapped in a read-lock, the write
>  methods are wrapped in a write lock (using underlying system slim
>  read/write locks, which are uber fast).  (Basically, you can have
>  unlimited readers holding the read lock, but only one writer can hold
>  the write lock (excluding all the readers and other writers).)
> 
>  2. Detecting when parallel objects (objects created from within a
>  parallel thread, and thus, backed by the parallel context's heap)
>  have been assigned outside the context (in this case, to a
>  "protected" dict object that was created from the main thread).

    This is handled via _Px_objobjargproc_ass:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l900

    That is responsible for detecting when a parallel object is being
    assigned to a non-parallel object (and tries to persist the object
    where necessary).

> The first point is important as it ensures concurrent access doesn't
> corrupt the data structure.
> 
> The second point is important because it allows us to prevent the
> persisted object's context from automatically transitioning into the
> complete->release->heapdestroy lifecycle when the callback completes.
> 
> This is known as "persistence", as in, a context has been persisted.
> All sorts of things happen to the object when we detect that it's been
> persisted.  The biggest thing is that reference counting is enabled
> again for the object (from the perspective of the main thread; ref
> counting is still a no-op within the parallel thread) -- however, once
> the refcount hits zero, instead of free()ing the memory like we'd
> normally do in the main thread (or garbage collecting it), we decref
> the reference count of the owning context.

    That's the job of _Px_TryPersist (called via _Px_objobjargproc_ass
    as mentioned above):

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l861

    That makes use of yet-another-incredibly-useful-Windows-feature
    called 'init once'; basically, underlying system support for
    ensuring something only gets done *once*.  Perfect for avoiding race
    conditions.

> Once the owning context's refcount goes to zero, we know that no more
> references exist to objects created from that parallel thread's
> execution, and we're free to release the context (and thus, destroy
> the heap -> free the memory).

    All that magic is the unfortunate reason my lovely Py_INCREF/DECREF
    overrides when from very simple to quite-a-bit-more-involved.  i.e.
    originally Py_INCREF was just:

#define Py_INCREF(o) (Py_PXCTX ? (void)0; Py_REFCNT(o)++);

    With the advent of parallel object persistence and context-specific
    refcounts, things become less simple:

    Py_INCREF:
http://hg.python.org/sandbox/trent/file/7148209d5490/Include/object.h#l890
   890 __inline
   891 void
   892 _Py_IncRef(PyObject *op)
   893 {
   894     if ((!Py_PXCTX && (Py_ISPY(op) || Px_PERSISTED(op)))) {
   895         _Py_INC_REFTOTAL;
   896         (((PyObject*)(op))->ob_refcnt++);
   897     }
   898 }


    Py_DECREF:
http://hg.python.org/sandbox/trent/file/7148209d5490/Include/object.h#l911
   909 __inline
   910 void
   911 _Py_DecRef(PyObject *op)
   912 {
   913     if (!Py_PXCTX) {
   914         if (Px_PERSISTED(op))
   915             Px_DECREF(op);
   916         else if (!Px_ISPX(op)) {
   917             _Py_DEC_REFTOTAL;
   918             if ((--((PyObject *)(op))->ob_refcnt) != 0) {
   919                 _Py_CHECK_REFCNT(op);
   920             } else
   921                 _Py_Dealloc((PyObject *)(op));
   922         }
   923     }
   924 }


> That's currently implemented and works very well.  There are a few
> drawbacks: one, the user must only assign to an "async protected"
> object.  Use a normal dict and you're going to segfault or corrupt
> things (or worse) pretty quickly.
> 
> Second, we're persisting the entire context potentially for a single
> object.  The context may be huge; think of some data processing
> callback that ran for ages, racked up a 100MB footprint, but only
> generated a PyLong with the value 42 at the end, which consumes, like,
> 50 bytes (or whatever the size of a PyLong is these days).
> 
> It's crazy keeping a 100MB context around indefinitely until that
> PyLong object goes away, so, we need another option.  The idea I have
> for that is "promotion".  Rather than persist the context, the object
> is "promoted"; basically, the parallel thread palms it off to the main
> thread, which proceeds to deep-copy the object, and take over
> ownership.  This removes the need for the context to be persisted.
> 
> Now, I probably shouldn't have said "deep-copy" there.  Promotion is a
> terrible option for anything other than simple objects (scalars).  If
> you've got a huge list that consumes 98% of your 100MB heap footprint,
> well, persistence is perfect.  If it's a 50 byte scalar, promotion is
> perfect.  (Also, deep-copy implies collection interrogation, which has
> all sorts of complexities, so, err, I'll probably end up supporting
> promotion if the object is a scalar that can be shallow-copied.  Any
> form of collection or non-scalar type will get persisted by default.)
> 
> I haven't implemented promotion yet (persistence works well enough for
> now).  And none of this is integrated into the heap snapshot/rollback
> logic -- i.e. we don't detect if a client/server callback assigned an
> object created in the parallel context to a main-thread object -- we
> just roll back blindly as soon as the callback completes.
> 
> Before this ever has a chance of being eligible for adoption into
> CPython, those problems will need to be addressed.  As much as I'd
> like to ignore those corner cases that violate the shared-nothing
> approach -- it's inevitable someone, somewhere, will be assigning
> parallel objects outside of the context, maybe for good reason, maybe
> by accident, maybe because they don't know any better.  Whatever the
> reason, the result shouldn't be corruption.
> 
> So, the remaining challenge is preventing the use case alluded to
> earlier where someone tries to modify an object that hasn't been
> "async protected".  That's a bit harder.  The idea I've got in mind is
> to instrument the main CPython ceval loop, such that we do these
> checks as part of opcode processing.  That allows us to keep all the
> logic in the one spot and not have to go hacking the internals of
> every single object's C backend to ensure correctness.
> 
> Now, that'll probably work to an extent.  I mean, after all, there are
> opcodes for all the things we'd be interested in instrumenting,
> LOAD_GLOBAL, STORE_GLOBAL, SETITEM etc.  What becomes challenging is
> detecting arbitrary mutations via object calls, i.e. how do we know,
> during the ceval loop, that foo.append(x) needs to be treated
> specially if foo is a main-thread object and x is a parallel thread
> object?
> 
> There may be no way to handle that *other* than hacking the internals
> of each object, unfortunately.  So, the viability of this whole
> approach may rest on whether or that's deemed as an acceptable
> tradeoff (a necessary evil, even) to the Python developer community.

    Actually, I'd sort of forgotten that I started adding protection
    support for lists in _PyObject_PrepOrigType.  Well, technically,
    support for intercepting PySequenceMethods:

http://hg.python.org/sandbox/trent/file/7148209d5490/Python/pyparallel.c#l1126

    I settled for just intercepting PyMappingMethods initially, which
    is why that chunk of code is commented out.  Intercepting the
    mapping methods allowed me to implement the async protection for
    dicts and generic objects, which was sufficient for testing
    purposes at the time.

    So, er, I guess my point is that automatically detecting object
    mutation might not be as hard as I'm alluding to above.  I'll be
    happy if we're able to simply raise an exception if you attempt to
    mutate a non-protected main-thread object.  That's infinitely better
    than segfaulting or silent corruption.


        Trent.

From trent at snakebite.org  Thu Mar 14 22:49:02 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 14:49:02 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314212949.GE24307@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<20130314184519.GD24307@snakebite.org>
	<20130314212949.GE24307@snakebite.org>
Message-ID: <20130314214901.GF24307@snakebite.org>

On Thu, Mar 14, 2013 at 02:30:14PM -0700, Trent Nelson wrote:
>     Then it dawned on me to just add the snapshot/rollback stuff to
>     normal Context objects.  In retrospect, it's silly I didn't think of
>     this in the first place -- the biggest advantage of the Context
>     abstraction is that it's thread-local, but not bindingly so (as in,
>     it'll only ever run on one thread at a time, but it doesn't matter
>     which one, which is essential, because the ).
> 
>     Once I switched ...

    $10 if you can guess when I took a break for lunch.

    "....but it doesn't matter which one, which is essential, because
    there are no guarantees with regards to which thread runs which
    context."

    Is along the lines of what I was going to say.

        Trent.

From ani at aristanetworks.com  Thu Mar 14 23:15:06 2013
From: ani at aristanetworks.com (Ani Sinha)
Date: Thu, 14 Mar 2013 15:15:06 -0700
Subject: [Python-Dev] About issue 6560
Message-ID: <CAOxq_8Mb=FMUA+ssUbimduKXr6g61dvNQGwS-q_jQ4NgBFUg5w@mail.gmail.com>

Hi :

I was looking into a mechanism to get the aux fields from recvmsg() in
python and I came across this issue. Looks like this feature was added
in python 3.3. Is there any reason why this feature was not added for
python 2.7? I am now trying to backport the patch to python 2.7.

any insight into this would be appreciated.

Thanks
ani

From trent at snakebite.org  Thu Mar 14 23:23:37 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 15:23:37 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
Message-ID: <20130314222337.GG24307@snakebite.org>

On Thu, Mar 14, 2013 at 12:59:57PM -0700, Stefan Ring wrote:
> >     Yup, in fact, if I hadn't come up with the __read[gf]sword() trick,
> >     my only other option would have been TLS (or the GetCurrentThreadId
> >     /pthread_self() approach in the presentation).  TLS is fantastic,
> >     and it's definitely an intrinsic part of the solution (the "Y" part
> >     of "if we're a parallel thread, do Y"), but it definitely more
> >     costly than a simple FS/GS register read.
> 
> I think you should be able to just take the address of a static
> __thread variable to achieve the same thing in a more portable way.

    Sure, but, uh, that's kinda' trivial in comparison to all the wildly
    unportable Windows-only functionality I'm using to achieve all of
    this at the moment :-)

    For the record, here are all the Windows calls I'm using that have
    no *direct* POSIX equivalent:

        Interlocked singly-linked lists:
            - InitializeSListHead()
            - InterlockedFlushSList()
            - QueryDepthSList()
            - InterlockedPushEntrySList()
            - InterlockedPushListSList()
            - InterlockedPopEntrySlist()

        Synchronisation and concurrency primitives:
            - Critical sections
                - InitializeCriticalSectionAndSpinCount()
                - EnterCriticalSection()
                - LeaveCriticalSection()
                - TryEnterCriticalSection()
            - Slim read/writer locks (some pthread implements have
              rwlocks)*:
                - InitializeSRWLock()
                - AcquireSRWLockShared()
                - AcquireSRWLockExclusive()
                - ReleaseSRWLockShared()
                - ReleaseSRWLockExclusive()
                - TryAcquireSRWLockExclusive()
                - TryAcquireSRWLockShared()
            - One-time initialization:
                - InitOnceBeginInitialize()
                - InitOnceComplete()
            - Generic event, signalling and wait facilities:
                - CreateEvent()
                - SetEvent()
                - WaitForSingleObject()
                - WaitForMultipleObjects()
                - SignalObjectAndWait()

        Native thread pool facilities:
            - TrySubmitThreadpoolCallback()
            - StartThreadpoolIo()
            - CloseThreadpoolIo()
            - CancelThreadpoolIo()
            - DisassociateCurrentThreadFromCallback()
            - CallbackMayRunLong()
            - CreateThreadpoolWait()
            - SetThreadpoolWait()

        Memory management:
            - HeapCreate()
            - HeapAlloc()
            - HeapDestroy()

        Structured Exception Handling (#ifdef Py_DEBUG):
            - __try/__except

        Sockets:
            - ConnectEx()
            - AcceptEx()
            - WSAEventSelect(FD_ACCEPT)
            - DisconnectEx(TF_REUSE_SOCKET)
            - Overlapped WSASend()
            - Overlapped WSARecv()


    Don't get me wrong, I grew up with UNIX and love it as much as the
    next guy, but you can't deny the usefulness of Windows' facilities
    for writing high-performance, multi-threaded IO code.  It's decades
    ahead of POSIX.  (Which is also why it bugs me when I see select()
    being used on Windows, or IOCP being used as if it were a poll-type
    "generic IO multiplexor" -- that's like having a Ferrari and speed
    limiting it to 5mph!)

    So, before any of this has a chance of working on Linux/BSD, a lot
    more scaffolding will need to be written to provide the things we
    get for free on Windows (threadpools being the biggest freebie).


        Trent.

From martin at v.loewis.de  Thu Mar 14 23:56:33 2013
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Mar 2013 15:56:33 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314182352.GC24307@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
Message-ID: <514255A1.1020308@v.loewis.de>

Am 14.03.13 11:23, schrieb Trent Nelson:
>> ARM CPUs don't have segment registers because they have a simpler
>> addressing model. The register CP15 came up after a couple of Google
>> searches.
>
>      Noted, thanks!
>
>      Yeah that's my general sentiment too.  I'm definitely curious to see
>      if other ISAs offer similar facilities (Sparc, IA64, POWER etc), but
>      the hierarchy will be x86/x64 > ARM > * for the foreseeable future.

Most (in particular the RISC ones) do have a general-purpose register 
reserved for TLS. For ARM, the interesting thing is that CP15 apparently
is not available on all ARM implementations, and Linux then emulates it
on processors that don't have it (by handling the trap), which is
costly. Additionally, it appears that Android fails to provide that 
emulation (in some versions, on some processors), so that seems to be
tricky ground.

>      Porting the Py_PXCTX part is trivial compared to the work that is
>      going to be required to get this stuff working on POSIX where none
>      of the sublime Windows concurrency, synchronisation and async IO
>      primitives exist.

I couldn't understand from your presentation why this is essential
to your approach. IIUC, you are "just" relying on the OS providing
a thread pool, (and the sublime concurrency and synchronization
routines are nothing more than that, ISTM). Implementing a thread
pool on top of select/poll/kqueue seems straight-forward.

Regards,
Martin



From martin at v.loewis.de  Thu Mar 14 23:48:33 2013
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Mar 2013 15:48:33 -0700
Subject: [Python-Dev] About issue 6560
In-Reply-To: <CAOxq_8Mb=FMUA+ssUbimduKXr6g61dvNQGwS-q_jQ4NgBFUg5w@mail.gmail.com>
References: <CAOxq_8Mb=FMUA+ssUbimduKXr6g61dvNQGwS-q_jQ4NgBFUg5w@mail.gmail.com>
Message-ID: <514253C1.9060209@v.loewis.de>

Am 14.03.13 15:15, schrieb Ani Sinha:
> I was looking into a mechanism to get the aux fields from recvmsg() in
> python and I came across this issue. Looks like this feature was added
> in python 3.3. Is there any reason why this feature was not added for
> python 2.7?

Most certainly: Python 2.7 (and thus Python 2) is feature-frozen; no
new features can be added to it. People wanting new features need to
port to Python 3.

Regards,
Martin


From trent at snakebite.org  Fri Mar 15 00:00:09 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 16:00:09 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <51425433.1090700@v.loewis.de>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<51425433.1090700@v.loewis.de>
Message-ID: <20130314230007.GA24799@snakebite.org>

On Thu, Mar 14, 2013 at 03:50:27PM -0700, "Martin v. L?wis" wrote:
> Am 14.03.13 12:59, schrieb Stefan Ring:
> > I think you should be able to just take the address of a static
> > __thread variable to achieve the same thing in a more portable way.
> 
> That assumes that the compiler supports __thread variables, which
> isn't that portable in the first place.

    FWIW, I make extensive use of __declspec(thread).  I'm aware of GCC
    and Clang's __thread alternative.  No idea what IBM xlC, Sun Studio
    and others offer, if anything.

        Trent.

From martin at v.loewis.de  Thu Mar 14 23:50:27 2013
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 14 Mar 2013 15:50:27 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
Message-ID: <51425433.1090700@v.loewis.de>

Am 14.03.13 12:59, schrieb Stefan Ring:
> I think you should be able to just take the address of a static
> __thread variable to achieve the same thing in a more portable way.

That assumes that the compiler supports __thread variables, which
isn't that portable in the first place.

Regards,
Martin


From tjreedy at udel.edu  Fri Mar 15 00:19:11 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 14 Mar 2013 19:19:11 -0400
Subject: [Python-Dev] About issue 6560
In-Reply-To: <514253C1.9060209@v.loewis.de>
References: <CAOxq_8Mb=FMUA+ssUbimduKXr6g61dvNQGwS-q_jQ4NgBFUg5w@mail.gmail.com>
	<514253C1.9060209@v.loewis.de>
Message-ID: <khtltc$527$2@ger.gmane.org>

On 3/14/2013 6:48 PM, "Martin v. L?wis" wrote:
> Am 14.03.13 15:15, schrieb Ani Sinha:
>> I was looking into a mechanism to get the aux fields from recvmsg() in
>> python and I came across this issue. Looks like this feature was added
>> in python 3.3. Is there any reason why this feature was not added for
>> python 2.7?
>
> Most certainly: Python 2.7 (and thus Python 2) is feature-frozen; no

As are 3.2 and now 3.3. Every version is feature frozen when released. 
Bugfix releases only contain bugfixes.

> new features can be added to it. People wanting new features need to
> port to Python 3.

In particular 3.3.

-- 
Terry Jan Reedy



From trent at snakebite.org  Fri Mar 15 00:21:14 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 14 Mar 2013 16:21:14 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <514255A1.1020308@v.loewis.de>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<514255A1.1020308@v.loewis.de>
Message-ID: <20130314232111.GB24799@snakebite.org>

On Thu, Mar 14, 2013 at 03:56:33PM -0700, "Martin v. L?wis" wrote:
> Am 14.03.13 11:23, schrieb Trent Nelson:
> >      Porting the Py_PXCTX part is trivial compared to the work that is
> >      going to be required to get this stuff working on POSIX where none
> >      of the sublime Windows concurrency, synchronisation and async IO
> >      primitives exist.
> 
> I couldn't understand from your presentation why this is essential
> to your approach. IIUC, you are "just" relying on the OS providing
> a thread pool, (and the sublime concurrency and synchronization
> routines are nothing more than that, ISTM).

    Right, there's nothing Windows* does that can't be achieved on
    Linux/BSD, it'll just take more scaffolding (i.e. we'll need to
    manage our own thread pool at the very least).

    [*]: actually, the interlocked singly-linked list stuff concerns me;
    the API seems straightforward enough but the implementation becomes
    deceptively complex once you factor in the ABA problem.  (I'm not
    aware of a portable open source alternative for that stuff.)

> Implementing a thread pool on top of select/poll/kqueue seems
> straight-forward.

    Nod, that's exactly what I've got in mind.  Spin up a bunch of
    threads that sit there and call poll/kqueue in an endless loop.
    That'll work just fine for Linux/BSD/OSX.

    Actually, what's really interesting is the new registered IO
    facilities in Windows 8/2012.  The Microsoft recommendation for
    achieving the ultimate performance (least amount of jitter, lowest
    latency, highest throughput) is to do something like this:

        while (1) {

            if (!DequeueCompletionRequests(...)) {
                YieldProcessor();
                continue;
            } else {
                /* Handle requests */
            }
        }

    That pattern looks a lot more like what you'd do on Linux/BSD (spin
    up a thread per CPU and call epoll/kqueue endlessly) than any of the
    previous Windows IO patterns.


        Trent.

From tjreedy at udel.edu  Fri Mar 15 02:33:05 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 14 Mar 2013 21:33:05 -0400
Subject: [Python-Dev] Matching __all__ to doc: bugfix or enhancement?
Message-ID: <khttof$eed$1@ger.gmane.org>

The timeit doc describes four public attributes.
The current timeit.__all__ only lists one.
http://bugs.python.org/issue17414
proposes to expand __all__ to include all four:
-__all__ = ["Timer"]
+__all__ = ["Timer", "timeit", "repeat", "default_timer"]

The effect of the change is
a) help(timit) will mention the three functions as well as the class;
b) IDLE's attribute completion box* will list all four instead just Timer;
c) unknow other users of .__all__ will see the expanded list, for better 
or worse.

* Typing 'xxx.' and either waiting or typing cntl-space brings up a 
listbox of attributes to select from.

Is the code change an all-version bugfix or a default-only enhancement?
I can see it both ways, but a decision is required to act.

PS: I think the devguide should gain a new 'Behavior versus Enhancement' 
section after the current "11.1.2. Type" to clarify issues like this.

-- 
Terry Jan Reedy


From fred at fdrake.net  Fri Mar 15 02:47:52 2013
From: fred at fdrake.net (Fred Drake)
Date: Thu, 14 Mar 2013 21:47:52 -0400
Subject: [Python-Dev] Matching __all__ to doc: bugfix or enhancement?
In-Reply-To: <khttof$eed$1@ger.gmane.org>
References: <khttof$eed$1@ger.gmane.org>
Message-ID: <CAFT4OTGNr-LA-pTjmGNifaUX=qyb8q-4+i-mT_LE3WRum6a43g@mail.gmail.com>

On Thu, Mar 14, 2013 at 9:33 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> Is the code change an all-version bugfix or a default-only enhancement?
> I can see it both ways, but a decision is required to act.

This is actually backward-incompatible, so should not be considered a
simple bugfix.  If determined to be desirable, it should not be applied
to any version before 3.4.


  -Fred

-- 
Fred L. Drake, Jr.    <fred at fdrake.net>
"A storm broke loose in my mind."  --Albert Einstein

From eliben at gmail.com  Fri Mar 15 02:54:34 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 14 Mar 2013 18:54:34 -0700
Subject: [Python-Dev] Matching __all__ to doc: bugfix or enhancement?
In-Reply-To: <khttof$eed$1@ger.gmane.org>
References: <khttof$eed$1@ger.gmane.org>
Message-ID: <CAF-Rda997ArsmywGoZ28GTuoo-XrhdFWA1OY7zrXLtUb7PNgog@mail.gmail.com>

On Thu, Mar 14, 2013 at 6:33 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> The timeit doc describes four public attributes.
> The current timeit.__all__ only lists one.
> http://bugs.python.org/**issue17414 <http://bugs.python.org/issue17414>
> proposes to expand __all__ to include all four:
> -__all__ = ["Timer"]
> +__all__ = ["Timer", "timeit", "repeat", "default_timer"]
>
> The effect of the change is
> a) help(timit) will mention the three functions as well as the class;
> b) IDLE's attribute completion box* will list all four instead just Timer;
> c) unknow other users of .__all__ will see the expanded list, for better
> or worse.
>
>
Another effect is that existing code that does:

from timeit import *

May break. The above may not be the recommended best practice in Python,
but it's perfectly valid and widely used.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130314/60b3f519/attachment.html>

From guido at python.org  Fri Mar 15 05:15:08 2013
From: guido at python.org (Guido van Rossum)
Date: Thu, 14 Mar 2013 21:15:08 -0700
Subject: [Python-Dev] Matching __all__ to doc: bugfix or enhancement?
In-Reply-To: <CAF-Rda997ArsmywGoZ28GTuoo-XrhdFWA1OY7zrXLtUb7PNgog@mail.gmail.com>
References: <khttof$eed$1@ger.gmane.org>
	<CAF-Rda997ArsmywGoZ28GTuoo-XrhdFWA1OY7zrXLtUb7PNgog@mail.gmail.com>
Message-ID: <CAP7+vJ+_hpLGDtvtcUgxqR=FtW=tJ38WokqUReaoA4JwD5S6zw@mail.gmail.com>

So it's a new feature, albeit a small one. I do see that it shouldn't
be backported, but I don't see any worries about doing it in 3.4.
Adding new functions/classes/constants to modules happens all the
time, and we never give a second thought to users of import *. :-)

On Thu, Mar 14, 2013 at 6:54 PM, Eli Bendersky <eliben at gmail.com> wrote:
>
>
>
> On Thu, Mar 14, 2013 at 6:33 PM, Terry Reedy <tjreedy at udel.edu> wrote:
>>
>> The timeit doc describes four public attributes.
>> The current timeit.__all__ only lists one.
>> http://bugs.python.org/issue17414
>> proposes to expand __all__ to include all four:
>> -__all__ = ["Timer"]
>> +__all__ = ["Timer", "timeit", "repeat", "default_timer"]
>>
>> The effect of the change is
>> a) help(timit) will mention the three functions as well as the class;
>> b) IDLE's attribute completion box* will list all four instead just Timer;
>> c) unknow other users of .__all__ will see the expanded list, for better
>> or worse.
>>
>
> Another effect is that existing code that does:
>
> from timeit import *
>
> May break. The above may not be the recommended best practice in Python, but
> it's perfectly valid and widely used.
>
> Eli
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org
>



--
--Guido van Rossum (python.org/~guido)

From eliben at gmail.com  Fri Mar 15 05:24:51 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 14 Mar 2013 21:24:51 -0700
Subject: [Python-Dev] Matching __all__ to doc: bugfix or enhancement?
In-Reply-To: <CAP7+vJ+_hpLGDtvtcUgxqR=FtW=tJ38WokqUReaoA4JwD5S6zw@mail.gmail.com>
References: <khttof$eed$1@ger.gmane.org>
	<CAF-Rda997ArsmywGoZ28GTuoo-XrhdFWA1OY7zrXLtUb7PNgog@mail.gmail.com>
	<CAP7+vJ+_hpLGDtvtcUgxqR=FtW=tJ38WokqUReaoA4JwD5S6zw@mail.gmail.com>
Message-ID: <CAF-Rda_iBDtj=cHfDbyRZKCmQdy5a7f_Q-cyFujL=ndKd6XB-w@mail.gmail.com>

On Thu, Mar 14, 2013 at 9:15 PM, Guido van Rossum <guido at python.org> wrote:

> So it's a new feature, albeit a small one. I do see that it shouldn't
> be backported, but I don't see any worries about doing it in 3.4.
> Adding new functions/classes/constants to modules happens all the
> time, and we never give a second thought to users of import *. :-)
>
>
Oh yes, I agree there should be no problem in the default branch. My
comment was mainly aimed at backporting it; I should've made it clearer.

Eli



> On Thu, Mar 14, 2013 at 6:54 PM, Eli Bendersky <eliben at gmail.com> wrote:
> >
> >
> >
> > On Thu, Mar 14, 2013 at 6:33 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> >>
> >> The timeit doc describes four public attributes.
> >> The current timeit.__all__ only lists one.
> >> http://bugs.python.org/issue17414
> >> proposes to expand __all__ to include all four:
> >> -__all__ = ["Timer"]
> >> +__all__ = ["Timer", "timeit", "repeat", "default_timer"]
> >>
> >> The effect of the change is
> >> a) help(timit) will mention the three functions as well as the class;
> >> b) IDLE's attribute completion box* will list all four instead just
> Timer;
> >> c) unknow other users of .__all__ will see the expanded list, for better
> >> or worse.
> >>
> >
> > Another effect is that existing code that does:
> >
> > from timeit import *
> >
> > May break. The above may not be the recommended best practice in Python,
> but
> > it's perfectly valid and widely used.
> >
> > Eli
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> > http://mail.python.org/mailman/options/python-dev/guido%40python.org
> >
>
>
>
> --
> --Guido van Rossum (python.org/~guido)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130314/f94e3acf/attachment.html>

From nad at acm.org  Fri Mar 15 06:55:10 2013
From: nad at acm.org (Ned Deily)
Date: Thu, 14 Mar 2013 22:55:10 -0700
Subject: [Python-Dev] Followup - Re: Bad python 2.5 build on OSX 10.8
	mountain lion
References: <CAP7+vJLQLLgDOB83XBg9-=+FyS7+1qEkr7inAL5igD4bJHpiiA@mail.gmail.com>
	<CAP7+vJLYBgPCmKNT5UVzQ9Na3qBdumE9CmJ5PL6yZqR75AyAnw@mail.gmail.com>
	<nad-AD48B0.00182002102012@news.gmane.org>
	<20121002073135.GA26567@sleipnir.bytereef.org>
	<nad-B4E67A.00475902102012@news.gmane.org>
	<nad-7C84D6.16451105102012@news.gmane.org>
Message-ID: <nad-C3016B.22551014032013@news.gmane.org>

Way back on 2012-10-05 23:45:11 GMT in article 
<nad-7C84D6.16451105102012 at news.gmane.org>, I wrote:

> In article <nad-B4E67A.00475902102012 at news.gmane.org>,
>  Ned Deily <nad at acm.org> wrote:
> > In article <20121002073135.GA26567 at sleipnir.bytereef.org>,
> >  Stefan Krah <stefan at bytereef.org> wrote:
> > > Ned Deily <nad at acm.org> wrote:
> > > > > Forgot the link...
> > > > > http://code.google.com/p/googleappengine/issues/detail?id=7885
> > > > > On Monday, October 1, 2012, Guido van Rossum wrote:
> > > > > > As discussed here, the python 2.5 binary distributed by Apple on 
> > > > > > mountain
> > > > > > lion is broken. Could someone file an official complaint?
> > > > I've filed a bug against 10.8 python2.5.   The 10.8 versions of Apple's 
> > > > pythons are compile with clang and we did see some sign extension 
> > > > issues 
> > > > with ctypes.  The 10.7 version of Apple's python2.5 is compiled with 
> > > > llvm-gcc and handles 2**31 correctly.
> > > Yes, this looks like http://bugs.python.org/issue11149 . 
> > Ah, right, thanks.  I've updated the Apple issue accordingly.
> 
> Update: the bug I filed has been closed as a duplicate of #11932488 
> which apparently at the moment is still open.  No other information is 
> available.

FYI, today Apple finally released OS X 10.8.3, the next maintenance 
release of Mountain Lion, and it does include a recompiled version of 
Python 2.5.6 that appears to solve the sign-extension problem:
2**31-1 is now 2147483647L.

-- 
 Ned Deily,
 nad at acm.org


From tjreedy at udel.edu  Fri Mar 15 08:09:58 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Fri, 15 Mar 2013 03:09:58 -0400
Subject: [Python-Dev] Matching __all__ to doc: bugfix or enhancement?
In-Reply-To: <CAP7+vJ+_hpLGDtvtcUgxqR=FtW=tJ38WokqUReaoA4JwD5S6zw@mail.gmail.com>
References: <khttof$eed$1@ger.gmane.org>
	<CAF-Rda997ArsmywGoZ28GTuoo-XrhdFWA1OY7zrXLtUb7PNgog@mail.gmail.com>
	<CAP7+vJ+_hpLGDtvtcUgxqR=FtW=tJ38WokqUReaoA4JwD5S6zw@mail.gmail.com>
Message-ID: <khuhg5$9do$1@ger.gmane.org>

On 3/15/2013 12:15 AM, Guido van Rossum wrote:
> So it's a new feature, albeit a small one. I do see that it shouldn't
> be backported, but I don't see any worries about doing it in 3.4.
> Adding new functions/classes/constants to modules happens all the
> time, and we never give a second thought to users of import *. :-)

Thanks all. Pushed to 3.4 only.

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Fri Mar 15 08:19:51 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 15 Mar 2013 08:19:51 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<514255A1.1020308@v.loewis.de>
	<20130314232111.GB24799@snakebite.org>
Message-ID: <20130315081951.43e30adb@pitrou.net>

On Thu, 14 Mar 2013 16:21:14 -0700
Trent Nelson <trent at snakebite.org> wrote:
> 
>     Actually, what's really interesting is the new registered IO
>     facilities in Windows 8/2012.  The Microsoft recommendation for
>     achieving the ultimate performance (least amount of jitter, lowest
>     latency, highest throughput) is to do something like this:
> 
>         while (1) {
> 
>             if (!DequeueCompletionRequests(...)) {
>                 YieldProcessor();
>                 continue;
>             } else {
>                 /* Handle requests */
>             }
>         }

Does Microsoft change their recommendations every couple of years? :)

Regards

Antoine.



From martin at v.loewis.de  Fri Mar 15 14:09:49 2013
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 15 Mar 2013 06:09:49 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130315081951.43e30adb@pitrou.net>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<514255A1.1020308@v.loewis.de>
	<20130314232111.GB24799@snakebite.org>
	<20130315081951.43e30adb@pitrou.net>
Message-ID: <51431D9D.5040103@v.loewis.de>

Am 15.03.13 00:19, schrieb Antoine Pitrou:
> Does Microsoft change their recommendations every couple of years? :)

Indeed they do. In fact, it's not really the recommendation that 
changes, but APIs that are added to new Windows releases. In the
specific case, Windows 8 adds an API called "Registered IO" (RIO).
They (of course) do these API addition in expecting some gain,
and then they (of course) promote these new APIs as actually achieving
the gain.

In the socket APIs, the Unix world went through a similar evolution,
with select, poll, epoll, kqueue, and whatnot.

The rate at which they change async APIs is actually low, compared
to the rate at which they change relational-database APIs (ODBC,
ADO, OLEDB, DAO, ADO.NET, LINQ, ... :-)

Regards,
Martin



From status at bugs.python.org  Fri Mar 15 18:07:26 2013
From: status at bugs.python.org (Python tracker)
Date: Fri, 15 Mar 2013 18:07:26 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20130315170726.142F8568E9@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2013-03-08 - 2013-03-15)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3888 ( -7)
  closed 25316 (+51)
  total  29204 (+44)

Open issues with patches: 1707 


Issues opened (36)
==================

#12466: sporadic failures of test_close_fds and test_pass_fds in test_
http://bugs.python.org/issue12466  reopened by ned.deily

#13918: locale.atof documentation is missing func argument
http://bugs.python.org/issue13918  reopened by ced

#17386: Bring Doc/make.bat as close to Doc/Makefile as possible
http://bugs.python.org/issue17386  opened by zach.ware

#17387: Error in C API documentation of PySequenceMethods
http://bugs.python.org/issue17387  opened by Alex.Orange

#17389: Optimize Event.wait()
http://bugs.python.org/issue17389  opened by pitrou

#17390: display python version on idle title bar
http://bugs.python.org/issue17390  opened by bagratte

#17391: _cursesmodule Fails to Build on GCC 2.95 (static)
http://bugs.python.org/issue17391  opened by Jeffrey.Armstrong

#17393: stdlib import mistaken for local by import_fixer
http://bugs.python.org/issue17393  opened by lregebro

#17394: Add slicing support to collections.deque
http://bugs.python.org/issue17394  opened by rhettinger

#17396: modulefinder fails if module contains syntax error
http://bugs.python.org/issue17396  opened by jgosmann

#17397: ttk::themes missing from ttk.py
http://bugs.python.org/issue17397  opened by klappnase

#17398: document url argument of RobotFileParser
http://bugs.python.org/issue17398  opened by tshepang

#17399: test_multiprocessing hang on Windows, non-sockets
http://bugs.python.org/issue17399  opened by terry.reedy

#17400: ipaddress.is_private needs to take into account of rfc6598
http://bugs.python.org/issue17400  opened by leim

#17401: io.FileIO closefd parameter is not documented nor shown in rep
http://bugs.python.org/issue17401  opened by rbcollins

#17403: Robotparser fails to parse some robots.txt
http://bugs.python.org/issue17403  opened by benmezger

#17404: ValueError: can't have unbuffered text I/O for io.open(1,	'wt'
http://bugs.python.org/issue17404  opened by rbcollins

#17405: Add _Py_memset_s() to securely clear memory
http://bugs.python.org/issue17405  opened by christian.heimes

#17408: second python execution fails when embedding
http://bugs.python.org/issue17408  opened by theDarkBrainer

#17409: resource.setrlimit doesn't respect -1
http://bugs.python.org/issue17409  opened by Paul.Price

#17410: Generator-based HTMLParser
http://bugs.python.org/issue17410  opened by flying sheep

#17411: Build failures with non-NDEBUG, non-Py_DEBUG builds.
http://bugs.python.org/issue17411  opened by twouters

#17413: format_exception() breaks on exception tuples from trace funct
http://bugs.python.org/issue17413  opened by inducer

#17415: Clarify docs of os.path.normpath()
http://bugs.python.org/issue17415  opened by gsingh

#17416: Documentation Ambiguity 2
http://bugs.python.org/issue17416  opened by gsingh

#17417: Documentation Modification Suggestion: os.walk, fwalk
http://bugs.python.org/issue17417  opened by gsingh

#17418: Documentation Bug
http://bugs.python.org/issue17418  opened by gsingh

#17419: bdist_wininst installer should allow install in user directory
http://bugs.python.org/issue17419  opened by Sergio.Callegari

#17420: bdist_wininst does not play well with unicode descriptions
http://bugs.python.org/issue17420  opened by Sergio.Callegari

#17421: Drop restriction that meta.__prepare__() must return a dict (s
http://bugs.python.org/issue17421  opened by eric.snow

#17422: language reference should specify restrictions on class namesp
http://bugs.python.org/issue17422  opened by eric.snow

#17423: libffi on 32bit is broken on linux
http://bugs.python.org/issue17423  opened by fijall

#17424: help() should use the class signature
http://bugs.python.org/issue17424  opened by jafo

#17425: Update OpenSSL versions in Windows builds
http://bugs.python.org/issue17425  opened by pitrou

#17428: replace readdir to readdir_r in function posix_listdir
http://bugs.python.org/issue17428  opened by Rock

#17429: platform.platform() can throw Unicode error
http://bugs.python.org/issue17429  opened by a.badger



Most recent 15 issues with no replies (15)
==========================================

#17429: platform.platform() can throw Unicode error
http://bugs.python.org/issue17429

#17428: replace readdir to readdir_r in function posix_listdir
http://bugs.python.org/issue17428

#17424: help() should use the class signature
http://bugs.python.org/issue17424

#17422: language reference should specify restrictions on class namesp
http://bugs.python.org/issue17422

#17418: Documentation Bug
http://bugs.python.org/issue17418

#17417: Documentation Modification Suggestion: os.walk, fwalk
http://bugs.python.org/issue17417

#17416: Documentation Ambiguity 2
http://bugs.python.org/issue17416

#17411: Build failures with non-NDEBUG, non-Py_DEBUG builds.
http://bugs.python.org/issue17411

#17403: Robotparser fails to parse some robots.txt
http://bugs.python.org/issue17403

#17401: io.FileIO closefd parameter is not documented nor shown in rep
http://bugs.python.org/issue17401

#17398: document url argument of RobotFileParser
http://bugs.python.org/issue17398

#17396: modulefinder fails if module contains syntax error
http://bugs.python.org/issue17396

#17394: Add slicing support to collections.deque
http://bugs.python.org/issue17394

#17391: _cursesmodule Fails to Build on GCC 2.95 (static)
http://bugs.python.org/issue17391

#17372: provide pretty printer for xml.etree.ElementTree
http://bugs.python.org/issue17372



Most recent 15 issues waiting for review (15)
=============================================

#17429: platform.platform() can throw Unicode error
http://bugs.python.org/issue17429

#17428: replace readdir to readdir_r in function posix_listdir
http://bugs.python.org/issue17428

#17423: libffi on 32bit is broken on linux
http://bugs.python.org/issue17423

#17421: Drop restriction that meta.__prepare__() must return a dict (s
http://bugs.python.org/issue17421

#17411: Build failures with non-NDEBUG, non-Py_DEBUG builds.
http://bugs.python.org/issue17411

#17410: Generator-based HTMLParser
http://bugs.python.org/issue17410

#17405: Add _Py_memset_s() to securely clear memory
http://bugs.python.org/issue17405

#17397: ttk::themes missing from ttk.py
http://bugs.python.org/issue17397

#17396: modulefinder fails if module contains syntax error
http://bugs.python.org/issue17396

#17391: _cursesmodule Fails to Build on GCC 2.95 (static)
http://bugs.python.org/issue17391

#17390: display python version on idle title bar
http://bugs.python.org/issue17390

#17389: Optimize Event.wait()
http://bugs.python.org/issue17389

#17386: Bring Doc/make.bat as close to Doc/Makefile as possible
http://bugs.python.org/issue17386

#17375: Add docstrings to methods in the threading module
http://bugs.python.org/issue17375

#17373: Add inspect.Signature.from_callable()
http://bugs.python.org/issue17373



Top 10 most discussed issues (10)
=================================

#17399: test_multiprocessing hang on Windows, non-sockets
http://bugs.python.org/issue17399  12 msgs

#17340: Handle malformed cookie
http://bugs.python.org/issue17340  11 msgs

#13564: ftplib and sendfile()
http://bugs.python.org/issue13564   9 msgs

#16895: Batch file to mimic 'make' on Windows
http://bugs.python.org/issue16895   8 msgs

#17410: Generator-based HTMLParser
http://bugs.python.org/issue17410   8 msgs

#17317: Benchmark driver should calculate actual benchmark count in -h
http://bugs.python.org/issue17317   7 msgs

#13918: locale.atof documentation is missing func argument
http://bugs.python.org/issue13918   6 msgs

#14468: Update cloning guidelines in devguide
http://bugs.python.org/issue14468   6 msgs

#15244: Support for opening files with FILE_SHARE_DELETE on Windows
http://bugs.python.org/issue15244   6 msgs

#16389: re._compiled_typed's lru_cache causes significant degradation 
http://bugs.python.org/issue16389   6 msgs



Issues closed (48)
==================

#3701: test_ntpath.test_relpath fails when launched from a different 
http://bugs.python.org/issue3701  closed by ezio.melotti

#4099: dir on a compiled re does not show pattern as a part of the li
http://bugs.python.org/issue4099  closed by ezio.melotti

#5017: import suds help( suds ) fails
http://bugs.python.org/issue5017  closed by ezio.melotti

#6933: Threading issue with Tkinter Frame.insert
http://bugs.python.org/issue6933  closed by terry.reedy

#8318: Deprecation of multifile inappropriate or incomplete
http://bugs.python.org/issue8318  closed by terry.reedy

#9686: asyncore infinite loop on raise
http://bugs.python.org/issue9686  closed by terry.reedy

#11029: Crash, 2.7.1, Tkinter and threads and line drawing
http://bugs.python.org/issue11029  closed by terry.reedy

#11367: xml.etree.ElementTree.find(all): docs are wrong
http://bugs.python.org/issue11367  closed by eli.bendersky

#11656: Debug builds for Windows would be very helpful
http://bugs.python.org/issue11656  closed by ezio.melotti

#11869: Include information about the bug tracker Rietveld code review
http://bugs.python.org/issue11869  closed by ezio.melotti

#11963: Remove human verification from test suite (test_parser and tes
http://bugs.python.org/issue11963  closed by ezio.melotti

#12921: http.server.BaseHTTPRequestHandler.send_error , ability send a
http://bugs.python.org/issue12921  closed by orsenthil

#14639: Different behavior for urllib2 in Python 2.7
http://bugs.python.org/issue14639  closed by ezio.melotti

#15121: devguide doesn't document all bug tracker components
http://bugs.python.org/issue15121  closed by ezio.melotti

#15158: Add support for multi-character delimiters in csv
http://bugs.python.org/issue15158  closed by terry.reedy

#15806: Add context manager for the "try: ... except: pass" pattern
http://bugs.python.org/issue15806  closed by rhettinger

#16004: Add `make touch` to 2.7 Makefile
http://bugs.python.org/issue16004  closed by ezio.melotti

#16471: upgrade to sphinx 1.1
http://bugs.python.org/issue16471  closed by terry.reedy

#16643: Wrong documented default value for timefunc parameter in sched
http://bugs.python.org/issue16643  closed by python-dev

#16659: Pure Python implementation of random
http://bugs.python.org/issue16659  closed by rhettinger

#17047: Fix double double words words
http://bugs.python.org/issue17047  closed by terry.reedy

#17066: Fix test discovery for test_robotparser.py
http://bugs.python.org/issue17066  closed by ezio.melotti

#17099: Raise ValueError when __loader__ not defined for importlib.fin
http://bugs.python.org/issue17099  closed by brett.cannon

#17117: Update importlib.util.module_for_loader/set_loader to set when
http://bugs.python.org/issue17117  closed by brett.cannon

#17138: XPath error in xml.etree.ElementTree
http://bugs.python.org/issue17138  closed by eli.bendersky

#17176: Document imp.NullImporter is NOT used anymore by import
http://bugs.python.org/issue17176  closed by brett.cannon

#17222: py_compile.compile() explicitly sets st_mode for written files
http://bugs.python.org/issue17222  closed by brett.cannon

#17299: Test cPickle with real files
http://bugs.python.org/issue17299  closed by serhiy.storchaka

#17307: HTTP PUT request Example
http://bugs.python.org/issue17307  closed by orsenthil

#17332: typo in json docs - "convered" should be "converted"
http://bugs.python.org/issue17332  closed by terry.reedy

#17348: Unicode - encoding seems to be lost for inputs of unicode char
http://bugs.python.org/issue17348  closed by terry.reedy

#17351: Remove explicit "object" inheritance in Python 3 docs
http://bugs.python.org/issue17351  closed by ezio.melotti

#17368: Python version of JSON decoder does not work with object_pairs
http://bugs.python.org/issue17368  closed by ezio.melotti

#17370: PEP should note related PEPs
http://bugs.python.org/issue17370  closed by barry

#17376: TimedRotatingFileHandler documentation regarding 'Week day' la
http://bugs.python.org/issue17376  closed by python-dev

#17382: debugging with idle: current line not highlighted
http://bugs.python.org/issue17382  closed by terry.reedy

#17384: test_logging failures on Windows
http://bugs.python.org/issue17384  closed by vinay.sajip

#17385: Use deque instead of list the threading.Condition waiter queue
http://bugs.python.org/issue17385  closed by rhettinger

#17388: Providing invalid value to random.sample can result in incorre
http://bugs.python.org/issue17388  closed by rhettinger

#17392: Python installer for Windows packages wrong zipfile.py
http://bugs.python.org/issue17392  closed by Simon.Wagner

#17395: Wait for live children in test_multiprocessing
http://bugs.python.org/issue17395  closed by ezio.melotti

#17402: In mmap doc examples map() is shadowed
http://bugs.python.org/issue17402  closed by ezio.melotti

#17406: Upload Windows 9x/NT4 build
http://bugs.python.org/issue17406  closed by ezio.melotti

#17407: RotatingFileHandler issue when using multiple loggers instance
http://bugs.python.org/issue17407  closed by vinay.sajip

#17412: Windows make.bat fails on 2.7
http://bugs.python.org/issue17412  closed by terry.reedy

#17414: timeit.timeit not in __all__ even though documented
http://bugs.python.org/issue17414  closed by terry.reedy

#17426: \0 in re.sub substitutes to space
http://bugs.python.org/issue17426  closed by gvanrossum

#17427: spam
http://bugs.python.org/issue17427  closed by ezio.melotti

From stefan at bytereef.org  Sat Mar 16 10:17:50 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Sat, 16 Mar 2013 10:17:50 +0100
Subject: [Python-Dev] [PEP 437] A DSL for specifying signatures,
 annotations and argument converters
Message-ID: <20130316091750.GA24061@sleipnir.bytereef.org>

This PEP hasn't been announced yet, so here's a short notice for all people
(like myself) who are not at PyCon. This is the counter proposal for the
argument parsing DSL from PEP 436:

   http://www.python.org/dev/peps/pep-0437/


Both PEPs were discussed at PyCon.  The state of affairs is that a
compromise is being worked upon and will be published by Larry in
a revised PEP.



Stefan Krah




From ezio.melotti at gmail.com  Sat Mar 16 22:09:08 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sat, 16 Mar 2013 23:09:08 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge default ->
	default): Merge heads default.
In-Reply-To: <3ZSvqs5jdLzSMT@mail.python.org>
References: <3ZSvqs5jdLzSMT@mail.python.org>
Message-ID: <CACBhJdHGQdCRLmYNUFTMBALjVUF5X_YHYNUc1CtcxYkbFKbKmg@mail.gmail.com>

Hi,

On Sat, Mar 16, 2013 at 10:08 PM, terry.reedy
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/9a2f4418e65a
> changeset:   82699:9a2f4418e65a
> parent:      82691:0a15a58ac4a1
> parent:      82695:533a60251b9d
> user:        Terry Jan Reedy <tjreedy at udel.edu>
> date:        Sat Mar 16 16:08:12 2013 -0400
> summary:
>   Merge heads default.
>
> files:
>   Doc/library/functions.rst |  4 ++--
>   1 files changed, 2 insertions(+), 2 deletions(-)
>

You forgot a couple of merges here.
If you look at the graph at
http://hg.python.org/cpython/graph/9a2f4418e65a you will see that all
the heads in the individual branches got merged, but then you forgot
to merge 3.2 -> 3.3 -> default (i.e. steps 5 and 6 of
http://bugs.python.org/issue14468#msg184130).  Serhiy fixed that
shortly after: http://hg.python.org/cpython/graph/5da005db8166
At least now that the worst case scenario that doesn't really happen
often?[0] happened I can point to some actual graphs that will
hopefully clarify why all these merges are necessary :)

Best Regards,
Ezio Melotti

[0]: http://bugs.python.org/issue14468#msg184140

From ezio.melotti at gmail.com  Sat Mar 16 23:02:35 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Sun, 17 Mar 2013 00:02:35 +0200
Subject: [Python-Dev] [Python-checkins] cpython (merge default ->
	default): Merge heads default.
In-Reply-To: <5144E893.2070405@udel.edu>
References: <3ZSvqs5jdLzSMT@mail.python.org>
	<CACBhJdHGQdCRLmYNUFTMBALjVUF5X_YHYNUc1CtcxYkbFKbKmg@mail.gmail.com>
	<5144E893.2070405@udel.edu>
Message-ID: <CACBhJdHY535HDGSW-16s1tQnHUWB-P1vG=sQ_rXwwuxdJhApnw@mail.gmail.com>

On Sat, Mar 16, 2013 at 11:48 PM, Terry Reedy <tjreedy at udel.edu> wrote:
> The FAQ says "...  using hg merge 3.3 as usual." Serhiy's commit message
> said 'Null merge', which to me is not 'as usual', as there are extra steps
> given in the FAQ above. So, do he really do a 'null merge' and is that the
> right thing to do in this situation?
>

It's probably just a matter of terminology.  I assume he did a "usual
merge" (i.e. "hg merge 3.2; hg ci -m '...';") and call it "null merge"
because there was no code that changed.  I prefer to use the term
"null merge" when I explicitly revert the code before committing, and
in this case I would have used "Merge with 3.x.".

FWIW I might add http://bugs.python.org/issue15917 at some point, to
prevent these situations.

Best Regards,
Ezio Melotti


> I have no doubt the the extra merges are needed ;-).

From glyph at twistedmatrix.com  Sun Mar 17 01:58:10 2013
From: glyph at twistedmatrix.com (Glyph)
Date: Sat, 16 Mar 2013 17:58:10 -0700
Subject: [Python-Dev] About issue 6560
In-Reply-To: <514253C1.9060209@v.loewis.de>
References: <CAOxq_8Mb=FMUA+ssUbimduKXr6g61dvNQGwS-q_jQ4NgBFUg5w@mail.gmail.com>
	<514253C1.9060209@v.loewis.de>
Message-ID: <D2105777-7531-489C-8662-5834759A9DAF@twistedmatrix.com>

On Mar 14, 2013, at 3:48 PM, Martin v. L?wis <martin at v.loewis.de> wrote:

> Am 14.03.13 15:15, schrieb Ani Sinha:
>> I was looking into a mechanism to get the aux fields from recvmsg() in
>> python and I came across this issue. Looks like this feature was added
>> in python 3.3. Is there any reason why this feature was not added for
>> python 2.7?
> 
> Most certainly: Python 2.7 (and thus Python 2) is feature-frozen; no
> new features can be added to it. People wanting new features need to
> port to Python 3.


Or you can use Twisted: <http://twistedmatrix.com/trac/browser/trunk/twisted/python/sendmsg.c>

That module ought to have no dependencies outside of Twisted.

We only use it for passing file descriptors between processes, but I believe it should be able to deal with whatever other types of auxiliary data that you need from recvmsg; if not, please file a bug (at <http://twistedmatrix.com/>).

-glyph


From larry at hastings.org  Sun Mar 17 08:14:04 2013
From: larry at hastings.org (Larry Hastings)
Date: Sun, 17 Mar 2013 00:14:04 -0700
Subject: [Python-Dev] [PEP 437] A DSL for specifying signatures,
 annotations and argument converters
In-Reply-To: <20130316091750.GA24061@sleipnir.bytereef.org>
References: <20130316091750.GA24061@sleipnir.bytereef.org>
Message-ID: <51456D3C.10608@hastings.org>

On 03/16/2013 02:17 AM, Stefan Krah wrote:
> Both PEPs were discussed at PyCon.  The state of affairs is that a
> compromise is being worked upon and will be published by Larry in
> a revised PEP.

I've pushed an update to PEP 436, the Argument Clinic PEP.  It's now 
live on python.org.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130317/453dba91/attachment.html>

From benjamin at python.org  Sun Mar 17 17:02:56 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 17 Mar 2013 09:02:56 -0700
Subject: [Python-Dev] 2.7.4 is inevitable
Message-ID: <CAPZV6o9FX3H3QHjBfPt7vk57ncOKaqeKAiE68E_9t-9Zyjg_kg@mail.gmail.com>

I am going to cut the 2.7.4 release branch next weekend (March 23,
24). Things which are breaking the buildbots at the point will be
backed out. Owners of current release blockers will be poked, but
nothing is going to hold up the release. The show must go on.

Benjamin

From christian at python.org  Sun Mar 17 17:37:37 2013
From: christian at python.org (Christian Heimes)
Date: Sun, 17 Mar 2013 17:37:37 +0100
Subject: [Python-Dev] Status of XML fixes
Message-ID: <5145F151.3060302@python.org>

Hello,

I like to give an update on the XML vulnerability fixes. Brett has asked
me a couple of days ago but I haven't had time to answer. I was/am busy
with my daily job.

Any attempt to fix the XML issues *will* change the behavior of the
library and result into an incompatibility with older releases. Benjamin
doesn't want to change the behavior of our XML libraries. IIRC Georg and
Barry are +0. I think that we should keep the current and unsafe
settings as default and add a simmple API to enable limitations and
protections.

What's available?
-----------------

https://bitbucket.org/tiran/defusedexpat contains everything we need to
fix the issues in the stdlib. All modifications to C code are available
for all relevant Python versions. They have been tested on Linux and
Windows, too.

* modified expat library with checks and workarounds for entity
expansion attacks. All fixes can be enabled or disabled by default at
compile time. The default settings can also be configured globally
(process wide, may be an issue for subinterpreters) and overwritten on
the expat parser instance.

* patched copies of pyexpat and _elementtree C extensions from Python
2.6, 2.7, 3.1, 3.2, 3.3 and 3.4 (a separate copy of each version). The
patches provide the functions and attributes to modifiy the global and
instance settings.

* defusedexpat.py contains the patches for sax and dom parsers to
disable external entity parsing.

* http://bugs.python.org/issue17239 contains an old patch for the issues
with a bunch of tests for each issue.


What needs to be done?
----------------------

* agree on default settings: secure by default or backwards compatible
by default?

* review of the changes to expat, pyexpat and _elementtree. Antoine,
Brett and Fred Drake have done some reviews.

* design and implement an API to enable the protective restrictions.

* documentation

* perhaps more tests

* finish the CVE reports


In the mean time ...
--------------------

https://pypi.python.org/pypi/defusedxml provides documentation, examples
and fixes for all Python versions w/o any C extension.


Christian


From eliben at gmail.com  Sun Mar 17 19:25:19 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Sun, 17 Mar 2013 11:25:19 -0700
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <5145F151.3060302@python.org>
References: <5145F151.3060302@python.org>
Message-ID: <CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>

I like to give an update on the XML vulnerability fixes. Brett has asked

> me a couple of days ago but I haven't had time to answer. I was/am busy
> with my daily job.
>
> Any attempt to fix the XML issues *will* change the behavior of the
> library and result into an incompatibility with older releases. Benjamin
> doesn't want to change the behavior of our XML libraries. IIRC Georg and
> Barry are +0. I think that we should keep the current and unsafe
> settings as default and add a simmple API to enable limitations and
> protections.
>
>
IMHO Benjamin is right, given that this attack has been known to exist
since 2003. Moreover, as it appears that no changes whatsoever are going to
make it into 2.7, I don't see why patching of 3.1, 3.2 and 3.3 is needed.
As for 3.4, it can't hurt to add an opt-in option for a safe mode to the
affected libraries.

* review of the changes to expat, pyexpat and _elementtree. Antoine,

> Brett and Fred Drake have done some reviews.
>
>
I'll gladly review the _elementtree changes and can help with the expat &
pyexpat changes as well. Until now I had the impression that the patches
aren't ready for review yet. If they are, that's great.

Do you have a patch in the issue tracker (so it can be reviewed with
Rietveld)? ISTM the current form is just a file (say _elementtree.c) in
your Bitbucket repo. Should that be just diffed with the trunk file to see
the changes?

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130317/2388fc54/attachment.html>

From stefan_ml at behnel.de  Sun Mar 17 20:00:19 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Sun, 17 Mar 2013 20:00:19 +0100
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
References: <5145F151.3060302@python.org>
	<CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
Message-ID: <ki53s0$jh4$1@ger.gmane.org>

Eli Bendersky, 17.03.2013 19:25:
> IMHO Benjamin is right, given that this attack has been known to exist
> since 2003. Moreover, as it appears that no changes whatsoever are going to
> make it into 2.7, I don't see why patching of 3.1, 3.2 and 3.3 is needed.
> As for 3.4, it can't hurt to add an opt-in option for a safe mode to the
> affected libraries.

Why keep the libraries vulnerable for another year (3.4 final is expected
for early 2014), if there is something we can do about them now? The fact
that the attacks have been known for a decade doesn't mean an attacker will
need another ten years to exploit them.

Stefan



From solipsis at pitrou.net  Sun Mar 17 19:59:52 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 17 Mar 2013 19:59:52 +0100
Subject: [Python-Dev] Status of XML fixes
References: <5145F151.3060302@python.org>
	<CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
	<ki53s0$jh4$1@ger.gmane.org>
Message-ID: <20130317195952.2bfa0dc8@pitrou.net>

On Sun, 17 Mar 2013 20:00:19 +0100
Stefan Behnel <stefan_ml at behnel.de> wrote:
> Eli Bendersky, 17.03.2013 19:25:
> > IMHO Benjamin is right, given that this attack has been known to exist
> > since 2003. Moreover, as it appears that no changes whatsoever are going to
> > make it into 2.7, I don't see why patching of 3.1, 3.2 and 3.3 is needed.
> > As for 3.4, it can't hurt to add an opt-in option for a safe mode to the
> > affected libraries.
> 
> Why keep the libraries vulnerable for another year (3.4 final is expected
> for early 2014), if there is something we can do about them now?

Well, Christian said that his stdlib patch wasn't ready yet.

Regards

Antoine.



From eliben at gmail.com  Sun Mar 17 21:03:21 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Sun, 17 Mar 2013 13:03:21 -0700
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <ki53s0$jh4$1@ger.gmane.org>
References: <5145F151.3060302@python.org>
	<CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
	<ki53s0$jh4$1@ger.gmane.org>
Message-ID: <CAF-Rda_OR=1nG3LTN7OorL3HwG5M1+-Z3iKLw1DC62Fvh9YKug@mail.gmail.com>

On Sun, Mar 17, 2013 at 12:00 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:

> Eli Bendersky, 17.03.2013 19:25:
> > IMHO Benjamin is right, given that this attack has been known to exist
> > since 2003. Moreover, as it appears that no changes whatsoever are going
> to
> > make it into 2.7, I don't see why patching of 3.1, 3.2 and 3.3 is needed.
> > As for 3.4, it can't hurt to add an opt-in option for a safe mode to the
> > affected libraries.
>
> Why keep the libraries vulnerable for another year (3.4 final is expected
> for early 2014), if there is something we can do about them now? The fact
> that the attacks have been known for a decade doesn't mean an attacker will
> need another ten years to exploit them.
>

 I'm using a conditional argument here. *If* we don't deem the changes
important enough to go into 2.7, *then* they aren't important enough to go
into 3.1 and 3.2; 3.3 is a question. That's because 2.7 is arguably more
important in this respect, having no direct upgrade path, whereas for 3.x
users the fix will be available with 3.4 anyway.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130317/b8dc63e6/attachment.html>

From stefan at bytereef.org  Sun Mar 17 23:26:07 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Sun, 17 Mar 2013 23:26:07 +0100
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <3ZTBVD26DlzR9T@mail.python.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
Message-ID: <20130317222607.GA16540@sleipnir.bytereef.org>

[PEP 436 revised syntax]

While I like the syntax better and appreciate the option to condense the
function declaration I still fear that the amount of implicitness will
distract from what is important: programming in C.

This applies especially if people start declaring converters using the
[python] feature.

So I hope that at least converters can be declared statically in a header
file, like I suggested in PEP 437.


A couple of comments:


> As of CPython 3.3, builtin functions nearly always parse their arguments
> with one of two functions: the original ``PyArg_ParseTuple()``, [1]_ and
> the more modern ``PyArg_ParseTupleAndKeywords()``. [2]_ The former
> only handles positional parameters; the latter also accommodates keyword
> and keyword-only parameters, and is preferred for new code.

What is the source for this? I seem to remember a discussion on python-ideas
(but cannot find it now) where some developers preferred non-keyword functions
for some use cases.

For example it's strange to write div(x=10, y=3), or worse, div(y=3, x=10).
Using positional-only arguments prevents this "feature".


>  /*[clinic]
>  os.stat as os_stat_fn -> stat result
>
>    path: path_t(allow_fd=1)
>        Path to be examined; can be string, bytes, or open-file-descriptor int.

I do not see where the C initialization or the cleanup are specified. Are
they part of the converter specification?


> /*[clinic]
> curses.window.addch
>
>    [
>    x: int
>      X-coordinate.
>
>    y: int
>      Y-coordinate.
>    ]

The parameters appear to be in the wrong order.


> The return annotation is also optional.  If skipped, the arrow ("``->``")
> must also be omitted.

Why is it optional? Aren't type annotations important?


> Clinic will ship with a number of built-in converters; new converters can
> also be added dynamically.

How are the converters specified? Inside the preprocessor source? Are initialization
and cleanup part of the specification, e.g. is a converter represented by a class?

I would prefer if the converters were in a header file, like I suggested in
PEP 437. Any tool can read such a file and custom converters can be redeclared
above their definition.


> The default value is dynamically assigned, "live" in the generated C code,
> and although it's specified as a Python value, it's translated into a native
> C value in the generated C code. Few default values are permitted, owing to
> this manual translation step.

I think there should be a table that lists which values are converted and what
the result of the conversion is.


> ``[``
>    Establishes the start of an optional "group" of parameters.
>    Note that "groups" may nest inside other "groups".
>    See `Functions With Positional-Only Parameters`_ below.

I don't quite understand the terminology: Functions with the ``/`` are also
"positional-only".  Why not reserve this syntax exclusively for the legacy
left-and-right optional case?


> ``/``
>    This hints to Argument Clinic that this function is performance-sensitive,
>    and that it's acceptable to forego supporting keyword parameters when parsing.
>    (In early implementations of Clinic, this will switch Clinic from generating
>    code using ``PyArg_ParseTupleAndKeywords`` to using ``PyArg_ParseTuple``.
>    The hope is that in the future there will be no appreciable speed difference,
>    rendering this syntax irrelevant and deprecated but harmless.)

Here I would use "positional-only" and mention that the slash plays essentially
the same role as the vertical bar in the existing syntax. If this isn't the
intention, then I simply did not understand the paragraph.


> types
>
>    A list of strings representing acceptable Python types for this object.
>    There are also four strings which represent Python protocols:

I don't quite follow: Aren't input types always specified by the converter
function?


> Argument Clinic also permits embedding Python code inside C files, which
> is executed in-place when Argument Clinic processes the file. Embedded code
> looks like this:

The example in posixmodule.c takes up a lot of space and from the perspective
of auditing the effects it's a little like following a longjmp.



Stefan Krah




From barry at python.org  Mon Mar 18 04:48:23 2013
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Mar 2013 20:48:23 -0700
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <5145F151.3060302@python.org>
References: <5145F151.3060302@python.org>
Message-ID: <20130317204823.68c07a0e@anarchist>

On Mar 17, 2013, at 05:37 PM, Christian Heimes wrote:

>Any attempt to fix the XML issues *will* change the behavior of the
>library and result into an incompatibility with older releases. Benjamin
>doesn't want to change the behavior of our XML libraries. IIRC Georg and
>Barry are +0. I think that we should keep the current and unsafe
>settings as default and add a simmple API to enable limitations and
>protections.

I strongly believe that the decision must be the same for all stable versions.
We can't impose the madness of version checks on people for them to know what
to do.

-Barry

From v+python at g.nevcal.com  Mon Mar 18 05:16:57 2013
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Sun, 17 Mar 2013 21:16:57 -0700
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <20130317204823.68c07a0e@anarchist>
References: <5145F151.3060302@python.org> <20130317204823.68c07a0e@anarchist>
Message-ID: <51469539.5030202@g.nevcal.com>

On 3/17/2013 8:48 PM, Barry Warsaw wrote:
> On Mar 17, 2013, at 05:37 PM, Christian Heimes wrote:
>
>> Any attempt to fix the XML issues *will* change the behavior of the
>> library and result into an incompatibility with older releases. Benjamin
>> doesn't want to change the behavior of our XML libraries. IIRC Georg and
>> Barry are +0. I think that we should keep the current and unsafe
>> settings as default and add a simmple API to enable limitations and
>> protections.
> I strongly believe that the decision must be the same for all stable versions.
> We can't impose the madness of version checks on people for them to know what
> to do.

try:
    newSimpleXMLAPI()
    newapi = True
except Exception:
    newapi = False
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130317/a99c1a74/attachment.html>

From andrew.svetlov at gmail.com  Mon Mar 18 06:00:55 2013
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Sun, 17 Mar 2013 22:00:55 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <20130317222607.GA16540@sleipnir.bytereef.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
Message-ID: <CAL3CFcWytCih-VX98Vkrp9eviv4uMJUN3Pqwp2=K-FfVjE8JoQ@mail.gmail.com>

On Sun, Mar 17, 2013 at 3:26 PM, Stefan Krah <stefan at bytereef.org> wrote:
> [PEP 436 revised syntax]
>
> While I like the syntax better and appreciate the option to condense the
> function declaration I still fear that the amount of implicitness will
> distract from what is important: programming in C.
>
> This applies especially if people start declaring converters using the
> [python] feature.
>
> So I hope that at least converters can be declared statically in a header
> file, like I suggested in PEP 437.
>
>
> A couple of comments:
>
>


>> As of CPython 3.3, builtin functions nearly always parse their arguments
>> with one of two functions: the original ``PyArg_ParseTuple()``, [1]_ and
>> the more modern ``PyArg_ParseTupleAndKeywords()``. [2]_ The former
>> only handles positional parameters; the latter also accommodates keyword
>> and keyword-only parameters, and is preferred for new code.
>
> What is the source for this? I seem to remember a discussion on python-ideas
> (but cannot find it now) where some developers preferred non-keyword functions
> for some use cases.
>
> For example it's strange to write div(x=10, y=3), or worse, div(y=3, x=10).
> Using positional-only arguments prevents this "feature".
IIRC objection was about functions like abs(5).
If function has single and obvious argument why you need to name that parameter?
The issue has related to documentation for existing one-argument functions only.



>
>
>>  /*[clinic]
>>  os.stat as os_stat_fn -> stat result
>>
>>    path: path_t(allow_fd=1)
>>        Path to be examined; can be string, bytes, or open-file-descriptor int.
>
> I do not see where the C initialization or the cleanup are specified. Are
> they part of the converter specification?
>
>
>> /*[clinic]
>> curses.window.addch
>>
>>    [
>>    x: int
>>      X-coordinate.
>>
>>    y: int
>>      Y-coordinate.
>>    ]
>
> The parameters appear to be in the wrong order.
>
>
>> The return annotation is also optional.  If skipped, the arrow ("``->``")
>> must also be omitted.
>
> Why is it optional? Aren't type annotations important?
>
>
>> Clinic will ship with a number of built-in converters; new converters can
>> also be added dynamically.
>
> How are the converters specified? Inside the preprocessor source? Are initialization
> and cleanup part of the specification, e.g. is a converter represented by a class?
>
> I would prefer if the converters were in a header file, like I suggested in
> PEP 437. Any tool can read such a file and custom converters can be redeclared
> above their definition.
>
>
>> The default value is dynamically assigned, "live" in the generated C code,
>> and although it's specified as a Python value, it's translated into a native
>> C value in the generated C code. Few default values are permitted, owing to
>> this manual translation step.
>
> I think there should be a table that lists which values are converted and what
> the result of the conversion is.
>
>
>> ``[``
>>    Establishes the start of an optional "group" of parameters.
>>    Note that "groups" may nest inside other "groups".
>>    See `Functions With Positional-Only Parameters`_ below.
>
> I don't quite understand the terminology: Functions with the ``/`` are also
> "positional-only".  Why not reserve this syntax exclusively for the legacy
> left-and-right optional case?
>
>
>> ``/``
>>    This hints to Argument Clinic that this function is performance-sensitive,
>>    and that it's acceptable to forego supporting keyword parameters when parsing.
>>    (In early implementations of Clinic, this will switch Clinic from generating
>>    code using ``PyArg_ParseTupleAndKeywords`` to using ``PyArg_ParseTuple``.
>>    The hope is that in the future there will be no appreciable speed difference,
>>    rendering this syntax irrelevant and deprecated but harmless.)
>
> Here I would use "positional-only" and mention that the slash plays essentially
> the same role as the vertical bar in the existing syntax. If this isn't the
> intention, then I simply did not understand the paragraph.
>
>
>> types
>>
>>    A list of strings representing acceptable Python types for this object.
>>    There are also four strings which represent Python protocols:
>
> I don't quite follow: Aren't input types always specified by the converter
> function?
>
>
>> Argument Clinic also permits embedding Python code inside C files, which
>> is executed in-place when Argument Clinic processes the file. Embedded code
>> looks like this:
>
> The example in posixmodule.c takes up a lot of space and from the perspective
> of auditing the effects it's a little like following a longjmp.
>
>
>
> Stefan Krah
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov

From barry at python.org  Mon Mar 18 05:52:16 2013
From: barry at python.org (Barry Warsaw)
Date: Sun, 17 Mar 2013 21:52:16 -0700
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <51469539.5030202@g.nevcal.com>
References: <5145F151.3060302@python.org> <20130317204823.68c07a0e@anarchist>
	<51469539.5030202@g.nevcal.com>
Message-ID: <20130317215216.161280d3@anarchist>

On Mar 17, 2013, at 09:16 PM, Glenn Linderman wrote:

>try:
>    newSimpleXMLAPI()
>    newapi = True
>except Exception:
>    newapi = False

try:
    True
except NameError:
    True = 1
    False = 0

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130317/2d9049f6/attachment-0001.pgp>

From benjamin at python.org  Mon Mar 18 06:53:19 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 17 Mar 2013 22:53:19 -0700
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <20130317215216.161280d3@anarchist>
References: <5145F151.3060302@python.org> <20130317204823.68c07a0e@anarchist>
	<51469539.5030202@g.nevcal.com> <20130317215216.161280d3@anarchist>
Message-ID: <CAPZV6o__FFD=0srwi=_KGH7HPfzXdDBwJZB2VE55Amt3tuCV8Q@mail.gmail.com>

2013/3/17 Barry Warsaw <barry at python.org>:
> On Mar 17, 2013, at 09:16 PM, Glenn Linderman wrote:
>
>>try:
>>    newSimpleXMLAPI()
>>    newapi = True
>>except Exception:
>>    newapi = False
>
> try:
>     True
> except NameError:
>     True = 1
>     False = 0
>
> -Barry

I understand why your bedtime is 21:30. :)


-- 
Regards,
Benjamin

From larry at hastings.org  Mon Mar 18 08:16:56 2013
From: larry at hastings.org (Larry Hastings)
Date: Mon, 18 Mar 2013 00:16:56 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <20130317222607.GA16540@sleipnir.bytereef.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
Message-ID: <5146BF68.8030903@hastings.org>



On 03/17/2013 03:26 PM, Stefan Krah wrote:
> While I like the syntax better and appreciate the option to condense the
> function declaration I still fear that the amount of implicitness will
> distract from what is important: programming in C.
>
> This applies especially if people start declaring converters using the
> [python] feature.
>
> So I hope that at least converters can be declared statically in a header
> file, like I suggested in PEP 437.

The Argument Clinic prototype is written in Python; I don't know how 
"declared static in a header file" applies to a Python implementation.  
Currently the converters are declared directly in clinic.py, somewhere 
in the middle.

For what it's worth, I think the new syntax (dictated more-or-less by 
Guido and Nick) makes things slightly less explicit.  In my original 
syntax, you declared the *type* of each parameter in C, and the flags 
dictated how Clinic should perform the conversion.  In the new syntax, 
you declare the *converter function* for each parameter, and the type of 
the resulting C variable is inferred.


> A couple of comments:
>> As of CPython 3.3, builtin functions nearly always parse their arguments
>> with one of two functions: the original ``PyArg_ParseTuple()``, [1]_ and
>> the more modern ``PyArg_ParseTupleAndKeywords()``. [2]_ The former
>> only handles positional parameters; the latter also accommodates keyword
>> and keyword-only parameters, and is preferred for new code.
> What is the source for this? I seem to remember a discussion on python-ideas
> (but cannot find it now) where some developers preferred non-keyword functions
> for some use cases.
>
> For example it's strange to write div(x=10, y=3), or worse, div(y=3, x=10).
> Using positional-only arguments prevents this "feature".

(I don't know to what "div" function you refer, but its arguments are 
poorly named.)

I thought this was obviously true.  But realized I didn't have any 
justification for it.  So I checked into it.  Thomas Wouters found me a 
thread from python-ideas from last March about changing range() to 
support keyword-only parameters.  Guido participated in the thread, and 
responded with this pronouncement:

    [...] let's forget about "fixing" range. And that's final.

    http://mail.python.org/pipermail/python-ideas/2012-March/014380.html

I double-checked with him about this and he confirmed: positional 
parameters are here to stay.

This has some consequences.  For example, inspect.getfullargspec, 
inspect.Signature, and indeed types.FunctionObject and types.CodeObject 
have no currently defined mechanism for communicating that a parameter 
is positional-only.  I strongly assert we need such a mechanism, though 
it could be as simple as having the parameter name be an empty string or 
None.

Anyway, it turns out this "keyword-only is preferred" was a 
misconception on my part.  Python supports, and will continue to 
support, positional-only parameters as part of the language. Currently 
it isn't possible to define functions in Python that have them.  But 
builtins have them, and will continue to have them, and therefore 
Argument Clinic needs to support them.

I'll amend my PEP soonish to reflect this.  Specifically the semantics 
of the /, [, and ] lines in the parameter section.


>>     path: path_t(allow_fd=1)
>>
>> I do not see where the C initialization or the cleanup are specified. Are
>> they part of the converter specification?

The extension interface isn't yet well-enough defined to be in the PEP.  
But yes, the generation of cleanup code will be part of the job of the 
conversion functions.  I'm not sure I have any provision for 
initialization apart from assignment, but I agree that it should have one.


>> curses.window.addch
> The parameters appear to be in the wrong order.

Fixed.


>> The return annotation is also optional.  If skipped, the arrow ("``->``")
>> must also be omitted.
> Why is it optional? Aren't type annotations important?

I'm not aware of any builtins that have annotations.  In fact, I'm not 
sure there's any place in the API where you can attach annotations to a 
builtin.  I expect this will change when we add reflection support for 
signatures, though I doubt many builtins will use the facility.

This raises an interesting point, though: Argument Clinic provides a 
place to provide a function return annotation, but has never supported 
per-parameter annotations.  I'll meditate on this and see if I can come 
up with a reasonable amendment to the current syntax.


> How are the converters specified? Inside the preprocessor source? Are initialization
> and cleanup part of the specification, e.g. is a converter represented by a class?

Respectively: in Python, yes, and yes.  The current prototype has an 
experimental extension interface; this is used to add support for 
"path_t" in Modules/posixmodule.c.  It supports initialization and cleanup.


> I think there should be a table that lists which values are converted and what
> the result of the conversion is.

The documentation is TBD.  In the current prototype, it explicitly turns 
"None" into "Py_None", and otherwise flushes whatever you specified as 
the default value through from the DSL to the generated C code.


>> types
>>
>>     A list of strings representing acceptable Python types for this object.
>>     There are also four strings which represent Python protocols:
> I don't quite follow: Aren't input types always specified by the converter
> function?

I'm not sure yet.  The purpose of parameterizing the converter functions 
is to cut down on having eleventy-billion (in other words, "forty") 
different converter functions, some with only slight differences.  For 
example, 'C' accepts a str of length 1, whereas 'c' accepts a bytes or 
bytearray of length 1.  Should this be two converter functions, or one 
converter taking a list of types?  I'll see what feels better.


> The example in posixmodule.c takes up a lot of space and from the perspective
> of auditing the effects it's a little like following a longjmp.

I got strong feedback that I needed more examples.  That was the logical 
place for them.  Can you suggest a better spot, or spots?


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/9eaaaccf/attachment.html>

From larry at hastings.org  Mon Mar 18 09:48:37 2013
From: larry at hastings.org (Larry Hastings)
Date: Mon, 18 Mar 2013 01:48:37 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <5146BF68.8030903@hastings.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org>
Message-ID: <5146D4E5.1020302@hastings.org>



On 03/18/2013 12:16 AM, Larry Hastings wrote:
> I'll amend my PEP soonish to reflect this.  Specifically the semantics 
> of the /, [, and ] lines in the parameter section.

I've just posted this revision.

I'd like to draw everyone's attention to the top entry in the Notes 
section, reproduced below:

* The DSL currently makes no provision for specifying per-parameter
   type annotations.  This is something explicitly supported in Python;
   it should be supported for builtins too, once we have reflection support.

   It seems to me that the syntax for parameter lines--dictated by
   Guido--suggests conversion functions are themselves type annotations.
   This makes intuitive sense.  But my thought experiments in how to
   convert the conversion function specification into a per-parameter
   type annotation ranged from obnoxious to toxic; I don't think that
   line of thinking will bear fruit.

   Instead, I think we need to add a new syntax allowing functions
   to explicitly specify a per-parameter type annotation.  The problem:
   what should that syntax be?  I've only had one idea so far, and I
   don't find it all that appealing: allow a optional second colon
   on the parameter line, and the type annotation would be specified...
   somewhere, either between the first and second colons, or between
   the second colon and the (optional) default.

   Also, I don't think this could specify any arbitrary Python value.
   I suspect it would suffer heavy restrictions on what types and
   literals it could use.  Perhaps the best solution would be to
   store the exact string in static data, and have Python evaluate
   it on demand?  If so, it would be safest to restrict it to Python
   literal syntax, permitting no function calls (even to builtins).


Syntax suggestions welcome,


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/3e370770/attachment.html>

From ronaldoussoren at mac.com  Mon Mar 18 10:29:49 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 18 Mar 2013 10:29:49 +0100
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
	slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <5146BF68.8030903@hastings.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org>
Message-ID: <80206D7B-ACAB-4108-BC13-D6AC2BB2E7CB@mac.com>


On 18 Mar, 2013, at 8:16, Larry Hastings <larry at hastings.org> wrote:

> 
> This has some consequences.  For example, inspect.getfullargspec, inspect.Signature, and indeed types.FunctionObject and types.CodeObject have no currently defined mechanism for communicating that a parameter is positional-only.  I strongly assert we need such a mechanism, though it could be as simple as having the parameter name be an empty string or None.

inspect.Signature does have support for positional-only arguments, they have inspect.Parameter.POSITIONAL_ONLY as their kind.  The others probably don't have support for this kind of parameters because there is no Python syntax for creating them.

Ronald

From stefan at bytereef.org  Mon Mar 18 11:13:11 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 18 Mar 2013 11:13:11 +0100
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <5146BF68.8030903@hastings.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org>
Message-ID: <20130318101311.GA21364@sleipnir.bytereef.org>

Larry Hastings <larry at hastings.org> wrote:
>     So I hope that at least converters can be declared statically in a header
>     file, like I suggested in PEP 437.
> 
> 
> The Argument Clinic prototype is written in Python; I don't know how "declared
> static in a header file" applies to a Python implementation.  Currently the
> converters are declared directly in clinic.py, somewhere in the middle.

It applies in the same way to a Python implementation as declaring the
DSL comment blocks in a C file applies to a Python implementation. This
is exactly the same.

1) I think that third party tools should be able to extract *all* required
   information from the DSL only.

2) After writing a new custom converter, I'd rather edit a header file and
   not the preprocessor source.

3) Likewise, I'd rather edit a header file than inserting a magic [python]
   block into the C source that registers the required information with
   the preprocessor in a completely implementation defined way.


> 
>     The example in posixmodule.c takes up a lot of space and from the perspective
>     of auditing the effects it's a little like following a longjmp.
> 
> 
> I got strong feedback that I needed more examples.  That was the logical place
> for them.  Can you suggest a better spot, or spots?

I'm concerned about the whole concept (see above).


Stefan Krah



From stefan at bytereef.org  Mon Mar 18 11:36:43 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 18 Mar 2013 11:36:43 +0100
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <5146D4E5.1020302@hastings.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org> <5146D4E5.1020302@hastings.org>
Message-ID: <20130318103643.GA21550@sleipnir.bytereef.org>

Larry Hastings <larry at hastings.org> wrote:
> * The DSL currently makes no provision for specifying per-parameter
>   type annotations.  This is something explicitly supported in Python;
>   it should be supported for builtins too, once we have reflection support.
> 
>   It seems to me that the syntax for parameter lines--dictated by
>   Guido--suggests conversion functions are themselves type annotations.
>   This makes intuitive sense.

Really, did you read PEP 437? It's all in there.


>  But my thought experiments in how to convert the conversion function
>  specification into a per-parameter type annotation ranged from obnoxious
>  to toxic; I don't think that
>  line of thinking will bear fruit.

Did you look at the patch that I posted in issue #16612? It's already
implemented:

$ ./printsemant Tools/preprocess/testcases/posix_stat.c
PROGRAM[
  SOURCE[...],
  DEFINE[
    CNAME posix_stat,
    SPEC[
      DECLARATION
         { fun_fqname = os.stat,
           fun_name = stat,
           fun_cname = posix_stat,
           fun_kind = Keywords,
           fun_params = [
             { param_name = path,
               param_type = [bytes, int, str],     <== here it is
               param_default = NONE,
               param_kind = (PosOrKwd, Required),
               param_conv = path_converter,
               param_parseargs = [
                 ConvArg { arg_name = path_converter,
                           arg_type = int (*converter)(PyObject *, void *)
                           arg_use_ptr = false },
                 MainArg { arg_name = path,
                           arg_type = path_t,
                           arg_use_ptr = true }]},
[...]



Stefan Krah



From stefan at bytereef.org  Mon Mar 18 12:05:52 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Mon, 18 Mar 2013 12:05:52 +0100
Subject: [Python-Dev] [Python-checkins] peps: Update for 436,
 explicitly supporting positional parameters forever, amen.
In-Reply-To: <3ZTrcp6hZCzSLM@mail.python.org>
References: <3ZTrcp6hZCzSLM@mail.python.org>
Message-ID: <20130318110552.GA21900@sleipnir.bytereef.org>

larry.hastings <python-checkins at python.org> wrote:
> +    Establishes that all the *proceeding* arguments are
> +    positional-only.  For now, Argument Clinic does not
> +    support functions with both positional-only and
> +    non-positional-only arguments; therefore, if ``/``
> +    is specified for a function, currently it must always
> +    be after the last parameter.  Also, Argument Clinic
> +    does not currently support default values for
> +    positional-only parameters.
> +
> +(The semantics of ``/`` follow a syntax for positional-only
> +parameters in Python once proposed by Guido. [5]_ )

I think the entire PEP would be easier to understand if the main sections
only contained the envisaged end result and all current preprocessor
deficiencies were listed in a single isolated section.


Stefan Krah




From ndbecker2 at gmail.com  Mon Mar 18 14:50:17 2013
From: ndbecker2 at gmail.com (Neal Becker)
Date: Mon, 18 Mar 2013 09:50:17 -0400
Subject: [Python-Dev] can't assign to function call
Message-ID: <ki762l$gfs$1@ger.gmane.org>

def F(x):
    return x

x = 2
F(x) = 3

    F(x) = 3
SyntaxError: can't assign to function call

Do we really need this restriction?  There do exist other languages without it.


From steve at pearwood.info  Mon Mar 18 15:03:47 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 19 Mar 2013 01:03:47 +1100
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <ki762l$gfs$1@ger.gmane.org>
References: <ki762l$gfs$1@ger.gmane.org>
Message-ID: <51471EC3.4020101@pearwood.info>

On 19/03/13 00:50, Neal Becker wrote:
> def F(x):
>      return x
>
> x = 2
> F(x) = 3
>
>      F(x) = 3
> SyntaxError: can't assign to function call
>
> Do we really need this restriction?  There do exist other languages without it.


What meaning would you give to "F(x) = 3", and why?



-- 
Steven

From jsbueno at python.org.br  Mon Mar 18 15:04:51 2013
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Mon, 18 Mar 2013 11:04:51 -0300
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <ki762l$gfs$1@ger.gmane.org>
References: <ki762l$gfs$1@ger.gmane.org>
Message-ID: <CAH0mxTTX4M46TRxN+wr5Nid=CiH4S8vDGRNgc1CrTyqO27cW=A@mail.gmail.com>

On 18 March 2013 10:50, Neal Becker <ndbecker2 at gmail.com> wrote:
> def F(x):
>     return x
>
> x = 2
> F(x) = 3
>
>     F(x) = 3
> SyntaxError: can't assign to function call
>
> Do we really need this restriction?  There do exist other languages without it.

What?
I mean...what are you even talking about?
Assignments are to "names" - names are not Python objects and it is
not something that can be returned from a function call.

If you are meaning mathematical equation like functionality, I
recommend you to try "SymPy" - the Library for symbolic mathematics.

I can't make sense of what you want to perform by "assigning to a
function call", and given the time without a reply to this e-mail, I
think I am not the only one there.


  js
 -><-





>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jsbueno%40python.org.br

From rosuav at gmail.com  Mon Mar 18 15:23:07 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Tue, 19 Mar 2013 01:23:07 +1100
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <ki762l$gfs$1@ger.gmane.org>
References: <ki762l$gfs$1@ger.gmane.org>
Message-ID: <CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>

On Tue, Mar 19, 2013 at 12:50 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> def F(x):
>     return x
>
> x = 2
> F(x) = 3
>
>     F(x) = 3
> SyntaxError: can't assign to function call
>
> Do we really need this restriction?  There do exist other languages without it.

The languages that permit you to assign to a function call all have
some notion of a reference type. In C++, for instance, you can return
a reference to an object, and assigning to the function call achieves
the same thing as assigning to the referent. C has similar semantics
with pointers; you can dereference a returned pointer:

int somevalue;
int *F() {return &somevalue;}

*F() = 5; /* will assign to somevalue */

With Python, there are no pointers, there are no variables. But you
can do something somewhat similar:

>>> x = [0]
>>> def F():
	return x

>>> F()[0]=3;
>>> x[0]
3

If you think of x as a pointer to the value x[0], then Python lets you
"dereference" the function return value. But this is fiddling with
terminology; the concept of assigning to a function return value
doesn't really make sense in Python.

Further discussion on exactly _why_ this is the case can be found on
python-list's or python-tutor's archives, such as this excellent post
by Steven D'Aprano:

http://mail.python.org/pipermail/tutor/2010-December/080505.html

TLDR: Python != C. :)

ChrisA

From python-dev at masklinn.net  Mon Mar 18 15:44:44 2013
From: python-dev at masklinn.net (Xavier Morel)
Date: Mon, 18 Mar 2013 15:44:44 +0100
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
Message-ID: <9D32D1B2-4C08-4704-AC8E-F44FFC3203D1@masklinn.net>

On 2013-03-18, at 15:23 , Chris Angelico wrote:

> On Tue, Mar 19, 2013 at 12:50 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
>> def F(x):
>>    return x
>> 
>> x = 2
>> F(x) = 3
>> 
>>    F(x) = 3
>> SyntaxError: can't assign to function call
>> 
>> Do we really need this restriction?  There do exist other languages without it.
> 
> The languages that permit you to assign to a function call all have
> some notion of a reference type.

Alternatively they're functional language defining "match cases" e.g. in
Haskell a function is defined as

    foo a b c = someOperation a b c

which is functionally equivalent to Python's

    def foo(a, b, c):
        return someOperation(a, b, c)

From christian at python.org  Mon Mar 18 15:54:34 2013
From: christian at python.org (Christian Heimes)
Date: Mon, 18 Mar 2013 15:54:34 +0100
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
References: <5145F151.3060302@python.org>
	<CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
Message-ID: <ki79r7$sc4$1@ger.gmane.org>

Am 17.03.2013 19:25, schrieb Eli Bendersky:
> I'll gladly review the _elementtree changes and can help with the expat
> & pyexpat changes as well. Until now I had the impression that the
> patches aren't ready for review yet. If they are, that's great.

The modifications to expat, pyexpat and _elementtree are available for
weeks. I just hadn't have time to create proper patches yet.

> Do you have a patch in the issue tracker (so it can be reviewed with
> Rietveld)? ISTM the current form is just a file (say _elementtree.c) in
> your Bitbucket repo. Should that be just diffed with the trunk file to
> see the changes?

I have pushed all changes from defusedexpat to a clone of Python's hg
repository. You can find the clone at https://bitbucket.org/tiran/xmlbomb/ .

The repository also contains a quick draft for a XML security API.
https://bitbucket.org/tiran/xmlbomb/commits/c033abd0f7747c5b215e1b32f90372dd96e397ba

I have to port tests from my other branch and add tests for the new API,
too.

Christian


From christian at python.org  Mon Mar 18 16:00:21 2013
From: christian at python.org (Christian Heimes)
Date: Mon, 18 Mar 2013 16:00:21 +0100
Subject: [Python-Dev] Status of XML fixes
In-Reply-To: <20130317195952.2bfa0dc8@pitrou.net>
References: <5145F151.3060302@python.org>
	<CAF-Rda9bOjyCQy5Tu94tfW2-dT2EM4-cybtJ44kEQd-8uxvZrQ@mail.gmail.com>
	<ki53s0$jh4$1@ger.gmane.org> <20130317195952.2bfa0dc8@pitrou.net>
Message-ID: <51472C05.1050806@python.org>

Am 17.03.2013 19:59, schrieb Antoine Pitrou:
>> Why keep the libraries vulnerable for another year (3.4 final is expected
>> for early 2014), if there is something we can do about them now?
> 
> Well, Christian said that his stdlib patch wasn't ready yet.

The patch is > 90% finished. All the hard work is already done.

Christian


From hrvoje.niksic at avl.com  Mon Mar 18 16:01:12 2013
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Mon, 18 Mar 2013 16:01:12 +0100
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
Message-ID: <51472C38.9090408@avl.com>

On 03/18/2013 03:23 PM, Chris Angelico wrote:
> The languages that permit you to assign to a function call all have
> some notion of a reference type.

Assigning to function calls is orthogonal to reference types.  For 
example, Python manages assignment to subscripts without having 
references just fine:

val = obj[index]      # val = obj.__getitem__(index)
obj[index] = val      # obj.__setitem__(index, val)

In analogy with that, Python could implement what looks like assignment 
to function call like this:

val = f(arg)          # val = f.__call__(arg)
f(arg) = val          # f.__setcall__(arg, val)

I am not arguing that this should be added, I'm only pointing out that 
Python's object customization is not fundamentally at odds with 
assignment to function calls.  Having said that, I am in fact arguing 
that Python doesn't need them.  All C++ uses of operator() overloads can 
be implemented with the subscript operator.

Even if one needs more different assignments than there are operators, 
Python can provide it as easily as C++.  For example, on 
std::vector::operator[] provides access to the container without error 
checking, and std::vector::at() checks bounds:

vec[i] = val      // no error checking
vec.at(i) = val   // error checking

This is trivially translated to Python as:

vec[i] = val      # primary functionality, use __setitem__
vec.at[i] = val   # secondary functionality, __setitem__ on a proxy


From steve at pearwood.info  Mon Mar 18 16:40:37 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Tue, 19 Mar 2013 02:40:37 +1100
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <51472C38.9090408@avl.com>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
	<51472C38.9090408@avl.com>
Message-ID: <51473575.7050406@pearwood.info>

On 19/03/13 02:01, Hrvoje Niksic wrote:
> On 03/18/2013 03:23 PM, Chris Angelico wrote:
>> The languages that permit you to assign to a function call all have
>> some notion of a reference type.
>
> Assigning to function calls is orthogonal to reference types.  For example, Python manages assignment to subscripts without having references just fine:
>
> val = obj[index]      # val = obj.__getitem__(index)
> obj[index] = val      # obj.__setitem__(index, val)
>
> In analogy with that, Python could implement what looks like assignment to function call like this:
>
> val = f(arg)          # val = f.__call__(arg)
> f(arg) = val          # f.__setcall__(arg, val)

That's all very well, but what would it do? It's not enough to say that the syntax could exist, we also need to have semantics. What's the use-case here? (That question is mostly aimed at the original poster.)

Aside: I'd reverse the order of the arg, val in any such hypothetical __setcall__, so as to support functions with zero or more arguments:

f(*args, **kwargs) = val  <=>  f.__setcall__(val, *args, **kwargs)



-- 
Steven

From ncoghlan at gmail.com  Mon Mar 18 16:50:08 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Mar 2013 08:50:08 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <20130318101311.GA21364@sleipnir.bytereef.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org>
	<20130318101311.GA21364@sleipnir.bytereef.org>
Message-ID: <CADiSq7d5oRTnR95eZ12ZuNyYzEFaBmwNOg=R1JRmqJWY5-dKMA@mail.gmail.com>

On Mon, Mar 18, 2013 at 3:13 AM, Stefan Krah <stefan at bytereef.org> wrote:
> Larry Hastings <larry at hastings.org> wrote:
>>     So I hope that at least converters can be declared statically in a header
>>     file, like I suggested in PEP 437.
>>
>>
>> The Argument Clinic prototype is written in Python; I don't know how "declared
>> static in a header file" applies to a Python implementation.  Currently the
>> converters are declared directly in clinic.py, somewhere in the middle.
>
> It applies in the same way to a Python implementation as declaring the
> DSL comment blocks in a C file applies to a Python implementation. This
> is exactly the same.
>
> 1) I think that third party tools should be able to extract *all* required
>    information from the DSL only.
>
> 2) After writing a new custom converter, I'd rather edit a header file and
>    not the preprocessor source.
>
> 3) Likewise, I'd rather edit a header file than inserting a magic [python]
>    block into the C source that registers the required information with
>    the preprocessor in a completely implementation defined way.

We didn't spend much time on this (we were focused on the per-function
DSL), but I agree a DSL for converters would be highly desirable, and
quite like the one in PEP 437.

It would require some tweaks to correctly handle the converter
parameterisation (for example, in the "es#" case, "encoding" is an
input to the converter rather than an output. This is why PEP 436 now
allows a callable notation for type converter references)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Mon Mar 18 16:57:05 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Mar 2013 08:57:05 -0700
Subject: [Python-Dev] [Python-checkins] peps: Update for 436,
 explicitly supporting positional parameters forever, amen.
In-Reply-To: <3ZTrcp6hZCzSLM@mail.python.org>
References: <3ZTrcp6hZCzSLM@mail.python.org>
Message-ID: <CADiSq7e01xs7Bpny=yZxepO7NGJBTP_QANAmXXqPQ100+dVxhQ@mail.gmail.com>

On Mon, Mar 18, 2013 at 1:47 AM, larry.hastings
<python-checkins at python.org> wrote:
>  Notes / TBD
>  ===========
>
> +* The DSL currently makes no provision for specifying per-parameter
> +  type annotations.  This is something explicitly supported in Python;
> +  it should be supported for builtins too, once we have reflection support.
> +
> +  It seems to me that the syntax for parameter lines--dictated by
> +  Guido--suggests conversion functions are themselves type annotations.
> +  This makes intuitive sense.  But my thought experiments in how to
> +  convert the conversion function specification into a per-parameter
> +  type annotation ranged from obnoxious to toxic; I don't think that
> +  line of thinking will bear fruit.
> +
> +  Instead, I think wee need to add a new syntax allowing functions
> +  to explicitly specify a per-parameter type annotation.  The problem:
> +  what should that syntax be?  I've only had one idea so far, and I
> +  don't find it all that appealing: allow a optional second colon
> +  on the parameter line, and the type annotation would be specified...
> +  somewhere, either between the first and second colons, or between
> +  the second colon and the (optional) default.
> +
> +  Also, I don't think this could specify any arbitrary Python value.
> +  I suspect it would suffer heavy restrictions on what types and
> +  literals it could use.  Perhaps the best solution would be to
> +  store the exact string in static data, and have Python evaluate
> +  it on demand?  If so, it would be safest to restrict it to Python
> +  literal syntax, permitting no function calls (even to builtins).
> +

I think the hack we're using for the default-as-shown-in-Python will
work here as well: use the converter parameterisation notation.

Then "pydefault" (I think that is a better name than the current
"default") and "pynote" would control what is shown for the conversion
in the first line of the docstring and in any future introspection
support. If either is not given, then the C default would be passed
through as the Python default and the annotation would be left blank.

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From skip at pobox.com  Mon Mar 18 16:57:03 2013
From: skip at pobox.com (Skip Montanaro)
Date: Mon, 18 Mar 2013 10:57:03 -0500
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <ki762l$gfs$1@ger.gmane.org>
References: <ki762l$gfs$1@ger.gmane.org>
Message-ID: <CANc-5UzryGeFZqqHFFChS+viKV2xnb0MUf5kfvK+bX32nAP-kA@mail.gmail.com>

On Mon, Mar 18, 2013 at 8:50 AM, Neal Becker <ndbecker2 at gmail.com> wrote:
> def F(x):
>     return x
>
> x = 2
> F(x) = 3
>
>     F(x) = 3
> SyntaxError: can't assign to function call
>
> Do we really need this restriction?  There do exist other languages without it.

I think this belongs on python-ideas before launching into discussions
of syntax and semantics on python-dev.

Skip

From ncoghlan at gmail.com  Mon Mar 18 17:28:26 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Mar 2013 09:28:26 -0700
Subject: [Python-Dev] [PEP 437] A DSL for specifying signatures,
 annotations and argument converters
In-Reply-To: <51456D3C.10608@hastings.org>
References: <20130316091750.GA24061@sleipnir.bytereef.org>
	<51456D3C.10608@hastings.org>
Message-ID: <CADiSq7cKhefabdJXKA0p44Oruh12wDu1-RnxWRBG02Dh=-BB+A@mail.gmail.com>

On Sun, Mar 17, 2013 at 12:14 AM, Larry Hastings <larry at hastings.org> wrote:
> On 03/16/2013 02:17 AM, Stefan Krah wrote:
>
> Both PEPs were discussed at PyCon.  The state of affairs is that a
> compromise is being worked upon and will be published by Larry in
> a revised PEP.
>
>
> I've pushed an update to PEP 436, the Argument Clinic PEP.  It's now live on
> python.org.

Thanks for that. A few comments.

* I'm confused by the leading table - is it possible/expected to have
a module declaration, class declaration *and* function declaration in
the same block? If not, it seems more appropriate to have 3 tables, or
else simplify the initial table to omit the declaration details, and
explain the available options later.

* Is it possible to use "module_name.class_name.method_name" when
declaring a function that will be used as a method in a class?

* To match the behaviour of Python functions, function docstrings
should also be optional in Argument Clinic. We'll always include at
least the function prototype, even if no other content is specified.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From hrvoje.niksic at avl.com  Mon Mar 18 17:36:17 2013
From: hrvoje.niksic at avl.com (Hrvoje Niksic)
Date: Mon, 18 Mar 2013 17:36:17 +0100
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <51473575.7050406@pearwood.info>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
	<51472C38.9090408@avl.com> <51473575.7050406@pearwood.info>
Message-ID: <51474281.7060501@avl.com>

On 03/18/2013 04:40 PM, Steven D'Aprano wrote:
>> In analogy with that, Python could implement what looks like assignment to function call like this:
>>
>> val = f(arg)          # val = f.__call__(arg)
>> f(arg) = val          # f.__setcall__(arg, val)
>
> That's all very well, but what would it do? It's not enough to say
 > that the syntax could exist, we also need to have semantics.

I am not the best person to answer because I go on to argue that this 
syntax is not needed in Python at all (anything it can do can be 
implemented with __setitem__ at no loss of clarity).  Still, if such a 
feature existed in Python, I imagine people would use it to set the same 
resource that the function obtains, where such a thing is applicable.

> Aside: I'd reverse the order of the arg, val in any such hypothetical
 > __setcall__, so as to support functions with zero or more arguments:
>
> f(*args, **kwargs) = val  <=>  f.__setcall__(val, *args, **kwargs)

That would be a better design, I agree.


From storchaka at gmail.com  Mon Mar 18 18:25:02 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Mon, 18 Mar 2013 19:25:02 +0200
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <51473575.7050406@pearwood.info>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
	<51472C38.9090408@avl.com> <51473575.7050406@pearwood.info>
Message-ID: <ki7ild$v3f$1@ger.gmane.org>

On 18.03.13 17:40, Steven D'Aprano wrote:
> On 19/03/13 02:01, Hrvoje Niksic wrote:
>> Assigning to function calls is orthogonal to reference types.  For
>> example, Python manages assignment to subscripts without having
>> references just fine:
>>
>> val = obj[index]      # val = obj.__getitem__(index)
>> obj[index] = val      # obj.__setitem__(index, val)
>>
>> In analogy with that, Python could implement what looks like
>> assignment to function call like this:
>>
>> val = f(arg)          # val = f.__call__(arg)
>> f(arg) = val          # f.__setcall__(arg, val)
>
> That's all very well, but what would it do? It's not enough to say that
> the syntax could exist, we also need to have semantics. What's the
> use-case here? (That question is mostly aimed at the original poster.)

Python could use parenthesis instead of brackets for indexing and a 
dictionary lookup. However it is too late to discuss this idea.



From guido at python.org  Mon Mar 18 19:02:49 2013
From: guido at python.org (Guido van Rossum)
Date: Mon, 18 Mar 2013 11:02:49 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <20130318103643.GA21550@sleipnir.bytereef.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org> <5146D4E5.1020302@hastings.org>
	<20130318103643.GA21550@sleipnir.bytereef.org>
Message-ID: <CAP7+vJL4osXyUHdhi2=VjykkJngNFZw368t_-8CNnKeLPmBv7A@mail.gmail.com>

On Mon, Mar 18, 2013 at 3:36 AM, Stefan Krah <stefan at bytereef.org> wrote:
> Larry Hastings <larry at hastings.org> wrote:
>> * The DSL currently makes no provision for specifying per-parameter
>>   type annotations.  This is something explicitly supported in Python;
>>   it should be supported for builtins too, once we have reflection support.
>>
>>   It seems to me that the syntax for parameter lines--dictated by
>>   Guido--suggests conversion functions are themselves type annotations.
>>   This makes intuitive sense.
>
> Really, did you read PEP 437? It's all in there.

This attitude is unhelpful. Please stop being outright hostile. If you
want to have any influence on the outcome at all, consider looking
into compromises.

>>  But my thought experiments in how to convert the conversion function
>>  specification into a per-parameter type annotation ranged from obnoxious
>>  to toxic; I don't think that
>>  line of thinking will bear fruit.
>
> Did you look at the patch that I posted in issue #16612? It's already
> implemented:
>
> $ ./printsemant Tools/preprocess/testcases/posix_stat.c
> PROGRAM[
>   SOURCE[...],
>   DEFINE[
>     CNAME posix_stat,
>     SPEC[
>       DECLARATION
>          { fun_fqname = os.stat,
>            fun_name = stat,
>            fun_cname = posix_stat,
>            fun_kind = Keywords,
>            fun_params = [
>              { param_name = path,
>                param_type = [bytes, int, str],     <== here it is
>                param_default = NONE,
>                param_kind = (PosOrKwd, Required),
>                param_conv = path_converter,
>                param_parseargs = [
>                  ConvArg { arg_name = path_converter,
>                            arg_type = int (*converter)(PyObject *, void *)
>                            arg_use_ptr = false },
>                  MainArg { arg_name = path,
>                            arg_type = path_t,
>                            arg_use_ptr = true }]},
> [...]

I can assure you nobody downloaded your binaries. The security
implications are just too scary.

-- 
--Guido van Rossum (python.org/~guido)

From guido at python.org  Mon Mar 18 18:57:56 2013
From: guido at python.org (Guido van Rossum)
Date: Mon, 18 Mar 2013 10:57:56 -0700
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <ki7ild$v3f$1@ger.gmane.org>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
	<51472C38.9090408@avl.com> <51473575.7050406@pearwood.info>
	<ki7ild$v3f$1@ger.gmane.org>
Message-ID: <CAP7+vJLtvYbdcZLD+Zb-4uFZ_jtemHP0c2v5Et0vAh-JaS-KWQ@mail.gmail.com>

Move. This. Thread. Out. Of. Python-Dev. Now. (python-ideas is the right place.)

On Mon, Mar 18, 2013 at 10:25 AM, Serhiy Storchaka <storchaka at gmail.com> wrote:
> On 18.03.13 17:40, Steven D'Aprano wrote:
>>
>> On 19/03/13 02:01, Hrvoje Niksic wrote:
>>>
>>> Assigning to function calls is orthogonal to reference types.  For
>>> example, Python manages assignment to subscripts without having
>>> references just fine:
>>>
>>> val = obj[index]      # val = obj.__getitem__(index)
>>> obj[index] = val      # obj.__setitem__(index, val)
>>>
>>> In analogy with that, Python could implement what looks like
>>> assignment to function call like this:
>>>
>>> val = f(arg)          # val = f.__call__(arg)
>>> f(arg) = val          # f.__setcall__(arg, val)
>>
>>
>> That's all very well, but what would it do? It's not enough to say that
>> the syntax could exist, we also need to have semantics. What's the
>> use-case here? (That question is mostly aimed at the original poster.)
>
>
> Python could use parenthesis instead of brackets for indexing and a
> dictionary lookup. However it is too late to discuss this idea.
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/guido%40python.org



--
--Guido van Rossum (python.org/~guido)

From guido at python.org  Mon Mar 18 19:04:39 2013
From: guido at python.org (Guido van Rossum)
Date: Mon, 18 Mar 2013 11:04:39 -0700
Subject: [Python-Dev] [Python-checkins] peps: Update for 436,
 explicitly supporting positional parameters forever, amen.
In-Reply-To: <CADiSq7e01xs7Bpny=yZxepO7NGJBTP_QANAmXXqPQ100+dVxhQ@mail.gmail.com>
References: <3ZTrcp6hZCzSLM@mail.python.org>
	<CADiSq7e01xs7Bpny=yZxepO7NGJBTP_QANAmXXqPQ100+dVxhQ@mail.gmail.com>
Message-ID: <CAP7+vJ+_XoXSN+U4x2zoiqmNSR5=AJj4c=MhDa-R2igbc1_6ig@mail.gmail.com>

On Mon, Mar 18, 2013 at 8:57 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> On Mon, Mar 18, 2013 at 1:47 AM, larry.hastings
> <python-checkins at python.org> wrote:
>>  Notes / TBD
>>  ===========
>>
>> +* The DSL currently makes no provision for specifying per-parameter
>> +  type annotations.  This is something explicitly supported in Python;
>> +  it should be supported for builtins too, once we have reflection support.
>> +
>> +  It seems to me that the syntax for parameter lines--dictated by
>> +  Guido--suggests conversion functions are themselves type annotations.
>> +  This makes intuitive sense.  But my thought experiments in how to
>> +  convert the conversion function specification into a per-parameter
>> +  type annotation ranged from obnoxious to toxic; I don't think that
>> +  line of thinking will bear fruit.
>> +
>> +  Instead, I think wee need to add a new syntax allowing functions
>> +  to explicitly specify a per-parameter type annotation.  The problem:
>> +  what should that syntax be?  I've only had one idea so far, and I
>> +  don't find it all that appealing: allow a optional second colon
>> +  on the parameter line, and the type annotation would be specified...
>> +  somewhere, either between the first and second colons, or between
>> +  the second colon and the (optional) default.
>> +
>> +  Also, I don't think this could specify any arbitrary Python value.
>> +  I suspect it would suffer heavy restrictions on what types and
>> +  literals it could use.  Perhaps the best solution would be to
>> +  store the exact string in static data, and have Python evaluate
>> +  it on demand?  If so, it would be safest to restrict it to Python
>> +  literal syntax, permitting no function calls (even to builtins).
>> +
>
> I think the hack we're using for the default-as-shown-in-Python will
> work here as well: use the converter parameterisation notation.
>
> Then "pydefault" (I think that is a better name than the current
> "default") and "pynote" would control what is shown for the conversion
> in the first line of the docstring and in any future introspection
> support. If either is not given, then the C default would be passed
> through as the Python default and the annotation would be left blank.

Right. In fact, I think the decision of what (if anything) should be
put in the annotation should be up to the converter class. It can be a
specific method on the converter object.

-- 
--Guido van Rossum (python.org/~guido)

From "ja...py" at farowl.co.uk  Mon Mar 18 21:26:05 2013
From: "ja...py" at farowl.co.uk (Jeff Allen)
Date: Mon, 18 Mar 2013 20:26:05 +0000
Subject: [Python-Dev] Recent changes to TextIOWrapper and its tests
Message-ID: <5147785D.3070100@farowl.co.uk>

I'm pulling recent changes in the io module across to Jython. I am 
looking for help understanding the changes in
http://hg.python.org/cpython/rev/19a33ef3821d

That change set is about what should happen if the underlying buffer 
does not return bytes when read, but instead, for example, unicode 
characters. The test test_read_nonbytes() constructs a pathological text 
stream reader t where the usual BytesIO or BufferedReader is replaced 
with a StringIO. It then checks that r.read(1) and t.readlines() raise a 
TypeError, that is, it tests that TextIOWrapper checks the type of what 
it reads from the buffer.

The puzzle is that it requires t.read() to succeed.

When I insert a check for bytes type in all the places it seems 
necessary in my code, I pass the first two conditions, but since 
t.read() also raises TypeError, the overall test fails. Is reading the 
stream with read() intended to succeed? Why is this desired?

Jeff Allen




From greg.ewing at canterbury.ac.nz  Mon Mar 18 22:50:05 2013
From: greg.ewing at canterbury.ac.nz (Greg Ewing)
Date: Tue, 19 Mar 2013 10:50:05 +1300
Subject: [Python-Dev] can't assign to function call
In-Reply-To: <51474281.7060501@avl.com>
References: <ki762l$gfs$1@ger.gmane.org>
	<CAPTjJmqLo0MQYSZtRPe=tj-mx5bii72WVwKUA_evrBr9ApmA4Q@mail.gmail.com>
	<51472C38.9090408@avl.com> <51473575.7050406@pearwood.info>
	<51474281.7060501@avl.com>
Message-ID: <51478C0D.4090706@canterbury.ac.nz>

Hrvoje Niksic wrote:
> I am not the best person to answer because I go on to argue that this 
> syntax is not needed in Python at all (anything it can do can be 
> implemented with __setitem__ at no loss of clarity).

I would even argue that the proxy solution is even *better*
for that particular use case, because it makes both operations
look like forms of indexing, which they are.

-- 
Greg

From larry at hastings.org  Mon Mar 18 23:43:10 2013
From: larry at hastings.org (Larry Hastings)
Date: Mon, 18 Mar 2013 15:43:10 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <80206D7B-ACAB-4108-BC13-D6AC2BB2E7CB@mac.com>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org>
	<80206D7B-ACAB-4108-BC13-D6AC2BB2E7CB@mac.com>
Message-ID: <5147987E.9070902@hastings.org>

On 03/18/2013 02:29 AM, Ronald Oussoren wrote:
> On 18 Mar, 2013, at 8:16, Larry Hastings <larry at hastings.org> wrote:
>> This has some consequences.  For example, inspect.getfullargspec, inspect.Signature, and indeed types.FunctionObject and types.CodeObject have no currently defined mechanism for communicating that a parameter is positional-only.
> inspect.Signature does have support for positional-only arguments, they have inspect.Parameter.POSITIONAL_ONLY as their kind.

You're right!  And I should have remembered that--I was one of the 
authors of the inspect.Signature PEP.  It's funny, it can represent 
something that it has no way of inferring ;-)


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/e6ed2a1a/attachment.html>

From ronaldoussoren at mac.com  Mon Mar 18 23:50:21 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Mon, 18 Mar 2013 23:50:21 +0100
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
	slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <5147987E.9070902@hastings.org>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org>
	<80206D7B-ACAB-4108-BC13-D6AC2BB2E7CB@mac.com>
	<5147987E.9070902@hastings.org>
Message-ID: <7422CE81-912B-4F81-BC69-6590839B63C6@mac.com>


On 18 Mar, 2013, at 23:43, Larry Hastings <larry at hastings.org> wrote:

> On 03/18/2013 02:29 AM, Ronald Oussoren wrote:
>> On 18 Mar, 2013, at 8:16, Larry Hastings <larry at hastings.org> wrote:
>>> This has some consequences.  For example, inspect.getfullargspec, inspect.Signature, and indeed types.FunctionObject and types.CodeObject have no currently defined mechanism for communicating that a parameter is positional-only.
>> inspect.Signature does have support for positional-only arguments, they have inspect.Parameter.POSITIONAL_ONLY as their kind.
> 
> You're right!  And I should have remembered that--I was one of the authors of the inspect.Signature PEP.  It's funny, it can represent something that it has no way of inferring ;-)

It doesn't necessarily have to, builtin functions could grow a __signature__ attribute that calculates the signature (possibly from the DSL data). I've done something like that in a pre-release version of PyObjC, and with some patching of pydoc and inspect (see #17053) I now have useful help information for what are basicly builtin functions with positional-only arguments.

Ronald

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/16afbe8a/attachment.html>

From ncoghlan at gmail.com  Tue Mar 19 00:46:52 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 18 Mar 2013 16:46:52 -0700
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <CAP7+vJL4osXyUHdhi2=VjykkJngNFZw368t_-8CNnKeLPmBv7A@mail.gmail.com>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org> <5146D4E5.1020302@hastings.org>
	<20130318103643.GA21550@sleipnir.bytereef.org>
	<CAP7+vJL4osXyUHdhi2=VjykkJngNFZw368t_-8CNnKeLPmBv7A@mail.gmail.com>
Message-ID: <CADiSq7e3J0ZK-nO7gXMyG52ELjwuiC5JhP8gBAtLodMwJXyjGg@mail.gmail.com>

On Mon, Mar 18, 2013 at 11:02 AM, Guido van Rossum <guido at python.org> wrote:
> On Mon, Mar 18, 2013 at 3:36 AM, Stefan Krah <stefan at bytereef.org> wrote:
>> Larry Hastings <larry at hastings.org> wrote:
>>> * The DSL currently makes no provision for specifying per-parameter
>>>   type annotations.  This is something explicitly supported in Python;
>>>   it should be supported for builtins too, once we have reflection support.
>>>
>>>   It seems to me that the syntax for parameter lines--dictated by
>>>   Guido--suggests conversion functions are themselves type annotations.
>>>   This makes intuitive sense.
>>
>> Really, did you read PEP 437? It's all in there.
>
> This attitude is unhelpful. Please stop being outright hostile. If you
> want to have any influence on the outcome at all, consider looking
> into compromises.

While I actually agree with Stefan that it's important to eventually
have a converter DSL, I don't think it's necessary to have it in the
initial implementation.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From kristjan at ccpgames.com  Tue Mar 19 00:40:32 2013
From: kristjan at ccpgames.com (=?iso-8859-1?Q?Kristj=E1n_Valur_J=F3nsson?=)
Date: Mon, 18 Mar 2013 23:40:32 +0000
Subject: [Python-Dev] newbuffer support in python 2.7
Message-ID: <EFE3877620384242A686D52278B7CCD353817424@RKV-IT-EXCH104.ccp.ad.local>

Hi python dev.
I have two languishing defects regarding 2.7 and how buffer support isn't complete there.
http://bugs.python.org/issue10211
http://bugs.python.org/issue10212

In both cases, the new style buffer support is incomplete, and the patches close usability holes having to do with memoryview objects.
It was suggested to me that I put it to python-dev to decide if we should consider this a bug to be fixed or not, and hopefully get a consensus before 2.7.4 freeze.
I have been running a local patch of 2.7 with those fixes for two years now.

Cheers,
Kristj?n


-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/39c8afec/attachment-0001.html>

From storchaka at gmail.com  Tue Mar 19 00:56:51 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 Mar 2013 01:56:51 +0200
Subject: [Python-Dev] Recent changes to TextIOWrapper and its tests
In-Reply-To: <5147785D.3070100@farowl.co.uk>
References: <5147785D.3070100@farowl.co.uk>
Message-ID: <ki89k4$7q1$1@ger.gmane.org>

On 18.03.13 22:26, Jeff Allen wrote:
> The puzzle is that it requires t.read() to succeed.
>
> When I insert a check for bytes type in all the places it seems
> necessary in my code, I pass the first two conditions, but since
> t.read() also raises TypeError, the overall test fails. Is reading the
> stream with read() intended to succeed? Why is this desired?

This is not desired. I just registered the current behavior. Python 3 is 
more strict and always raises an exception.

Perhaps this test should be relaxed. I.e. use

     with self.maybeRaises(TypeError):
         t.read()

and define maybeRaises() as:

     @contextlib.contextmanager
     def maybeRaises(self, *args, **kwds):
         try:
             yield
         except args:
             pass



From tismer at stackless.com  Tue Mar 19 01:27:33 2013
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 18 Mar 2013 17:27:33 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314184519.GD24307@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<20130314184519.GD24307@snakebite.org>
Message-ID: <5147B0F5.2030107@stackless.com>

Hi Trent,

I just started to try to understand the idea and the implications.
Removing almost all of your message since that is already too long
to work with:
The reference is

http://mail.python.org/pipermail/python-dev/2013-March/124690.html

On 3/14/13 11:45 AM, Trent Nelson wrote:
> On Wed, Mar 13, 2013 at 07:05:41PM -0700, Trent Nelson wrote:
>>      Just posted the slides for those that didn't have the benefit of
>>      attending the language summit today:
>>
>>          https://speakerdeck.com/trent/parallelizing-the-python-interpreter-an-alternate-approach-to-async
>      Someone on /r/python asked if I could elaborate on the "do Y" part
>      of "if we're in a parallel thread, do Y, if not, do X", which I
>      (inadvertently) ended up replying to in detail.  I've included the
>      response below.  (I'll work on converting this into a TL;DR set of
>      slides soon.)
>
>> Can you go into a bit of depth about "X" here?
> That's a huge topic that I'm hoping to tackle ASAP.  The basic premise
> is that parallel 'Context' objects (well, structs) are allocated for
> each parallel thread callback.  The context persists for the lifetime of
> the "parallel work".

<big snip>

> So, the remaining challenge is preventing the use case alluded to
> earlier where someone tries to modify an object that hasn't been "async
> protected".  That's a bit harder.  The idea I've got in mind is to
> instrument the main CPython ceval loop, such that we do these checks as
> part of opcode processing.  That allows us to keep all the logic in the
> one spot and not have to go hacking the internals of every single
> object's C backend to ensure correctness.
>
> Now, that'll probably work to an extent.  I mean, after all, there are
> opcodes for all the things we'd be interested in instrumenting,
> LOAD_GLOBAL, STORE_GLOBAL, SETITEM etc.  What becomes challenging is
> detecting arbitrary mutations via object calls, i.e. how do we know,
> during the ceval loop, that foo.append(x) needs to be treated specially
> if foo is a main-thread object and x is a parallel thread object?
>
> There may be no way to handle that *other* than hacking the internals of
> each object, unfortunately.  So, the viability of this whole approach
> may rest on whether or that's deemed as an acceptable tradeoff (a
> necessary evil, even) to the Python developer community.

This is pretty much my concern:
In order to make this waterproof, as required for CPython, you will quite
likely have to do something on very many objects, and this is hard
to chime into CPython.

>
> If it's not, then it's unlikely this approach will ever see the light of
> day in CPython.  If that turns out to be the case, then I see this
> project taking the path that Stackless took (forking off and becoming a
> separate interpreter).

We had that discussion quite often for Stackless, and I would love to find
a solution that allows to add special versions and use cases to CPython
in a way that avoids the forking as we did it.

It would be a nice thing if we could come up with a way to keep CPython
in place, but to swap the interpreter out and replace it with a specialized
version, if the application needs it. I wonder to what extent that would be
possible.
What I would like to achieve, after having given up on Stackless integration
is a way to let it piggyback onto CPython that works like an extension
module, although it hat effectively replace larger parts of the interpreter.
I wonder if that might be the superior way to have more flexibility, 
without forcing
everything and all go into CPython.
If we can make the interpreter somehow pluggable at runtime, a lot of issues
would become much simpler.

>
> There's nothing wrong with that; I am really excited about the
> possibilities afforded by this approach, and I'm sure it will pique the
> interest of commercial entities out there that have problems perfectly
> suited to where this pattern excels (shared-nothing, highly concurrent),
> much like the relationship that developed between Stackless and Eve
> Online.
>

What do you think: does it make sense to think of a framework that
allows to replace the interpreter at runtime, without making normal
CPython really slower?

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
Software Consulting          :     Have a break! Take a ride on Python's
Karl-Liebknecht-Str. 121     :    *Starship* http://starship.python.net/
14482 Potsdam                :     PGP key -> http://pgp.uni-mainz.de
phone +49 173 24 18 776  fax +49 (30) 700143-0023
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
       whom do you want to sponsor today?   http://www.stackless.com/


From trent at snakebite.org  Tue Mar 19 02:26:45 2013
From: trent at snakebite.org (Trent Nelson)
Date: Mon, 18 Mar 2013 18:26:45 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <5147B0F5.2030107@stackless.com>
References: <20130314020540.GB22505@snakebite.org>
	<20130314184519.GD24307@snakebite.org>
	<5147B0F5.2030107@stackless.com>
Message-ID: <20130319012644.GF29928@snakebite.org>

On Mon, Mar 18, 2013 at 05:27:33PM -0700, Christian Tismer wrote:
> Hi Trent,

    Hi Christian!  Thanks for taking the time to read my walls of text
    ;-)

> > So, the remaining challenge is preventing the use case alluded to
> > earlier where someone tries to modify an object that hasn't been "async
> > protected".  That's a bit harder.  The idea I've got in mind is to
> > instrument the main CPython ceval loop, such that we do these checks as
> > part of opcode processing.  That allows us to keep all the logic in the
> > one spot and not have to go hacking the internals of every single
> > object's C backend to ensure correctness.
> >
> > Now, that'll probably work to an extent.  I mean, after all, there are
> > opcodes for all the things we'd be interested in instrumenting,
> > LOAD_GLOBAL, STORE_GLOBAL, SETITEM etc.  What becomes challenging is
> > detecting arbitrary mutations via object calls, i.e. how do we know,
> > during the ceval loop, that foo.append(x) needs to be treated specially
> > if foo is a main-thread object and x is a parallel thread object?
> >
> > There may be no way to handle that *other* than hacking the internals of
> > each object, unfortunately.  So, the viability of this whole approach
> > may rest on whether or that's deemed as an acceptable tradeoff (a
> > necessary evil, even) to the Python developer community.
> 
> This is pretty much my concern:
> In order to make this waterproof, as required for CPython, you will quite
> likely have to do something on very many objects, and this is hard
> to chime into CPython.

    Actually, I think I was unnecessarily pessimistic here.  When I sent
    that follow-up mail with cross-references, I realized I'd forgotten
    the nitty gritty details of how I implemented the async protection
    support.

    It turns out I'd already started on protecting lists (or rather,
    PySequenceMethods), but decided to stop as the work I'd done on the
    PyMappingMethods was sufficient for my needs at the time.

    All I *really* want to do is raise an exception if a parallel object
    gets assigned to a main-thread container object (list/dict etc) that
    hasn't been "async protected".  (As opposed to now, where it'll
    either segfault or silently corrupt stuff, then segfault later.)

    I've already got all the infrastructure in place to test that (I use
    it extensively within pyparallel.c):

        Py_ISPY(obj)         - detect a main-thread object
        Py_ISPX(obj)         - detect a parallel-thread object
        Py_IS_PROTECTED(obj) - detect if a main-thread object has
                               been protected*

            [*]: actually, this isn't in a macro form right now, it's a
            cheeky inline:

                __inline
                char
                _protected(PyObject *obj)
                {
                    return (obj->px_flags & Py_PXFLAGS_RWLOCK);
                }

    As those macros are exposed in the public <Include/pyparallel.h>,
    they can be used in other parts of the code base.  So, it's just a
    matter of finding the points where an `lvalue = rvalue` takes place;
    where: ``Py_ISPY(lvalue) && Py_ISPX(rvalue)``.  Then a test to see
    if lvalue is protected; if not, raise an exception.  If so, then
    nothing else needs to be done.

    And there aren't that many places where this happens.  (It didn't
    take long to get the PyMappingMethods intercepts nailed down.)

    That's the idea anyway.  I need to get back to coding to see how it
    all plays out in practice.  "And there aren't many places where this
    happens" might be my famous last words.

> > If it's not, then it's unlikely this approach will ever see the light of
> > day in CPython.  If that turns out to be the case, then I see this
> > project taking the path that Stackless took (forking off and becoming a
> > separate interpreter).
> 
> We had that discussion quite often for Stackless, and I would love to find
> a solution that allows to add special versions and use cases to CPython
> in a way that avoids the forking as we did it.
> 
> It would be a nice thing if we could come up with a way to keep CPython
> in place, but to swap the interpreter out and replace it with a specialized
> version, if the application needs it. I wonder to what extent that would be
> possible.
> What I would like to achieve, after having given up on Stackless integration
> is a way to let it piggyback onto CPython that works like an extension
> module, although it hat effectively replace larger parts of the interpreter.
> I wonder if that might be the superior way to have more flexibility, 
> without forcing
> everything and all go into CPython.
> If we can make the interpreter somehow pluggable at runtime, a lot of issues
> would become much simpler.
> 
> >
> > There's nothing wrong with that; I am really excited about the
> > possibilities afforded by this approach, and I'm sure it will pique the
> > interest of commercial entities out there that have problems perfectly
> > suited to where this pattern excels (shared-nothing, highly concurrent),
> > much like the relationship that developed between Stackless and Eve
> > Online.
> >
> 
> What do you think: does it make sense to think of a framework that
> allows to replace the interpreter at runtime, without making normal
> CPython really slower?

    I think there may actually be some interest in what you're
    suggesting.  I had a chat with various other groups over PyCon that
    had some interesting things in the pipeline, and being able to call
    back out to CPython internals like you mentioned would be useful to
    them.

    I don't want to take my pyparallel work in that direction yet; I'm
    still hoping I can fix all the show stoppers and have the whole
    thing eligible for CPython inclusion one day ;-)  (So, I'm like you,
    10 years ago? :P)

    Regards,

        Trent.

From larry at hastings.org  Tue Mar 19 05:45:09 2013
From: larry at hastings.org (Larry Hastings)
Date: Mon, 18 Mar 2013 21:45:09 -0700
Subject: [Python-Dev] Rough idea for adding introspection information for
	builtins
Message-ID: <5147ED55.30605@hastings.org>



The original impetus for Argument Clinic was adding introspection 
information for builtins--it seemed like any manual approach I came up 
with would push the builtins maintenance burden beyond the pale.

Assuming that we have Argument Clinic or something like it, we don't 
need to optimize for ease of use from the API end--we can optimize for 
data size.  So the approach writ large: store a blob of data associated 
with each entry point, as small as possible. Reconstitute the 
appropriate inspect.Signature on demand by reading that blob.

Where to store the data?  PyMethodDef is the obvious spot, but I think 
that structure is part of the stable ABI.  So we'd need a new 
PyMethodDefEx and that'd be a little tiresome.  Less violent to the ABI 
would be defining a new array of pointers-to-introspection-blobs, 
parallel to the PyMethodDef array, passed in via a new entry point.


On to the representation.  Consider the function

    def foo(arg, b=3, *, kwonly='a'):
         pass

I considered four approaches, each listed below along with its total 
size if it was stored as C static data.

1. A specialized bytecode format, something like pickle, like this:

    bytes([ PARAMETER_START_LENGTH_3, 'a', 'r', 'g',
       PARAMETER_START_LENGTH_1, 'b', PARAMETER_DEFAULT_LENGTH_1, '3',
       KEYWORD_ONLY,
       PARAMETER_START_LENGTH_6, 'k', 'w', 'o', 'n', 'l', 'y',
    PARAMETER_DEFAULT_LENGTH_3, '\'', 'a', '\'',
       END
       ])

Length: 20 bytes.

2. Just use pickle--pickle the result of inspect.signature() run on a 
mocked-up signature, just store that.   Length: 130 bytes. (Assume a 
two-byte size stored next to it.)

3. Store a string that, if eval'd, would produce the inspect.Signature.  
Length: 231 bytes.  (This could be made smaller if we could assume "from 
inspect import *" or "p = inspect.Parameter" or something, but it'd 
still be easily the heaviest.)

4. Store a string that looks like the Python declaration of the 
signature, and parse it (Nick's suggestion).  For foo above, this would 
be "(arg,b=3,*,kwonly='a')".  Length: 23 bytes.

Of those, Nick's suggestion seems best.  It's slightly bigger than the 
specialized bytecode format, but it's human-readable (and 
human-writable!), and it'd be the easiest to implement.


My first idea for implementation: add a "def x" to the front and ": 
pass" to the end, then run it through ast.parse.  Iterate over the tree, 
converting parameters into inspect.Parameters and handling the return 
annotation if present.  Default values and annotations would be turned 
into values by ast.eval_literal.  (It wouldn't surprise me if there's a 
cleaner way to do it than the fake function definition; I'm not familiar 
with the ast module.)

We'd want one more mild hack: the DSL will support positional 
parameters, and inspect.Signature supports positional parameters, so 
it'd be nice to render that information.  But we can't represent that in 
Python syntax (or at least not yet!), so we can't let ast.parse see it.  
My suggestion: run it through ast.parse, and if it throws a SyntaxError 
see if the problem was a slash.  If it was, remove the slash, reprocess 
through ast.parse, and remember that all parameters are positional-only 
(and barf if there are kwonly, args, or kwargs).


Thoughts?


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130318/1e7492be/attachment.html>

From stefan_ml at behnel.de  Tue Mar 19 07:08:48 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Tue, 19 Mar 2013 07:08:48 +0100
Subject: [Python-Dev] Rough idea for adding introspection information
	for builtins
In-Reply-To: <5147ED55.30605@hastings.org>
References: <5147ED55.30605@hastings.org>
Message-ID: <ki8vdd$ne3$1@ger.gmane.org>

Larry Hastings, 19.03.2013 05:45:
> The original impetus for Argument Clinic was adding introspection
> information for builtins [...]
> On to the representation.  Consider the function
> 
>    def foo(arg, b=3, *, kwonly='a'):
>         pass
> [...]
> 4. Store a string that looks like the Python declaration of the signature,
> and parse it (Nick's suggestion).  For foo above, this would be
> "(arg,b=3,*,kwonly='a')".

I had already noted that this would be generally useful, specifically for
Cython, so I'm all for going this route. No need to invent something new here.


>  Length: 23 bytes.

I can't see why the size would matter in any way.


> Of those, Nick's suggestion seems best.  It's slightly bigger than the
> specialized bytecode format, but it's human-readable (and human-writable!),
> and it'd be the easiest to implement.

Plus, if it becomes the format how C level signatures are expressed anyway,
it wouldn't require any additional build time preprocessing.


> My first idea for implementation: add a "def x" to the front and ": pass"
> to the end

Why not require it to be there already? Maybe more like

    def foo(arg, b=3, *, kwonly='a'):
         ...

(i.e. using Ellipsis instead of pass, so that it's clear that it's not an
empty function but one the implementation of which is hidden)


> then run it through ast.parse.  Iterate over the tree,
> converting parameters into inspect.Parameters and handling the return
> annotation if present.  Default values and annotations would be turned into
> values by ast.eval_literal.  (It wouldn't surprise me if there's a cleaner
> way to do it than the fake function definition; I'm not familiar with the
> ast module.)

IMHO, if there is no straight forward way currently to convert a function
header from a code blob into a Signature object in Python code, preferably
using the ast module (either explicitly or implicitly through inspect.py),
then that's a bug.


> We'd want one more mild hack: the DSL will support positional parameters,
> and inspect.Signature supports positional parameters, so it'd be nice to
> render that information.  But we can't represent that in Python syntax (or
> at least not yet!), so we can't let ast.parse see it.  My suggestion: run
> it through ast.parse, and if it throws a SyntaxError see if the problem was
> a slash.  If it was, remove the slash, reprocess through ast.parse, and
> remember that all parameters are positional-only (and barf if there are
> kwonly, args, or kwargs).

Is sounds simpler to me to just make it a Python syntax feature. Or at
least an optional one, supported by the ast module with a dedicated
compiler flag.

Stefan



From ncoghlan at gmail.com  Tue Mar 19 08:23:47 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Mar 2013 00:23:47 -0700
Subject: [Python-Dev] Rough idea for adding introspection information
	for builtins
In-Reply-To: <ki8vdd$ne3$1@ger.gmane.org>
References: <5147ED55.30605@hastings.org>
	<ki8vdd$ne3$1@ger.gmane.org>
Message-ID: <CADiSq7d-BzzbpYSr4nnyPJyDcnigLWvV8X40V=Pd_onwBL4yYQ@mail.gmail.com>

On Mon, Mar 18, 2013 at 11:08 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
> I can't see why the size would matter in any way.

We're mildly concerned about the possible impact on the size of the
ever-growing CPython binaries. However, it turns out that this is a
case where readability and brevity are allies rather than enemies, so
we don't need to choose one or the other.

>
>
>> Of those, Nick's suggestion seems best.  It's slightly bigger than the
>> specialized bytecode format, but it's human-readable (and human-writable!),
>> and it'd be the easiest to implement.
>
> Plus, if it becomes the format how C level signatures are expressed anyway,
> it wouldn't require any additional build time preprocessing.
>
>
>> My first idea for implementation: add a "def x" to the front and ": pass"
>> to the end
>
> Why not require it to be there already? Maybe more like
>
>     def foo(arg, b=3, *, kwonly='a'):
>          ...
>
> (i.e. using Ellipsis instead of pass, so that it's clear that it's not an
> empty function but one the implementation of which is hidden)

I like this notion. The groups notation and '/' will still cause the
parser to choke and require special handling, but OTOH, they have
deliberately been chosen as potentially acceptable notations for
providing the same features in actual Python function declarations.

>
>
>> then run it through ast.parse.  Iterate over the tree,
>> converting parameters into inspect.Parameters and handling the return
>> annotation if present.  Default values and annotations would be turned into
>> values by ast.eval_literal.  (It wouldn't surprise me if there's a cleaner
>> way to do it than the fake function definition; I'm not familiar with the
>> ast module.)
>
> IMHO, if there is no straight forward way currently to convert a function
> header from a code blob into a Signature object in Python code, preferably
> using the ast module (either explicitly or implicitly through inspect.py),
> then that's a bug.

The complexity here is that Larry would like to limit the annotations
to compatibility with ast.literal_eval. If we drop that restriction,
then the inspect module could handle the task directly. Given the
complexity of implementing it, I believe the restriction needs more
justification than is currently included in the PEP.

>
>
>> We'd want one more mild hack: the DSL will support positional parameters,
>> and inspect.Signature supports positional parameters, so it'd be nice to
>> render that information.  But we can't represent that in Python syntax (or
>> at least not yet!), so we can't let ast.parse see it.  My suggestion: run
>> it through ast.parse, and if it throws a SyntaxError see if the problem was
>> a slash.  If it was, remove the slash, reprocess through ast.parse, and
>> remember that all parameters are positional-only (and barf if there are
>> kwonly, args, or kwargs).
>
> Is sounds simpler to me to just make it a Python syntax feature. Or at
> least an optional one, supported by the ast module with a dedicated
> compiler flag.

Agreed. Guido had previously decided "not worth the hassle", but this
may be enough to make him change his mind. Also, Larry's "simple"
solution here isn't enough, since it doesn't handle optional groups
correctly.

While the support still has some odd limitations under the covers, I
think an explicit compiler flag is a good compromise between a lot of
custom hacks and exposing an unfinished implementation of a new
language feature.

Cheers,
Nick.

>
> Stefan
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ncoghlan%40gmail.com



-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From storchaka at gmail.com  Tue Mar 19 08:37:35 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 Mar 2013 09:37:35 +0200
Subject: [Python-Dev] Rough idea for adding introspection information
	for builtins
In-Reply-To: <5147ED55.30605@hastings.org>
References: <5147ED55.30605@hastings.org>
Message-ID: <ki94k1$1sk$1@ger.gmane.org>

On 19.03.13 06:45, Larry Hastings wrote:
> 4. Store a string that looks like the Python declaration of the
> signature, and parse it (Nick's suggestion).  For foo above, this would
> be "(arg,b=3,*,kwonly='a')".  Length: 23 bytes.

Strip parenthesis and it will be only 21 bytes long.

> We'd want one more mild hack: the DSL will support positional
> parameters, and inspect.Signature supports positional parameters, so
> it'd be nice to render that information.  But we can't represent that in
> Python syntax (or at least not yet!), so we can't let ast.parse see it.
> My suggestion: run it through ast.parse, and if it throws a SyntaxError
> see if the problem was a slash.  If it was, remove the slash, reprocess
> through ast.parse, and remember that all parameters are positional-only
> (and barf if there are kwonly, args, or kwargs).

It will be simpler to use some one-character separator which shouldn't 
be used unquoted in the signature. I.e. LF.


From storchaka at gmail.com  Tue Mar 19 08:52:51 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 Mar 2013 09:52:51 +0200
Subject: [Python-Dev] Tarfile CLI
Message-ID: <ki95gl$a0f$1@ger.gmane.org>

There is a proposition to add a command line interface to tarfile 
module. It will be useful on platforms where tar is not included in the 
base system.

The question is about interface. Should it be a subset of tar options 
(-t as --extract, -t as --list, -f to specify an archive name) or be 
similar to the zipfile module interface (-e as --extract, -l as --list, 
the first positional parameter is an archive name)? There are different 
opinions.


From storchaka at gmail.com  Tue Mar 19 08:56:43 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 Mar 2013 09:56:43 +0200
Subject: [Python-Dev] Tarfile CLI
In-Reply-To: <ki95gl$a0f$1@ger.gmane.org>
References: <ki95gl$a0f$1@ger.gmane.org>
Message-ID: <ki95no$a0f$2@ger.gmane.org>

On 19.03.13 09:52, Serhiy Storchaka wrote:
> There is a proposition to add a command line interface to tarfile
> module.

Link: http://bugs.python.org/issue13477



From storchaka at gmail.com  Tue Mar 19 09:03:35 2013
From: storchaka at gmail.com (Serhiy Storchaka)
Date: Tue, 19 Mar 2013 10:03:35 +0200
Subject: [Python-Dev] Recent changes to TextIOWrapper and its tests
In-Reply-To: <5147785D.3070100@farowl.co.uk>
References: <5147785D.3070100@farowl.co.uk>
Message-ID: <ki964o$evp$1@ger.gmane.org>

On 18.03.13 22:26, Jeff Allen wrote:
> The puzzle is that it requires t.read() to succeed.
>
> When I insert a check for bytes type in all the places it seems
> necessary in my code, I pass the first two conditions, but since
> t.read() also raises TypeError, the overall test fails. Is reading the
> stream with read() intended to succeed? Why is this desired?

And alternative option is to change C implementation of 
TextIOWrapper.read() to raise an exception in this case. However I worry 
that it can break backward compatibility. Are there other tests (in 
other test files) which fail with a new Jython TextIOWrapper?



From larry at hastings.org  Tue Mar 19 10:24:53 2013
From: larry at hastings.org (Larry Hastings)
Date: Tue, 19 Mar 2013 02:24:53 -0700
Subject: [Python-Dev] Rough idea for adding introspection information
 for builtins
In-Reply-To: <ki94k1$1sk$1@ger.gmane.org>
References: <5147ED55.30605@hastings.org> <ki94k1$1sk$1@ger.gmane.org>
Message-ID: <51482EE5.3010605@hastings.org>


On 03/19/2013 12:37 AM, Serhiy Storchaka wrote:
> On 19.03.13 06:45, Larry Hastings wrote:
>> 4. Store a string that looks like the Python declaration of the
>> signature, and parse it (Nick's suggestion).  For foo above, this would
>> be "(arg,b=3,*,kwonly='a')".  Length: 23 bytes.
>
> Strip parenthesis and it will be only 21 bytes long.

I left the parentheses there because the return annotation is outside 
them.  If we strip the parentheses, I would have to restore them, and if 
there was a return annotation I would have to parse the string to know 
where to put it, because there could be arbitrary Python rvalues on 
either side of it with quotes and everything, and now I can no longer 
use ast.parse because it's not legal Python because the parentheses are 
missing ;-)

We could omit the /left/ parenthesis and save one byte per builtin.  I 
honestly don't know how many builtins there are, but my guess is one 
extra byte per builtin isn't a big deal.  Let's leave it in for 
readability's sakes.


>> We'd want one more mild hack: the DSL will support positional
>> parameters, and inspect.Signature supports positional parameters, so
>> it'd be nice to render that information.  But we can't represent that in
>> Python syntax (or at least not yet!), so we can't let ast.parse see it.
>> My suggestion: run it through ast.parse, and if it throws a SyntaxError
>> see if the problem was a slash.  If it was, remove the slash, reprocess
>> through ast.parse, and remember that all parameters are positional-only
>> (and barf if there are kwonly, args, or kwargs).
>
> It will be simpler to use some one-character separator which shouldn't 
> be used unquoted in the signature. I.e. LF.

I had trouble understanding what you're suggesting.  What I think you're 
saying is, "normally these generated strings won't have LF in them.  So 
let's use LF as a harmless extra character that means 'this is a 
positional-only signature'."

At one point Guido suggested / as syntax for exactly this case.  And 
while the LF approach is simpler programmatically, removing the slash 
and reparsing isn't terribly complicated; this part will be in Python, 
after all.  Meanwhile, I suggest that for human readability the slash is 
way more obvious--having a LF in the string mean this is awfully subtle.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130319/cd544da1/attachment.html>

From ronaldoussoren at mac.com  Tue Mar 19 10:42:16 2013
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 19 Mar 2013 10:42:16 +0100
Subject: [Python-Dev] Rough idea for adding introspection information
	for builtins
In-Reply-To: <51482EE5.3010605@hastings.org>
References: <5147ED55.30605@hastings.org> <ki94k1$1sk$1@ger.gmane.org>
	<51482EE5.3010605@hastings.org>
Message-ID: <4E35883F-9F1F-4B93-80E7-9C3019A650FF@mac.com>


On 19 Mar, 2013, at 10:24, Larry Hastings <larry at hastings.org> wrote:
> 
> 
>>> We'd want one more mild hack: the DSL will support positional 
>>> parameters, and inspect.Signature supports positional parameters, so 
>>> it'd be nice to render that information.  But we can't represent that in 
>>> Python syntax (or at least not yet!), so we can't let ast.parse see it. 
>>> My suggestion: run it through ast.parse, and if it throws a SyntaxError 
>>> see if the problem was a slash.  If it was, remove the slash, reprocess 
>>> through ast.parse, and remember that all parameters are positional-only 
>>> (and barf if there are kwonly, args, or kwargs). 
>> 
>> It will be simpler to use some one-character separator which shouldn't be used unquoted in the signature. I.e. LF.
> 
> I had trouble understanding what you're suggesting.  What I think you're saying is, "normally these generated strings won't have LF in them.  So let's use LF as a harmless extra character that means 'this is a positional-only signature'."
> 
> At one point Guido suggested / as syntax for exactly this case.  And while the LF approach is simpler programmatically, removing the slash and reparsing isn't terribly complicated; this part will be in Python, after all.  Meanwhile, I suggest that for human readability the slash is way more obvious--having a LF in the string mean this is awfully subtle.

You could also add the slash to the start of the signature, for example "/(arg1, arg2)", that way the positional only can be detected without trying to parse it first and removing a slash at the start is easier than removing it somewhere along a signature with arbitrary default values, such as "(arg1='/', arg2=4 /) -> 'arg1/arg2'".  The disadvantage is that you can't specify that only some of the arguments are positional-only, but that's not supported by PyArg_Parse... anyway.

Ronald


From larry at hastings.org  Tue Mar 19 11:00:45 2013
From: larry at hastings.org (Larry Hastings)
Date: Tue, 19 Mar 2013 03:00:45 -0700
Subject: [Python-Dev] Rough idea for adding introspection information
 for builtins
In-Reply-To: <CADiSq7d-BzzbpYSr4nnyPJyDcnigLWvV8X40V=Pd_onwBL4yYQ@mail.gmail.com>
References: <5147ED55.30605@hastings.org> <ki8vdd$ne3$1@ger.gmane.org>
	<CADiSq7d-BzzbpYSr4nnyPJyDcnigLWvV8X40V=Pd_onwBL4yYQ@mail.gmail.com>
Message-ID: <5148374D.6050402@hastings.org>



On 03/19/2013 12:23 AM, Nick Coghlan wrote:
> On Mon, Mar 18, 2013 at 11:08 PM, Stefan Behnel <stefan_ml at behnel.de> wrote:
>>> My first idea for implementation: add a "def x" to the front and ": pass"
>>> to the end
>> Why not require it to be there already? Maybe more like
>>
>>      def foo(arg, b=3, *, kwonly='a'):
>>           ...
>>
>> (i.e. using Ellipsis instead of pass, so that it's clear that it's not an
>> empty function but one the implementation of which is hidden)
> I like this notion. The groups notation and '/' will still cause the
> parser to choke and require special handling, but OTOH, they have
> deliberately been chosen as potentially acceptable notations for
> providing the same features in actual Python function declarations.

I don't see the benefit of including the "def foo" and ":\n ...".  The 
name doesn't help; inspect.Signature pointedly does /not/ contain the 
name of the function, so it's irrelevant to this purpose.  And why have 
unnecessary boilerplate?

And if I can go one further: what we're talking about is essentially a 
textual representation of a Signature object.  I assert that the stuff 
inside the parentheses, and the return annotation, *is* the signature.  
The name isn't part of the signature, and the colon and what lies 
afterwards is definitely not part of its signature.  So I think it's 
entirely appropriate, and a happy coincidence, that it happens to 
reflect the minimum amount of text you need to communicate the signature.


>> IMHO, if there is no straight forward way currently to convert a function
>> header from a code blob into a Signature object in Python code, preferably
>> using the ast module (either explicitly or implicitly through inspect.py),
>> then that's a bug.
> The complexity here is that Larry would like to limit the annotations
> to compatibility with ast.literal_eval. If we drop that restriction,
> then the inspect module could handle the task directly. Given the
> complexity of implementing it, I believe the restriction needs more
> justification than is currently included in the PEP.

I concede that it's totally unjustified in the PEP.  It's more playing a 
hunch at the moment, a combination of YAGNI and that it'd be hard to put 
the genie back in the bottle if we let people use arbitrary values.

Let me restate what we're talking about.  We're debating what types of 
data should be permissible to use for a datum that so far is not only 
unused, but is /required/ to be unused.  PEP 8 states " The Python 
standard library will not use function annotations".  I don't know who 
among us has any experience using function annotations--or, at least, 
for their intended purpose.  It's hard to debate what are reasonable vs 
unreasonable restrictions on data we might be permitted to specify in 
the future for uses we don't know about.  Restricting it to Python's 
rich set of safe literal values seems entirely reasonable; if we get 
there and need to relax the restriction, we can do so there.

Also, you and I discussed this evening whether there was a credible 
attack vector here.  I figured, if you're running an untrustworthy 
extension, it's already game over.  You suggested that a miscreant could 
easily edit static data on a trusted shared library without having to 
recompile it to achieve their naughtiness.  I'm not sure I necessarily 
buy it, I just wanted to point out you were the one making the case for 
restricting it to ast.literal_eval.  ;-)


>> Is sounds simpler to me to just make it a Python syntax feature. Or at
>> least an optional one, supported by the ast module with a dedicated
>> compiler flag.
> Agreed. Guido had previously decided "not worth the hassle", but this
> may be enough to make him change his mind. Also, Larry's "simple"
> solution here isn't enough, since it doesn't handle optional groups
> correctly.

I certainly don't agree that "remove the slash and reparse" is more 
complicated than "add a new parameter metaphor to the Python language".  
Adding support for it may be worth doing--don't ask me, I'm still 
nursing my "positional-only arguments are part of Python and forever 
will be" Kool-aid.  I'm just dealing with cold harsh reality as I 
understand it.

As for handling optional argument groups, my gut feeling is that we're 
better off not leaking it out of Argument Clinic--don't expose it in 
this string we're talking about, and don't add support for it in the 
inspect.Parameter object.  I'm not going to debate range(), the syntax 
of which predates one of our release managers.  But I suggest option 
groups are simply a misfeature of the curses module. There are some 
other possible uses in builtins (I forgot to dig those out this evening) 
but so far we're talking adding complexity to an array of technologies 
(this representation, the parser, the Parameter object) to support a 
handful of uses of something we shouldn't have done in the first place, 
for consumers who I think won't care and won't appreciate the added 
conceptual complexity.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130319/b6f41ee4/attachment.html>

From solipsis at pitrou.net  Tue Mar 19 11:13:00 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 19 Mar 2013 11:13:00 +0100
Subject: [Python-Dev] Rough idea for adding introspection information
	for builtins
References: <5147ED55.30605@hastings.org> <ki8vdd$ne3$1@ger.gmane.org>
	<CADiSq7d-BzzbpYSr4nnyPJyDcnigLWvV8X40V=Pd_onwBL4yYQ@mail.gmail.com>
	<5148374D.6050402@hastings.org>
Message-ID: <20130319111300.46b70aa3@pitrou.net>

Le Tue, 19 Mar 2013 03:00:45 -0700,
Larry Hastings <larry at hastings.org> a ?crit :
> 
> As for handling optional argument groups, my gut feeling is that
> we're better off not leaking it out of Argument Clinic--don't expose
> it in this string we're talking about, and don't add support for it
> in the inspect.Parameter object.  I'm not going to debate range(),
> the syntax of which predates one of our release managers.  But I
> suggest option groups are simply a misfeature of the curses module.
> There are some other possible uses in builtins (I forgot to dig those
> out this evening) but so far we're talking adding complexity to an
> array of technologies (this representation, the parser, the Parameter
> object) to support a handful of uses of something we shouldn't have
> done in the first place, for consumers who I think won't care and
> won't appreciate the added conceptual complexity.

Agreed with Larry.

Regards

Antoine.



From stefan at bytereef.org  Tue Mar 19 15:23:35 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Tue, 19 Mar 2013 15:23:35 +0100
Subject: [Python-Dev] [Python-checkins] peps: New DSL syntax and
 slightly changed semantics for the Argument Clinic DSL.
In-Reply-To: <CAP7+vJL4osXyUHdhi2=VjykkJngNFZw368t_-8CNnKeLPmBv7A@mail.gmail.com>
References: <3ZTBVD26DlzR9T@mail.python.org>
	<20130317222607.GA16540@sleipnir.bytereef.org>
	<5146BF68.8030903@hastings.org> <5146D4E5.1020302@hastings.org>
	<20130318103643.GA21550@sleipnir.bytereef.org>
	<CAP7+vJL4osXyUHdhi2=VjykkJngNFZw368t_-8CNnKeLPmBv7A@mail.gmail.com>
Message-ID: <20130319142334.GA3546@sleipnir.bytereef.org>

Guido van Rossum <guido at python.org> wrote:
> On Mon, Mar 18, 2013 at 3:36 AM, Stefan Krah <stefan at bytereef.org> wrote:
> > Larry Hastings <larry at hastings.org> wrote:
> >> * The DSL currently makes no provision for specifying per-parameter
> >>   type annotations.  This is something explicitly supported in Python;
> >>   it should be supported for builtins too, once we have reflection support.
> >>
> >>   It seems to me that the syntax for parameter lines--dictated by
> >>   Guido--suggests conversion functions are themselves type annotations.
> >>   This makes intuitive sense.
> >
> > Really, did you read PEP 437? It's all in there.
> 
> This attitude is unhelpful. Please stop being outright hostile. If you
> want to have any influence on the outcome at all, consider looking
> into compromises.

My apologies, I agree that wasn't very constructive.


In case there's a misunderstanding: This wasn't an attempt to push the
whole of PEP 437 again.  Type-specifying converters were first mentioned
in issue #16612 and are central to PEP 437, so my response should have
been something like "I think that's already covered in section X of ...".



Regarding compromises: I'm not at all after getting as many parts of
PEP 437 into the end result as possible.


Apparently PEP 437 as a whole is unacceptable, so if Larry's original PEP 436
turns out to be more coherent than the revised PEP 436, then I'd actually favor
Larry's original.


I'm getting the impression though that the proposal that Guido, Larry and
Nick worked out at PyCon *is* reasonably coherent.

So my position is the same as Nick's in

http://mail.python.org/pipermail/python-dev/2013-March/124757.html



Stefan Krah



From ncoghlan at gmail.com  Tue Mar 19 15:34:30 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Tue, 19 Mar 2013 07:34:30 -0700
Subject: [Python-Dev] Rough idea for adding introspection information
	for builtins
In-Reply-To: <5148374D.6050402@hastings.org>
References: <5147ED55.30605@hastings.org> <ki8vdd$ne3$1@ger.gmane.org>
	<CADiSq7d-BzzbpYSr4nnyPJyDcnigLWvV8X40V=Pd_onwBL4yYQ@mail.gmail.com>
	<5148374D.6050402@hastings.org>
Message-ID: <CADiSq7dvpOLkPWhGr89-kxuD693-XA9J7_s+4YV1sKJhs054SA@mail.gmail.com>

On Tue, Mar 19, 2013 at 3:00 AM, Larry Hastings <larry at hastings.org> wrote:
> Why not require it to be there already? Maybe more like
>
>     def foo(arg, b=3, *, kwonly='a'):
>          ...
>
> (i.e. using Ellipsis instead of pass, so that it's clear that it's not an
> empty function but one the implementation of which is hidden)
>
> I like this notion. The groups notation and '/' will still cause the
> parser to choke and require special handling, but OTOH, they have
> deliberately been chosen as potentially acceptable notations for
> providing the same features in actual Python function declarations.
>
>
> I don't see the benefit of including the "def foo" and ":\n    ...".  The
> name doesn't help; inspect.Signature pointedly does not contain the name of
> the function, so it's irrelevant to this purpose.  And why have unnecessary
> boilerplate?

Also, we can already easily produce the extended form through:

    "def {}{}:\n    ...".format(f.__name__, inspect.signature(f))

So, agreed, capturing just the signature info is fine.

> Let me restate what we're talking about.  We're debating what types of data
> should be permissible to use for a datum that so far is not only unused, but
> is required to be unused.  PEP 8 states " The Python standard library will
> not use function annotations".  I don't know who among us has any experience
> using function annotations--or, at least, for their intended purpose.  It's
> hard to debate what are reasonable vs unreasonable restrictions on data we
> might be permitted to specify in the future for uses we don't know about.
> Restricting it to Python's rich set of safe literal values seems entirely
> reasonable; if we get there and need to relax the restriction, we can do so
> there.
>
> Also, you and I discussed this evening whether there was a credible attack
> vector here.  I figured, if you're running an untrustworthy extension, it's
> already game over.  You suggested that a miscreant could easily edit static
> data on a trusted shared library without having to recompile it to achieve
> their naughtiness.  I'm not sure I necessarily buy it, I just wanted to
> point out you were the one making the case for restricting it to
> ast.literal_eval.  ;-)

IIRC, I was arguing against allowing *pickle* because you can't audit
that just by looking at the generated source code. OTOH, I'm a big fan
of locking this kind of thing down by default and letting people make
the case for additional permissiveness, so I agree it's best to start
with literals only.

Here's a thought, though: instead of doing an Argument Clinic specific
hack, let's instead design a proper whitelist API for ast.literal_eval
that lets you accept additional constructs.

As a general sketch, the long if/elif chain in ast.literal_eval could
be replaced by:

    for converter in converters:
        ok, converted = converter(node)
        if ok:
            return converted
    raise ValueError('malformed node or string: ' + repr(node))

The _convert function would need to be lifted out and made public as
"ast.convert_node", so conversion functions could recurse
appropriately.

Both ast.literal_eval and ast.convert_node would accept a keyword-only
"allow" parameter that accepted an iterable of callables that return a
2-tuple to whitelist additional expressions beyond those normally
allowed. So, assuming we don't add it by default, you could allow
empty sets by doing:

    _empty_set = ast.dump(ast.parse("set()").body[0].value)
    def convert_empty_set(node):
        if ast.dump(node) == _empty_set:
            return True, set()
        return False, None

    ast.literal_eval(some_str, allow=[convert_empy_set])

This is quite powerful as a general tool to allow constrained
execution, since it could be used to whitelist builtins that accept
parameters, as well as to process class and function header lines
without executing their bodies. In the case of Argument Clinic, that
would mean writing a converter for the FunctionDef node.

> I certainly don't agree that "remove the slash and reparse" is more
> complicated than "add a new parameter metaphor to the Python language".
> Adding support for it may be worth doing--don't ask me, I'm still nursing my
> "positional-only arguments are part of Python and forever will be" Kool-aid.
> I'm just dealing with cold harsh reality as I understand it.
>
> As for handling optional argument groups, my gut feeling is that we're
> better off not leaking it out of Argument Clinic--don't expose it in this
> string we're talking about, and don't add support for it in the
> inspect.Parameter object.  I'm not going to debate range(), the syntax of
> which predates one of our release managers.  But I suggest option groups are
> simply a misfeature of the curses module.  There are some other possible
> uses in builtins (I forgot to dig those out this evening) but so far we're
> talking adding complexity to an array of technologies (this representation,
> the parser, the Parameter object) to support a handful of uses of something
> we shouldn't have done in the first place, for consumers who I think won't
> care and won't appreciate the added conceptual complexity.

Agreed on both points, but this should be articulated in the PEP.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From barry at python.org  Tue Mar 19 18:01:06 2013
From: barry at python.org (Barry Warsaw)
Date: Tue, 19 Mar 2013 10:01:06 -0700
Subject: [Python-Dev] Rough idea for adding introspection information
 for builtins
In-Reply-To: <5147ED55.30605@hastings.org>
References: <5147ED55.30605@hastings.org>
Message-ID: <20130319100106.0b32527e@anarchist>

On Mar 18, 2013, at 09:45 PM, Larry Hastings wrote:

>4. Store a string that looks like the Python declaration of the signature,
>and parse it (Nick's suggestion).  For foo above, this would be
>"(arg,b=3,*,kwonly='a')".  Length: 23 bytes.

Very nice.

-Barry

From senthil at uthcode.com  Tue Mar 19 20:17:20 2013
From: senthil at uthcode.com (Senthil Kumaran)
Date: Tue, 19 Mar 2013 12:17:20 -0700
Subject: [Python-Dev] [Python-checkins] cpython:
	../bug-fixes/http_error_interface/.hg/last-message.txt
In-Reply-To: <3ZVkG913khzS5n@mail.python.org>
References: <3ZVkG913khzS5n@mail.python.org>
Message-ID: <CAPOVWOTaHu9V9i2GMjbEU9Eox-iOzQjRB3kmLjMxc1DZAb80EQ@mail.gmail.com>

Looks like I used hg commit -m /path/to/.hg/last-message.txt instead
of hg commit -l /path/to/.hg/last-message.txt
I have amended it, merged it and pushed it again.

On Tue, Mar 19, 2013 at 12:04 PM, senthil.kumaran
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/4f2080e9eee2
> changeset:   82765:4f2080e9eee2
> parent:      82763:4c6463b96a2c
> user:        Senthil Kumaran <senthil at uthcode.com>
> date:        Tue Mar 19 12:07:43 2013 -0700
> summary:
>   ../bug-fixes/http_error_interface/.hg/last-message.txt
>
> files:
>   Lib/test/test_urllib2.py |  39 +++++++++++++--------------
>   1 files changed, 19 insertions(+), 20 deletions(-)
>
>
> diff --git a/Lib/test/test_urllib2.py b/Lib/test/test_urllib2.py
> --- a/Lib/test/test_urllib2.py
> +++ b/Lib/test/test_urllib2.py
> @@ -1387,6 +1387,10 @@
>
>  class MiscTests(unittest.TestCase):
>
> +    def opener_has_handler(self, opener, handler_class):
> +        self.assertTrue(any(h.__class__ == handler_class
> +                            for h in opener.handlers))
> +
>      def test_build_opener(self):
>          class MyHTTPHandler(urllib.request.HTTPHandler): pass
>          class FooHandler(urllib.request.BaseHandler):
> @@ -1439,10 +1443,22 @@
>          self.assertEqual(b"1234567890", request.data)
>          self.assertEqual("10", request.get_header("Content-length"))
>
> +    def test_HTTPError_interface(self):
> +        """
> +        Issue 13211 reveals that HTTPError didn't implement the URLError
> +        interface even though HTTPError is a subclass of URLError.
>
> -    def opener_has_handler(self, opener, handler_class):
> -        self.assertTrue(any(h.__class__ == handler_class
> -                            for h in opener.handlers))
> +        >>> msg = 'something bad happened'
> +        >>> url = code = fp = None
> +        >>> hdrs = 'Content-Length: 42'
> +        >>> err = urllib.error.HTTPError(url, code, msg, hdrs, fp)
> +        >>> assert hasattr(err, 'reason')
> +        >>> err.reason
> +        'something bad happened'
> +        >>> assert hasattr(err, 'headers')
> +        >>> err.headers
> +        'Content-Length: 42'
> +        """
>
>  class RequestTests(unittest.TestCase):
>
> @@ -1514,23 +1530,6 @@
>          req = Request(url)
>          self.assertEqual(req.get_full_url(), url)
>
> -def test_HTTPError_interface():
> -    """
> -    Issue 13211 reveals that HTTPError didn't implement the URLError
> -    interface even though HTTPError is a subclass of URLError.
> -
> -    >>> msg = 'something bad happened'
> -    >>> url = code = fp = None
> -    >>> hdrs = 'Content-Length: 42'
> -    >>> err = urllib.error.HTTPError(url, code, msg, hdrs, fp)
> -    >>> assert hasattr(err, 'reason')
> -    >>> err.reason
> -    'something bad happened'
> -    >>> assert hasattr(err, 'headers')
> -    >>> err.headers
> -    'Content-Length: 42'
> -    """
> -
>  def test_main(verbose=None):
>      from test import test_urllib2
>      support.run_doctest(test_urllib2, verbose)
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>

From kristjan at ccpgames.com  Tue Mar 19 20:34:21 2013
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Tue, 19 Mar 2013 19:34:21 +0000
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 :
 Error code 10035 calling socket.recv() on a socket with a timeout
In-Reply-To: <CAPOVWORAoGDrWbtBpq37Lxh9qfzPpEfp+x1sbwBUzpirNQka8g@mail.gmail.com>
References: <3ZVj1911wHzSrl@mail.python.org>
	<CAPOVWORAoGDrWbtBpq37Lxh9qfzPpEfp+x1sbwBUzpirNQka8g@mail.gmail.com>
Message-ID: <EFE3877620384242A686D52278B7CCD353818FDA@RKV-IT-EXCH104.ccp.ad.local>

Yes, it is a symbol problem on unix.  Working on it.

-----Original Message-----
From: Python-checkins [mailto:python-checkins-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Senthil Kumaran
Sent: 19. mars 2013 12:28
To: sweskman at gmail.com
Cc: python-checkins at python.org
Subject: Re: [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout

Looks like RHEL 2.7 buildbots are unhappy with this change.

--
Senthil

On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/8ec39bfd1f01
> changeset:   82764:8ec39bfd1f01
> branch:      2.7
> parent:      82740:b10ec5083a53
> user:        Kristj?n Valur J?nsson <sweskman at gmail.com>
> date:        Tue Mar 19 10:58:59 2013 -0700
> summary:
>   Issue #9090 : Error code 10035 calling socket.recv() on a socket 
> with a timeout  (WSAEWOULDBLOCK - A non-blocking socket operation 
> could not be completed
>  immediately)
>
> files:
>   Misc/NEWS              |    5 +
>   Modules/socketmodule.c |  104 ++++++++++++++++++++++++----
>   Modules/timemodule.c   |    7 +
>   3 files changed, 101 insertions(+), 15 deletions(-)
>
>
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -214,6 +214,11 @@
>  Library
>  -------
>
> +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or 
> +EAGAIN,
> +  retry the select() loop instead of bailing out.  This is because 
> +select()
> +  can incorrectly report a socket as ready for reading (for example, 
> +if it
> +  received some data with an invalid checksum).
> +
>  - Issue #1285086: Get rid of the refcounting hack and speed up urllib.unquote().
>
>  - Issue #17368: Fix an off-by-one error in the Python JSON decoder 
> that caused diff --git a/Modules/socketmodule.c 
> b/Modules/socketmodule.c
> --- a/Modules/socketmodule.c
> +++ b/Modules/socketmodule.c
> @@ -473,6 +473,17 @@
>      return NULL;
>  }
>
> +#ifdef MS_WINDOWS
> +#ifndef WSAEAGAIN
> +#define WSAEAGAIN WSAEWOULDBLOCK
> +#endif
> +#define CHECK_ERRNO(expected) \
> +    (WSAGetLastError() == WSA ## expected) #else #define 
> +CHECK_ERRNO(expected) \
> +    (errno == expected)
> +#endif
> +
>  /* Convenience function to raise an error according to errno
>     and return a NULL pointer from a function. */
>
> @@ -661,7 +672,7 @@
>     after they've reacquired the interpreter lock.
>     Returns 1 on timeout, -1 on error, 0 otherwise. */  static int 
> -internal_select(PySocketSockObject *s, int writing)
> +internal_select_ex(PySocketSockObject *s, int writing, double 
> +interval)
>  {
>      int n;
>
> @@ -673,6 +684,10 @@
>      if (s->sock_fd < 0)
>          return 0;
>
> +    /* Handling this condition here simplifies the select loops */
> +    if (interval < 0.0)
> +        return 1;
> +
>      /* Prefer poll, if available, since you can poll() any fd
>       * which can't be done with select(). */  #ifdef HAVE_POLL @@ 
> -684,7 +699,7 @@
>          pollfd.events = writing ? POLLOUT : POLLIN;
>
>          /* s->sock_timeout is in seconds, timeout in ms */
> -        timeout = (int)(s->sock_timeout * 1000 + 0.5);
> +        timeout = (int)(interval * 1000 + 0.5);
>          n = poll(&pollfd, 1, timeout);
>      }
>  #else
> @@ -692,8 +707,8 @@
>          /* Construct the arguments to select */
>          fd_set fds;
>          struct timeval tv;
> -        tv.tv_sec = (int)s->sock_timeout;
> -        tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6);
> +        tv.tv_sec = (int)interval;
> +        tv.tv_usec = (int)((interval - tv.tv_sec) * 1e6);
>          FD_ZERO(&fds);
>          FD_SET(s->sock_fd, &fds);
>
> @@ -712,6 +727,49 @@
>      return 0;
>  }
>
> +static int
> +internal_select(PySocketSockObject *s, int writing) {
> +    return internal_select_ex(s, writing, s->sock_timeout); }
> +
> +/*
> +   Two macros for automatic retry of select() in case of false positives
> +   (for example, select() could indicate a socket is ready for reading
> +    but the data then discarded by the OS because of a wrong checksum).
> +   Here is an example of use:
> +
> +    BEGIN_SELECT_LOOP(s)
> +    Py_BEGIN_ALLOW_THREADS
> +    timeout = internal_select_ex(s, 0, interval);
> +    if (!timeout)
> +        outlen = recv(s->sock_fd, cbuf, len, flags);
> +    Py_END_ALLOW_THREADS
> +    if (timeout == 1) {
> +        PyErr_SetString(socket_timeout, "timed out");
> +        return -1;
> +    }
> +    END_SELECT_LOOP(s)
> +*/
> +PyAPI_FUNC(double) _PyTime_floattime(void); /* defined in timemodule.c */
> +#define BEGIN_SELECT_LOOP(s) \
> +    { \
> +        double deadline, interval = s->sock_timeout; \
> +        int has_timeout = s->sock_timeout > 0.0; \
> +        if (has_timeout) { \
> +            deadline = _PyTime_floattime() + s->sock_timeout; \
> +        } \
> +        while (1) { \
> +            errno = 0; \
> +
> +#define END_SELECT_LOOP(s) \
> +            if (!has_timeout || \
> +                (!CHECK_ERRNO(EWOULDBLOCK) && !CHECK_ERRNO(EAGAIN))) \
> +                break; \
> +            interval = deadline - _PyTime_floattime(); \
> +        } \
> +    } \
> +
>  /* Initialize a new socket object. */
>
>  static double defaulttimeout = -1.0; /* Default timeout for new sockets */
> @@ -1656,8 +1714,9 @@
>      if (!IS_SELECTABLE(s))
>          return select_error();
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 0);
> +    timeout = internal_select_ex(s, 0, interval);
>      if (!timeout)
>          newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen);
>      Py_END_ALLOW_THREADS
> @@ -1666,6 +1725,7 @@
>          PyErr_SetString(socket_timeout, "timed out");
>          return NULL;
>      }
> +    END_SELECT_LOOP(s)
>
>  #ifdef MS_WINDOWS
>      if (newfd == INVALID_SOCKET)
> @@ -2355,8 +2415,9 @@
>      }
>
>  #ifndef __VMS
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 0);
> +    timeout = internal_select_ex(s, 0, interval);
>      if (!timeout)
>          outlen = recv(s->sock_fd, cbuf, len, flags);
>      Py_END_ALLOW_THREADS
> @@ -2365,6 +2426,7 @@
>          PyErr_SetString(socket_timeout, "timed out");
>          return -1;
>      }
> +    END_SELECT_LOOP(s)
>      if (outlen < 0) {
>          /* Note: the call to errorhandler() ALWAYS indirectly returned
>             NULL, so ignore its return value */
> @@ -2386,8 +2448,9 @@
>              segment = remaining;
>          }
>
> +        BEGIN_SELECT_LOOP(s)
>          Py_BEGIN_ALLOW_THREADS
> -        timeout = internal_select(s, 0);
> +        timeout = internal_select_ex(s, 0, interval);
>          if (!timeout)
>              nread = recv(s->sock_fd, read_buf, segment, flags);
>          Py_END_ALLOW_THREADS
> @@ -2396,6 +2459,8 @@
>              PyErr_SetString(socket_timeout, "timed out");
>              return -1;
>          }
> +        END_SELECT_LOOP(s)
> +
>          if (nread < 0) {
>              s->errorhandler();
>              return -1;
> @@ -2559,9 +2624,10 @@
>          return -1;
>      }
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
>      memset(&addrbuf, 0, addrlen);
> -    timeout = internal_select(s, 0);
> +    timeout = internal_select_ex(s, 0, interval);
>      if (!timeout) {
>  #ifndef MS_WINDOWS
>  #if defined(PYOS_OS2) && !defined(PYCC_GCC)
> @@ -2582,6 +2648,7 @@
>          PyErr_SetString(socket_timeout, "timed out");
>          return -1;
>      }
> +    END_SELECT_LOOP(s)
>      if (n < 0) {
>          s->errorhandler();
>          return -1;
> @@ -2719,8 +2786,9 @@
>      buf = pbuf.buf;
>      len = pbuf.len;
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 1);
> +    timeout = internal_select_ex(s, 1, interval);
>      if (!timeout)
>  #ifdef __VMS
>          n = sendsegmented(s->sock_fd, buf, len, flags);
> @@ -2728,13 +2796,14 @@
>          n = send(s->sock_fd, buf, len, flags);
>  #endif
>      Py_END_ALLOW_THREADS
> -
> -    PyBuffer_Release(&pbuf);
> -
>      if (timeout == 1) {
> +        PyBuffer_Release(&pbuf);
>          PyErr_SetString(socket_timeout, "timed out");
>          return NULL;
>      }
> +    END_SELECT_LOOP(s)
> +
> +    PyBuffer_Release(&pbuf);
>      if (n < 0)
>          return s->errorhandler();
>      return PyInt_FromLong((long)n);
> @@ -2768,8 +2837,9 @@
>      }
>
>      do {
> +        BEGIN_SELECT_LOOP(s)
>          Py_BEGIN_ALLOW_THREADS
> -        timeout = internal_select(s, 1);
> +        timeout = internal_select_ex(s, 1, interval);
>          n = -1;
>          if (!timeout) {
>  #ifdef __VMS
> @@ -2784,6 +2854,7 @@
>              PyErr_SetString(socket_timeout, "timed out");
>              return NULL;
>          }
> +        END_SELECT_LOOP(s)
>          /* PyErr_CheckSignals() might change errno */
>          saved_errno = errno;
>          /* We must run our signal handlers before looping again.
> @@ -2863,17 +2934,20 @@
>          return NULL;
>      }
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 1);
> +    timeout = internal_select_ex(s, 1, interval);
>      if (!timeout)
>          n = sendto(s->sock_fd, buf, len, flags, SAS2SA(&addrbuf), addrlen);
>      Py_END_ALLOW_THREADS
>
> -    PyBuffer_Release(&pbuf);
>      if (timeout == 1) {
> +        PyBuffer_Release(&pbuf);
>          PyErr_SetString(socket_timeout, "timed out");
>          return NULL;
>      }
> +    END_SELECT_LOOP(s)
> +    PyBuffer_Release(&pbuf);
>      if (n < 0)
>          return s->errorhandler();
>      return PyInt_FromLong((long)n);
> diff --git a/Modules/timemodule.c b/Modules/timemodule.c
> --- a/Modules/timemodule.c
> +++ b/Modules/timemodule.c
> @@ -1055,3 +1055,10 @@
>
>      return 0;
>  }
> +
> +/* export floattime to socketmodule.c */
> +PyAPI_FUNC(double)
> +_PyTime_floattime(void)
> +{
> +    return floattime();
> +}
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins

From kristjan at ccpgames.com  Tue Mar 19 20:49:16 2013
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Tue, 19 Mar 2013 19:49:16 +0000
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 :
 Error code 10035 calling socket.recv() on a socket with a timeout
In-Reply-To: <EFE3877620384242A686D52278B7CCD353818FDA@RKV-IT-EXCH104.ccp.ad.local>
References: <3ZVj1911wHzSrl@mail.python.org>
	<CAPOVWORAoGDrWbtBpq37Lxh9qfzPpEfp+x1sbwBUzpirNQka8g@mail.gmail.com>
	<EFE3877620384242A686D52278B7CCD353818FDA@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <EFE3877620384242A686D52278B7CCD353819072@RKV-IT-EXCH104.ccp.ad.local>

Apparently timemodule is not a built-in module on linux.  But it is on windows.  Funny!

-----Original Message-----
From: Python-Dev [mailto:python-dev-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Kristj?n Valur J?nsson
Sent: 19. mars 2013 12:34
To: python-dev at python.org
Subject: Re: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout

Yes, it is a symbol problem on unix.  Working on it.

-----Original Message-----
From: Python-checkins [mailto:python-checkins-bounces+kristjan=ccpgames.com at python.org] On Behalf Of Senthil Kumaran
Sent: 19. mars 2013 12:28
To: sweskman at gmail.com
Cc: python-checkins at python.org
Subject: Re: [Python-checkins] cpython (2.7): Issue #9090 : Error code 10035 calling socket.recv() on a socket with a timeout

Looks like RHEL 2.7 buildbots are unhappy with this change.

--
Senthil

On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/8ec39bfd1f01
> changeset:   82764:8ec39bfd1f01
> branch:      2.7
> parent:      82740:b10ec5083a53
> user:        Kristj?n Valur J?nsson <sweskman at gmail.com>
> date:        Tue Mar 19 10:58:59 2013 -0700
> summary:
>   Issue #9090 : Error code 10035 calling socket.recv() on a socket 
> with a timeout  (WSAEWOULDBLOCK - A non-blocking socket operation 
> could not be completed
>  immediately)
>
> files:
>   Misc/NEWS              |    5 +
>   Modules/socketmodule.c |  104 ++++++++++++++++++++++++----
>   Modules/timemodule.c   |    7 +
>   3 files changed, 101 insertions(+), 15 deletions(-)
>
>
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -214,6 +214,11 @@
>  Library
>  -------
>
> +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or 
> +EAGAIN,
> +  retry the select() loop instead of bailing out.  This is because
> +select()
> +  can incorrectly report a socket as ready for reading (for example, 
> +if it
> +  received some data with an invalid checksum).
> +
>  - Issue #1285086: Get rid of the refcounting hack and speed up urllib.unquote().
>
>  - Issue #17368: Fix an off-by-one error in the Python JSON decoder 
> that caused diff --git a/Modules/socketmodule.c 
> b/Modules/socketmodule.c
> --- a/Modules/socketmodule.c
> +++ b/Modules/socketmodule.c
> @@ -473,6 +473,17 @@
>      return NULL;
>  }
>
> +#ifdef MS_WINDOWS
> +#ifndef WSAEAGAIN
> +#define WSAEAGAIN WSAEWOULDBLOCK
> +#endif
> +#define CHECK_ERRNO(expected) \
> +    (WSAGetLastError() == WSA ## expected) #else #define
> +CHECK_ERRNO(expected) \
> +    (errno == expected)
> +#endif
> +
>  /* Convenience function to raise an error according to errno
>     and return a NULL pointer from a function. */
>
> @@ -661,7 +672,7 @@
>     after they've reacquired the interpreter lock.
>     Returns 1 on timeout, -1 on error, 0 otherwise. */  static int 
> -internal_select(PySocketSockObject *s, int writing)
> +internal_select_ex(PySocketSockObject *s, int writing, double
> +interval)
>  {
>      int n;
>
> @@ -673,6 +684,10 @@
>      if (s->sock_fd < 0)
>          return 0;
>
> +    /* Handling this condition here simplifies the select loops */
> +    if (interval < 0.0)
> +        return 1;
> +
>      /* Prefer poll, if available, since you can poll() any fd
>       * which can't be done with select(). */  #ifdef HAVE_POLL @@
> -684,7 +699,7 @@
>          pollfd.events = writing ? POLLOUT : POLLIN;
>
>          /* s->sock_timeout is in seconds, timeout in ms */
> -        timeout = (int)(s->sock_timeout * 1000 + 0.5);
> +        timeout = (int)(interval * 1000 + 0.5);
>          n = poll(&pollfd, 1, timeout);
>      }
>  #else
> @@ -692,8 +707,8 @@
>          /* Construct the arguments to select */
>          fd_set fds;
>          struct timeval tv;
> -        tv.tv_sec = (int)s->sock_timeout;
> -        tv.tv_usec = (int)((s->sock_timeout - tv.tv_sec) * 1e6);
> +        tv.tv_sec = (int)interval;
> +        tv.tv_usec = (int)((interval - tv.tv_sec) * 1e6);
>          FD_ZERO(&fds);
>          FD_SET(s->sock_fd, &fds);
>
> @@ -712,6 +727,49 @@
>      return 0;
>  }
>
> +static int
> +internal_select(PySocketSockObject *s, int writing) {
> +    return internal_select_ex(s, writing, s->sock_timeout); }
> +
> +/*
> +   Two macros for automatic retry of select() in case of false positives
> +   (for example, select() could indicate a socket is ready for reading
> +    but the data then discarded by the OS because of a wrong checksum).
> +   Here is an example of use:
> +
> +    BEGIN_SELECT_LOOP(s)
> +    Py_BEGIN_ALLOW_THREADS
> +    timeout = internal_select_ex(s, 0, interval);
> +    if (!timeout)
> +        outlen = recv(s->sock_fd, cbuf, len, flags);
> +    Py_END_ALLOW_THREADS
> +    if (timeout == 1) {
> +        PyErr_SetString(socket_timeout, "timed out");
> +        return -1;
> +    }
> +    END_SELECT_LOOP(s)
> +*/
> +PyAPI_FUNC(double) _PyTime_floattime(void); /* defined in 
> +timemodule.c */ #define BEGIN_SELECT_LOOP(s) \
> +    { \
> +        double deadline, interval = s->sock_timeout; \
> +        int has_timeout = s->sock_timeout > 0.0; \
> +        if (has_timeout) { \
> +            deadline = _PyTime_floattime() + s->sock_timeout; \
> +        } \
> +        while (1) { \
> +            errno = 0; \
> +
> +#define END_SELECT_LOOP(s) \
> +            if (!has_timeout || \
> +                (!CHECK_ERRNO(EWOULDBLOCK) && !CHECK_ERRNO(EAGAIN))) \
> +                break; \
> +            interval = deadline - _PyTime_floattime(); \
> +        } \
> +    } \
> +
>  /* Initialize a new socket object. */
>
>  static double defaulttimeout = -1.0; /* Default timeout for new 
> sockets */ @@ -1656,8 +1714,9 @@
>      if (!IS_SELECTABLE(s))
>          return select_error();
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 0);
> +    timeout = internal_select_ex(s, 0, interval);
>      if (!timeout)
>          newfd = accept(s->sock_fd, SAS2SA(&addrbuf), &addrlen);
>      Py_END_ALLOW_THREADS
> @@ -1666,6 +1725,7 @@
>          PyErr_SetString(socket_timeout, "timed out");
>          return NULL;
>      }
> +    END_SELECT_LOOP(s)
>
>  #ifdef MS_WINDOWS
>      if (newfd == INVALID_SOCKET)
> @@ -2355,8 +2415,9 @@
>      }
>
>  #ifndef __VMS
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 0);
> +    timeout = internal_select_ex(s, 0, interval);
>      if (!timeout)
>          outlen = recv(s->sock_fd, cbuf, len, flags);
>      Py_END_ALLOW_THREADS
> @@ -2365,6 +2426,7 @@
>          PyErr_SetString(socket_timeout, "timed out");
>          return -1;
>      }
> +    END_SELECT_LOOP(s)
>      if (outlen < 0) {
>          /* Note: the call to errorhandler() ALWAYS indirectly returned
>             NULL, so ignore its return value */ @@ -2386,8 +2448,9 @@
>              segment = remaining;
>          }
>
> +        BEGIN_SELECT_LOOP(s)
>          Py_BEGIN_ALLOW_THREADS
> -        timeout = internal_select(s, 0);
> +        timeout = internal_select_ex(s, 0, interval);
>          if (!timeout)
>              nread = recv(s->sock_fd, read_buf, segment, flags);
>          Py_END_ALLOW_THREADS
> @@ -2396,6 +2459,8 @@
>              PyErr_SetString(socket_timeout, "timed out");
>              return -1;
>          }
> +        END_SELECT_LOOP(s)
> +
>          if (nread < 0) {
>              s->errorhandler();
>              return -1;
> @@ -2559,9 +2624,10 @@
>          return -1;
>      }
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
>      memset(&addrbuf, 0, addrlen);
> -    timeout = internal_select(s, 0);
> +    timeout = internal_select_ex(s, 0, interval);
>      if (!timeout) {
>  #ifndef MS_WINDOWS
>  #if defined(PYOS_OS2) && !defined(PYCC_GCC) @@ -2582,6 +2648,7 @@
>          PyErr_SetString(socket_timeout, "timed out");
>          return -1;
>      }
> +    END_SELECT_LOOP(s)
>      if (n < 0) {
>          s->errorhandler();
>          return -1;
> @@ -2719,8 +2786,9 @@
>      buf = pbuf.buf;
>      len = pbuf.len;
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 1);
> +    timeout = internal_select_ex(s, 1, interval);
>      if (!timeout)
>  #ifdef __VMS
>          n = sendsegmented(s->sock_fd, buf, len, flags); @@ -2728,13 
> +2796,14 @@
>          n = send(s->sock_fd, buf, len, flags);  #endif
>      Py_END_ALLOW_THREADS
> -
> -    PyBuffer_Release(&pbuf);
> -
>      if (timeout == 1) {
> +        PyBuffer_Release(&pbuf);
>          PyErr_SetString(socket_timeout, "timed out");
>          return NULL;
>      }
> +    END_SELECT_LOOP(s)
> +
> +    PyBuffer_Release(&pbuf);
>      if (n < 0)
>          return s->errorhandler();
>      return PyInt_FromLong((long)n);
> @@ -2768,8 +2837,9 @@
>      }
>
>      do {
> +        BEGIN_SELECT_LOOP(s)
>          Py_BEGIN_ALLOW_THREADS
> -        timeout = internal_select(s, 1);
> +        timeout = internal_select_ex(s, 1, interval);
>          n = -1;
>          if (!timeout) {
>  #ifdef __VMS
> @@ -2784,6 +2854,7 @@
>              PyErr_SetString(socket_timeout, "timed out");
>              return NULL;
>          }
> +        END_SELECT_LOOP(s)
>          /* PyErr_CheckSignals() might change errno */
>          saved_errno = errno;
>          /* We must run our signal handlers before looping again.
> @@ -2863,17 +2934,20 @@
>          return NULL;
>      }
>
> +    BEGIN_SELECT_LOOP(s)
>      Py_BEGIN_ALLOW_THREADS
> -    timeout = internal_select(s, 1);
> +    timeout = internal_select_ex(s, 1, interval);
>      if (!timeout)
>          n = sendto(s->sock_fd, buf, len, flags, SAS2SA(&addrbuf), addrlen);
>      Py_END_ALLOW_THREADS
>
> -    PyBuffer_Release(&pbuf);
>      if (timeout == 1) {
> +        PyBuffer_Release(&pbuf);
>          PyErr_SetString(socket_timeout, "timed out");
>          return NULL;
>      }
> +    END_SELECT_LOOP(s)
> +    PyBuffer_Release(&pbuf);
>      if (n < 0)
>          return s->errorhandler();
>      return PyInt_FromLong((long)n);
> diff --git a/Modules/timemodule.c b/Modules/timemodule.c
> --- a/Modules/timemodule.c
> +++ b/Modules/timemodule.c
> @@ -1055,3 +1055,10 @@
>
>      return 0;
>  }
> +
> +/* export floattime to socketmodule.c */
> +PyAPI_FUNC(double)
> +_PyTime_floattime(void)
> +{
> +    return floattime();
> +}
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
_______________________________________________
Python-checkins mailing list
Python-checkins at python.org
http://mail.python.org/mailman/listinfo/python-checkins
_______________________________________________
Python-Dev mailing list
Python-Dev at python.org
http://mail.python.org/mailman/listinfo/python-dev
Unsubscribe: http://mail.python.org/mailman/options/python-dev/kristjan%40ccpgames.com

From guido at python.org  Tue Mar 19 20:55:53 2013
From: guido at python.org (Guido van Rossum)
Date: Tue, 19 Mar 2013 12:55:53 -0700
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue #9090 :
 Error code 10035 calling socket.recv() on a socket with a timeout
In-Reply-To: <3ZVj1911wHzSrl@mail.python.org>
References: <3ZVj1911wHzSrl@mail.python.org>
Message-ID: <CAP7+vJLorg6g8XSHo9wKOZE0z6dbRir=4c9kqEx3L6nj9Og=-A@mail.gmail.com>

On Tue, Mar 19, 2013 at 11:08 AM, kristjan.jonsson <
python-checkins at python.org> wrote:

> http://hg.python.org/cpython/rev/8ec39bfd1f01
> changeset:   82764:8ec39bfd1f01
> branch:      2.7
> parent:      82740:b10ec5083a53
> user:        Kristj?n Valur J?nsson <sweskman at gmail.com>
> date:        Tue Mar 19 10:58:59 2013 -0700
> summary:
>   Issue #9090 : Error code 10035 calling socket.recv() on a socket with a
> timeout
>  (WSAEWOULDBLOCK - A non-blocking socket operation could not be completed
>  immediately)
> [...]
> +- Issue #9090: When a socket with a timeout fails with EWOULDBLOCK or
> EAGAIN,
> +  retry the select() loop instead of bailing out.  This is because
> select()
> +  can incorrectly report a socket as ready for reading (for example, if it
> +  received some data with an invalid checksum).
>

Might I recommend treating EINTR the same way? It has the same issue of
popping up, rarely, when you least expect it, and messing with your code.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130319/29e55b78/attachment.html>

From larry at hastings.org  Wed Mar 20 01:14:07 2013
From: larry at hastings.org (Larry Hastings)
Date: Tue, 19 Mar 2013 17:14:07 -0700
Subject: [Python-Dev] Early results from Argument Clinic automation
	discussion
Message-ID: <5148FF4F.8020704@hastings.org>


Mark Shannon, Dmitry Jemerov (from PyCharm) and I sat down to talk about 
rearchitecting the Argument Clinic prototype to make it easily to 
interact with.  We came up with the following.

The DSL will now produce an intermediate representation.  The output 
will consume this intermediate representation.

The two defined outputs from the IR so far:

 1. The inplace generated C code (what it has right now), and
 2. Argument Clinic DSL code itself.


The intermediate representation will be subclasses of inspect.Signature 
and inspect.Parameter that add the extra bits of information needed by 
Argument Clinic, as follows:

class Function(inspect.Signature):
         name = 'function name'
         module = 'module name if any
         class = 'class name if any, can't actually call this class'
         docstring = 'function docstring'
         c_id = 'stub id to use for generated c functions'
         return_converter = SeeBelow()

class Parameter(inspect.Parameter):
          docstring = 'per-parameter docstring'
          group = <group membership integer or None>
          converter = ConverterFunction()

Parameter.group is an integer, specifying which "option group" the 
parameter is in.  This must be an integer for positional-only arguments, 
and None for other argument types.  0 indicates "required 
positional-only parameter".  Left-optional groups get negative numbers, 
decreasing as they get further from the required parameters; 
right-optional get positive numbers, increasing as they get further from 
the required parameters.  (Groups cannot nest, so a parameter cannot be 
in more than one group.)

Function.return_converter was suggested by Mark.  This is the inverse of 
the per-parameter converter function: it defines the return type of the 
impl function, and the conversion process to turn it into the PyObject * 
that gets returned to Python.  And, like the converter functions, it 
will define the actual return annotation of the function (if any).

Mark wants to write something that parses C code implementing builtins 
and produces the IR; he can then use that to write Argument Clinic DSL.  
This will make the conversion process go much more quickly.


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130319/60ab46b4/attachment.html>

From solipsis at pitrou.net  Wed Mar 20 01:26:33 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Mar 2013 01:26:33 +0100
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
 readlines support to
References: <3ZVsKZ6MKPzPF3@mail.python.org>
Message-ID: <20130320012633.57bcb451@pitrou.net>

On Wed, 20 Mar 2013 01:22:58 +0100 (CET)
michael.foord <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/684b75600fa9
> changeset:   82811:684b75600fa9
> user:        Michael Foord <michael at voidspace.org.uk>
> date:        Tue Mar 19 17:22:51 2013 -0700
> summary:
>   Closes issue 17467. Add readline and readlines support to unittest.mock.mock_open

Wasn't it possible to re-use an existing implementation (such as
TextIOBase or StringIO) rather than re-write your own?

(it's not even obvious your implementation is correct, BTW. How about
universal newlines?)

Regards

Antoine.



From ezio.melotti at gmail.com  Wed Mar 20 02:31:05 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Wed, 20 Mar 2013 03:31:05 +0200
Subject: [Python-Dev] [Python-checkins] cpython: Issue #17385: Fix
 quadratic behavior in threading.Condition
In-Reply-To: <CACBhJdE+HBf1ik9EF7rFFLupMxhad24kkXi1a6Gu4RfAmSKsjg@mail.gmail.com>
References: <3ZPLXL0jVYzS7C@mail.python.org>
	<CACBhJdE+HBf1ik9EF7rFFLupMxhad24kkXi1a6Gu4RfAmSKsjg@mail.gmail.com>
Message-ID: <CACBhJdGn-R6G-Ee+4SAOsdcMBG431z6nGb4o73qmQKZny4e6EQ@mail.gmail.com>

On Mon, Mar 11, 2013 at 3:14 AM, Ezio Melotti <ezio.melotti at gmail.com> wrote:
> Hi,
>
> On Mon, Mar 11, 2013 at 2:58 AM, raymond.hettinger
> <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/0f86b51f8f8b
>> changeset:   82592:0f86b51f8f8b
>> user:        Raymond Hettinger <python at rcn.com>
>> date:        Sun Mar 10 17:57:28 2013 -0700
>> summary:
>>   Issue #17385: Fix quadratic behavior in threading.Condition
>>
>> files:
>>   Lib/threading.py |  10 ++++++++--
>>   Misc/NEWS        |   3 +++
>>   2 files changed, 11 insertions(+), 2 deletions(-)
>>
>>
>> diff --git a/Lib/threading.py b/Lib/threading.py
>> --- a/Lib/threading.py
>> +++ b/Lib/threading.py
>> @@ -10,6 +10,12 @@
>>      from time import time as _time
>>  from traceback import format_exc as _format_exc
>>  from _weakrefset import WeakSet
>> +try:
>> +    from _itertools import islice as _slice
>> +    from _collections import deque as _deque
>> +except ImportError:
>> +    from itertools import islice as _islice
>> +    from collections import deque as _deque
>>
>
> Shouldn't the one in the 'try' be _islice too?
>

Also I don't seem to have an _itertools module.  Is this something
used by the other VMs?

> Best Regards,
> Ezio Melotti

From kristjan at ccpgames.com  Wed Mar 20 04:16:53 2013
From: kristjan at ccpgames.com (=?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?=)
Date: Wed, 20 Mar 2013 03:16:53 +0000
Subject: [Python-Dev] [Python-checkins] cpython: #15927: Fix cvs.reader
 parsing of escaped \r\n with quoting off.
In-Reply-To: <3ZVwQ43t3lzSHt@mail.python.org>
References: <3ZVwQ43t3lzSHt@mail.python.org>
Message-ID: <EFE3877620384242A686D52278B7CCD35381A13E@RKV-IT-EXCH104.ccp.ad.local>

The compiler complains about this line:
if (c == '\n' | c=='\r') {

Perhaps you wanted a Boolean operator?

-----Original Message-----
From: Python-checkins [mailto:python-checkins-bounces+kristjan=ccpgames.com at python.org] On Behalf Of r.david.murray
Sent: 19. mars 2013 19:42
To: python-checkins at python.org
Subject: [Python-checkins] cpython: #15927: Fix cvs.reader parsing of escaped \r\n with quoting off.

http://hg.python.org/cpython/rev/940748853712
changeset:   82815:940748853712
parent:      82811:684b75600fa9
user:        R David Murray <rdmurray at bitdance.com>
date:        Tue Mar 19 22:41:47 2013 -0400
summary:
  #15927: Fix cvs.reader parsing of escaped \r\n with quoting off.

This fix means that such values are correctly roundtripped, since cvs.writer already does the correct escaping.

Patch by Michael Johnson.

files:
  Lib/test/test_csv.py |   9 +++++++++
  Misc/ACKS            |   1 +
  Misc/NEWS            |   3 +++
  Modules/_csv.c       |  13 ++++++++++++-
  4 files changed, 25 insertions(+), 1 deletions(-)


diff --git a/Lib/test/test_csv.py b/Lib/test/test_csv.py
--- a/Lib/test/test_csv.py
+++ b/Lib/test/test_csv.py
@@ -308,6 +308,15 @@
             for i, row in enumerate(csv.reader(fileobj)):
                 self.assertEqual(row, rows[i])
 
+    def test_roundtrip_escaped_unquoted_newlines(self):
+        with TemporaryFile("w+", newline='') as fileobj:
+            writer = csv.writer(fileobj,quoting=csv.QUOTE_NONE,escapechar="\\")
+            rows = [['a\nb','b'],['c','x\r\nd']]
+            writer.writerows(rows)
+            fileobj.seek(0)
+            for i, row in enumerate(csv.reader(fileobj,quoting=csv.QUOTE_NONE,escapechar="\\")):
+                self.assertEqual(row,rows[i])
+
 class TestDialectRegistry(unittest.TestCase):
     def test_registry_badargs(self):
         self.assertRaises(TypeError, csv.list_dialects, None) diff --git a/Misc/ACKS b/Misc/ACKS
--- a/Misc/ACKS
+++ b/Misc/ACKS
@@ -591,6 +591,7 @@
 Fredrik Johansson
 Gregory K. Johnson
 Kent Johnson
+Michael Johnson
 Simon Johnston
 Matt Joiner
 Thomas Jollans
diff --git a/Misc/NEWS b/Misc/NEWS
--- a/Misc/NEWS
+++ b/Misc/NEWS
@@ -289,6 +289,9 @@
 Library
 -------
 
+- Issue #15927: CVS now correctly parses escaped newlines and carriage
+  when parsing with quoting turned off.
+
 - Issue #17467: add readline and readlines support to mock_open in
   unittest.mock.
 
diff --git a/Modules/_csv.c b/Modules/_csv.c
--- a/Modules/_csv.c
+++ b/Modules/_csv.c
@@ -51,7 +51,7 @@
 typedef enum {
     START_RECORD, START_FIELD, ESCAPED_CHAR, IN_FIELD,
     IN_QUOTED_FIELD, ESCAPE_IN_QUOTED_FIELD, QUOTE_IN_QUOTED_FIELD,
-    EAT_CRNL
+    EAT_CRNL,AFTER_ESCAPED_CRNL
 } ParserState;
 
 typedef enum {
@@ -644,6 +644,12 @@
         break;
 
     case ESCAPED_CHAR:
+        if (c == '\n' | c=='\r') {
+            if (parse_add_char(self, c) < 0)
+                return -1;
+            self->state = AFTER_ESCAPED_CRNL;
+            break;
+        }
         if (c == '\0')
             c = '\n';
         if (parse_add_char(self, c) < 0) @@ -651,6 +657,11 @@
         self->state = IN_FIELD;
         break;
 
+    case AFTER_ESCAPED_CRNL:
+        if (c == '\0')
+            break;
+        /*fallthru*/
+
     case IN_FIELD:
         /* in unquoted field */
         if (c == '\n' || c == '\r' || c == '\0') {

--
Repository URL: http://hg.python.org/cpython

From rdmurray at bitdance.com  Wed Mar 20 04:59:25 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 19 Mar 2013 23:59:25 -0400
Subject: [Python-Dev] [Python-checkins] cpython: #15927: Fix cvs.reader
	parsing of escaped \r\n with quoting off.
In-Reply-To: <EFE3877620384242A686D52278B7CCD35381A13E@RKV-IT-EXCH104.ccp.ad.local>
References: <3ZVwQ43t3lzSHt@mail.python.org>
	<EFE3877620384242A686D52278B7CCD35381A13E@RKV-IT-EXCH104.ccp.ad.local>
Message-ID: <20130320035926.75D5A2500B3@webabinitio.net>

On Wed, 20 Mar 2013 03:16:53 -0000, =?utf-8?B?S3Jpc3Rqw6FuIFZhbHVyIErDs25zc29u?= <kristjan at ccpgames.com> wrote:
> The compiler complains about this line:
> if (c == '\n' | c=='\r') {
> 
> Perhaps you wanted a Boolean operator?

Indeed, yes.

--David

From andrew.svetlov at gmail.com  Wed Mar 20 05:25:00 2013
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Tue, 19 Mar 2013 21:25:00 -0700
Subject: [Python-Dev] [Python-checkins] peps: update 2.7.4 release dates
In-Reply-To: <3ZVyV50rrYzSFs@mail.python.org>
References: <3ZVyV50rrYzSFs@mail.python.org>
Message-ID: <CAL3CFcXk3Fq6AjyZMUQj=di=6Um1vkQpLG5u7mGq=6R2NyJvFQ@mail.gmail.com>

Are you sure about 2.7.4 2012-04-06? I mean 2012 year.

On Tue, Mar 19, 2013 at 9:15 PM, benjamin.peterson
<python-checkins at python.org> wrote:
> http://hg.python.org/peps/rev/ce17779c395c
> changeset:   4810:ce17779c395c
> user:        Benjamin Peterson <benjamin at python.org>
> date:        Tue Mar 19 23:15:23 2013 -0500
> summary:
>   update 2.7.4 release dates
>
> files:
>   pep-0373.txt |  4 ++--
>   1 files changed, 2 insertions(+), 2 deletions(-)
>
>
> diff --git a/pep-0373.txt b/pep-0373.txt
> --- a/pep-0373.txt
> +++ b/pep-0373.txt
> @@ -56,8 +56,8 @@
>
>  Planned future release dates:
>
> -- 2.7.4rc1 2013-02-02
> -- 2.7.4 2012-02-16
> +- 2.7.4rc1 2013-03-23
> +- 2.7.4 2012-04-06
>
>  Dates of previous maintenance releases:
>
>
> --
> Repository URL: http://hg.python.org/peps
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>



-- 
Thanks,
Andrew Svetlov

From benjamin at python.org  Wed Mar 20 05:30:33 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 19 Mar 2013 21:30:33 -0700
Subject: [Python-Dev] [Python-checkins] peps: update 2.7.4 release dates
In-Reply-To: <CAL3CFcXk3Fq6AjyZMUQj=di=6Um1vkQpLG5u7mGq=6R2NyJvFQ@mail.gmail.com>
References: <3ZVyV50rrYzSFs@mail.python.org>
	<CAL3CFcXk3Fq6AjyZMUQj=di=6Um1vkQpLG5u7mGq=6R2NyJvFQ@mail.gmail.com>
Message-ID: <CAPZV6o8L31uwPJHK9K_YOYwUtt6uRJs1BaYV1Jhe7kB1J-K8ZA@mail.gmail.com>

Good catch.

2013/3/19 Andrew Svetlov <andrew.svetlov at gmail.com>:
> Are you sure about 2.7.4 2012-04-06? I mean 2012 year.
>
> On Tue, Mar 19, 2013 at 9:15 PM, benjamin.peterson
> <python-checkins at python.org> wrote:
>> http://hg.python.org/peps/rev/ce17779c395c
>> changeset:   4810:ce17779c395c
>> user:        Benjamin Peterson <benjamin at python.org>
>> date:        Tue Mar 19 23:15:23 2013 -0500
>> summary:
>>   update 2.7.4 release dates
>>
>> files:
>>   pep-0373.txt |  4 ++--
>>   1 files changed, 2 insertions(+), 2 deletions(-)
>>
>>
>> diff --git a/pep-0373.txt b/pep-0373.txt
>> --- a/pep-0373.txt
>> +++ b/pep-0373.txt
>> @@ -56,8 +56,8 @@
>>
>>  Planned future release dates:
>>
>> -- 2.7.4rc1 2013-02-02
>> -- 2.7.4 2012-02-16
>> +- 2.7.4rc1 2013-03-23
>> +- 2.7.4 2012-04-06
>>
>>  Dates of previous maintenance releases:
>>
>>
>> --
>> Repository URL: http://hg.python.org/peps
>>
>> _______________________________________________
>> Python-checkins mailing list
>> Python-checkins at python.org
>> http://mail.python.org/mailman/listinfo/python-checkins
>>
>
>
>
> --
> Thanks,
> Andrew Svetlov
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins



-- 
Regards,
Benjamin

From fuzzyman at voidspace.org.uk  Wed Mar 20 05:44:15 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 19 Mar 2013 21:44:15 -0700
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
	readlines support to
In-Reply-To: <20130320012633.57bcb451@pitrou.net>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
Message-ID: <F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>


On 19 Mar 2013, at 17:26, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Wed, 20 Mar 2013 01:22:58 +0100 (CET)
> michael.foord <python-checkins at python.org> wrote:
>> http://hg.python.org/cpython/rev/684b75600fa9
>> changeset:   82811:684b75600fa9
>> user:        Michael Foord <michael at voidspace.org.uk>
>> date:        Tue Mar 19 17:22:51 2013 -0700
>> summary:
>>  Closes issue 17467. Add readline and readlines support to unittest.mock.mock_open
> 
> Wasn't it possible to re-use an existing implementation (such as
> TextIOBase or StringIO) rather than re-write your own?
> 
> (it's not even obvious your implementation is correct, BTW. How about
> universal newlines?)

mock_open makes it easy to put a StringIO in place if that's what you want. It's just a simple helper function for providing some known data *along with the Mock api* to make asserts that it was used correctly. It isn't presenting a full file-system. My suggestion to the implementor of the patch was that read / readline / readlines be disconnected - but the patch provided allows them to be interleaved and I saw no reason to undo that.

If users want more complex behaviour (like universal newline support) they can use mock_open along with a StringIO.

Michael


> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk


--
http://www.voidspace.org.uk/


May you do good and not evil
May you find forgiveness for yourself and forgive others
May you share freely, never taking more than you give.
-- the sqlite blessing 
http://www.sqlite.org/different.html






From greg at krypto.org  Wed Mar 20 07:09:22 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Tue, 19 Mar 2013 23:09:22 -0700
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
 readlines support to
In-Reply-To: <F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
	<F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
Message-ID: <CAGE7PNJ_2gVe3YnQ4Nc8QTf1Fmdh0i6bPJ3z+vZ1HTqXjQzXKw@mail.gmail.com>

On Tue, Mar 19, 2013 at 9:44 PM, Michael Foord <fuzzyman at voidspace.org.uk>wrote:

>
> On 19 Mar 2013, at 17:26, Antoine Pitrou <solipsis at pitrou.net> wrote:
>
> > On Wed, 20 Mar 2013 01:22:58 +0100 (CET)
> > michael.foord <python-checkins at python.org> wrote:
> >> http://hg.python.org/cpython/rev/684b75600fa9
> >> changeset:   82811:684b75600fa9
> >> user:        Michael Foord <michael at voidspace.org.uk>
> >> date:        Tue Mar 19 17:22:51 2013 -0700
> >> summary:
> >>  Closes issue 17467. Add readline and readlines support to
> unittest.mock.mock_open
> >
> > Wasn't it possible to re-use an existing implementation (such as
> > TextIOBase or StringIO) rather than re-write your own?
> >
> > (it's not even obvious your implementation is correct, BTW. How about
> > universal newlines?)
>
> mock_open makes it easy to put a StringIO in place if that's what you
> want. It's just a simple helper function for providing some known data
> *along with the Mock api* to make asserts that it was used correctly. It
> isn't presenting a full file-system. My suggestion to the implementor of
> the patch was that read / readline / readlines be disconnected - but the
> patch provided allows them to be interleaved and I saw no reason to undo
> that.
>
> If users want more complex behaviour (like universal newline support) they
> can use mock_open along with a StringIO.
>

It'd be good to mention that in the unittest.mock.rst docs.


>
> Michael
>
>
> >
> > Regards
> >
> > Antoine.
> >
> >
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>
> --
> http://www.voidspace.org.uk/
>
>
> May you do good and not evil
> May you find forgiveness for yourself and forgive others
> May you share freely, never taking more than you give.
> -- the sqlite blessing
> http://www.sqlite.org/different.html
>
>
>
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130319/c0b042e8/attachment-0001.html>

From solipsis at pitrou.net  Wed Mar 20 08:09:29 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Mar 2013 08:09:29 +0100
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
 readlines support to
In-Reply-To: <F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
	<F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
Message-ID: <20130320080929.2cdbab56@pitrou.net>

On Tue, 19 Mar 2013 21:44:15 -0700
Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> 
> mock_open makes it easy to put a StringIO in place if that's what you want. It's just a simple helper function for providing some known data *along with the Mock api* to make asserts that it was used correctly. It isn't presenting a full file-system. My suggestion to the implementor of the patch was that read / readline / readlines be disconnected - but the patch provided allows them to be interleaved and I saw no reason to undo that.
> 
> If users want more complex behaviour (like universal newline support) they can use mock_open along with a StringIO.

This is not about complex behaviour but simply correct behaviour.
For the record, universal newlines are enabled by default in Python 3:

>>> with open("foo", "wb") as f: f.write(b"a\r\nb\rc\n")
... 
7
>>> with open("foo", "r") as f: print(list(f))
... 
['a\n', 'b\n', 'c\n']


Regards

Antoine.

From fuzzyman at voidspace.org.uk  Wed Mar 20 08:50:27 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 20 Mar 2013 00:50:27 -0700
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
	readlines support to
In-Reply-To: <20130320080929.2cdbab56@pitrou.net>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
	<F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
	<20130320080929.2cdbab56@pitrou.net>
Message-ID: <A4FC3146-8003-4883-A31D-388E4340BBD8@voidspace.org.uk>

On 20 Mar 2013, at 00:09, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Tue, 19 Mar 2013 21:44:15 -0700
> Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> 
>> mock_open makes it easy to put a StringIO in place if that's what you want. It's just a simple helper function for providing some known data *along with the Mock api* to make asserts that it was used correctly. It isn't presenting a full file-system. My suggestion to the implementor of the patch was that read / readline / readlines be disconnected - but the patch provided allows them to be interleaved and I saw no reason to undo that.
>> 
>> If users want more complex behaviour (like universal newline support) they can use mock_open along with a StringIO.
> 
> This is not about complex behaviour but simply correct behaviour.
> For the record, universal newlines are enabled by default in Python 3:
> 
>>>> with open("foo", "wb") as f: f.write(b"a\r\nb\rc\n")
> ... 
> 7
>>>> with open("foo", "r") as f: print(list(f))
> ... 
> ['a\n', 'b\n', 'c\n']
> 

mock_open is ?not? presenting a mock filesystem, but is about providing a mock object to avoid ?either? reading or writing to the real filesystem. You don't ?tend? to do both with a single file handle  - I know it's possible but mock_open is a convenience function for the common case.

This commit simply adds support for readline and readlines, whereas before only read was supported.

If you want to add support for additional functionality feel free to propose a patch.

Michael



> 
> Regards
> 
> Antoine.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk

From solipsis at pitrou.net  Wed Mar 20 09:03:20 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Mar 2013 09:03:20 +0100
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
 readlines support to
In-Reply-To: <A4FC3146-8003-4883-A31D-388E4340BBD8@voidspace.org.uk>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
	<F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
	<20130320080929.2cdbab56@pitrou.net>
	<A4FC3146-8003-4883-A31D-388E4340BBD8@voidspace.org.uk>
Message-ID: <20130320090320.6c5a22be@pitrou.net>

On Wed, 20 Mar 2013 00:50:27 -0700
Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> 
> If you want to add support for additional functionality feel free to propose a patch.

This isn't about additional functionality, this is about
correctness. You don't want to write multiple, slightly different,
implementations of readline() and friends (which is what Python 2 did).

Regards

Antoine.

From fuzzyman at voidspace.org.uk  Wed Mar 20 09:14:36 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 20 Mar 2013 01:14:36 -0700
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
	readlines support to
In-Reply-To: <20130320090320.6c5a22be@pitrou.net>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
	<F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
	<20130320080929.2cdbab56@pitrou.net>
	<A4FC3146-8003-4883-A31D-388E4340BBD8@voidspace.org.uk>
	<20130320090320.6c5a22be@pitrou.net>
Message-ID: <59A3CDC7-2D7E-4F28-AB14-01023144F637@voidspace.org.uk>



On 20 Mar 2013, at 01:03, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Wed, 20 Mar 2013 00:50:27 -0700
> Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> 
>> If you want to add support for additional functionality feel free to propose a patch.
> 
> This isn't about additional functionality, this is about
> correctness. You don't want to write multiple, slightly different,
> implementations of readline() and friends (which is what Python 2 did).
> 


This change allows you to set a series of return values for readlines when mocking open.

We are not reading data from anywhere the user is supplying it pre-canned.

Do you have a specific problem with it? 

Michael

> Regards
> 
> Antoine.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk

From fuzzyman at voidspace.org.uk  Wed Mar 20 10:14:46 2013
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Wed, 20 Mar 2013 02:14:46 -0700
Subject: [Python-Dev] cpython: Closes issue 17467. Add readline and
	readlines support to
In-Reply-To: <CAGE7PNJ_2gVe3YnQ4Nc8QTf1Fmdh0i6bPJ3z+vZ1HTqXjQzXKw@mail.gmail.com>
References: <3ZVsKZ6MKPzPF3@mail.python.org>
	<20130320012633.57bcb451@pitrou.net>
	<F4F47918-1356-4975-A4EE-63879EF2FC15@voidspace.org.uk>
	<CAGE7PNJ_2gVe3YnQ4Nc8QTf1Fmdh0i6bPJ3z+vZ1HTqXjQzXKw@mail.gmail.com>
Message-ID: <E52A7D0F-73CC-431E-BBED-423D4B1AC089@voidspace.org.uk>




On 19 Mar 2013, at 23:09, "Gregory P. Smith" <greg at krypto.org> wrote:

> 
> On Tue, Mar 19, 2013 at 9:44 PM, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>> 
>> On 19 Mar 2013, at 17:26, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> 
>> > On Wed, 20 Mar 2013 01:22:58 +0100 (CET)
>> > michael.foord <python-checkins at python.org> wrote:
>> >> http://hg.python.org/cpython/rev/684b75600fa9
>> >> changeset:   82811:684b75600fa9
>> >> user:        Michael Foord <michael at voidspace.org.uk>
>> >> date:        Tue Mar 19 17:22:51 2013 -0700
>> >> summary:
>> >>  Closes issue 17467. Add readline and readlines support to unittest.mock.mock_open
>> >
>> > Wasn't it possible to re-use an existing implementation (such as
>> > TextIOBase or StringIO) rather than re-write your own?
>> >
>> > (it's not even obvious your implementation is correct, BTW. How about
>> > universal newlines?)
>> 
>> mock_open makes it easy to put a StringIO in place if that's what you want. It's just a simple helper function for providing some known data *along with the Mock api* to make asserts that it was used correctly. It isn't presenting a full file-system. My suggestion to the implementor of the patch was that read / readline / readlines be disconnected - but the patch provided allows them to be interleaved and I saw no reason to undo that.
>> 
>> If users want more complex behaviour (like universal newline support) they can use mock_open along with a StringIO.
> 
> It'd be good to mention that in the unittest.mock.rst docs.
>  

I'll look at clarifying the intent and limitations of mock_open in the docs - plus an example of using it with a StringIO.

It maybe that the support for interleaving of read and readline (etc) is  just unnecessary and setting them separately is enough (which simplifies the implementation). I know Toshio had a specific use case needing readline support so I'll check with him.

Michael



>> 
>> Michael
>> 
>> 
>> >
>> > Regards
>> >
>> > Antoine.
>> >
>> >
>> > _______________________________________________
>> > Python-Dev mailing list
>> > Python-Dev at python.org
>> > http://mail.python.org/mailman/listinfo/python-dev
>> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>> 
>> 
>> --
>> http://www.voidspace.org.uk/
>> 
>> 
>> May you do good and not evil
>> May you find forgiveness for yourself and forgive others
>> May you share freely, never taking more than you give.
>> -- the sqlite blessing
>> http://www.sqlite.org/different.html
>> 
>> 
>> 
>> 
>> 
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/greg%40krypto.org
> 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/b7f1f369/attachment.html>

From eliben at gmail.com  Wed Mar 20 13:23:43 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 05:23:43 -0700
Subject: [Python-Dev] [Python-checkins] cpython: Issue #13248: removed
 deprecated and undocumented difflib.isbjunk, isbpopular.
In-Reply-To: <3ZVrTH23v4zPCm@mail.python.org>
References: <3ZVrTH23v4zPCm@mail.python.org>
Message-ID: <CAF-Rda_-rXSs2VVZM6eht=ZP2aq2rgqNj+M4q7Zw4h7Lu-hsew@mail.gmail.com>

A mention in Misc/NEWS can't hurt here, Terry. Even though it's
undocumented, some old code could rely on it being there and this code will
break with the transition to 3.4

Eli


On Tue, Mar 19, 2013 at 4:44 PM, terry.reedy <python-checkins at python.org>wrote:

> http://hg.python.org/cpython/rev/612d8bbcfa3a
> changeset:   82807:612d8bbcfa3a
> user:        Terry Jan Reedy <tjreedy at udel.edu>
> date:        Tue Mar 19 19:44:04 2013 -0400
> summary:
>   Issue #13248: removed deprecated and undocumented difflib.isbjunk,
> isbpopular.
>
> files:
>   Lib/difflib.py |  14 --------------
>   1 files changed, 0 insertions(+), 14 deletions(-)
>
>
> diff --git a/Lib/difflib.py b/Lib/difflib.py
> --- a/Lib/difflib.py
> +++ b/Lib/difflib.py
> @@ -336,20 +336,6 @@
>              for elt in popular: # ditto; as fast for 1% deletion
>                  del b2j[elt]
>
> -    def isbjunk(self, item):
> -        "Deprecated; use 'item in SequenceMatcher().bjunk'."
> -        warnings.warn("'SequenceMatcher().isbjunk(item)' is deprecated;\n"
> -                      "use 'item in SMinstance.bjunk' instead.",
> -                      DeprecationWarning, 2)
> -        return item in self.bjunk
> -
> -    def isbpopular(self, item):
> -        "Deprecated; use 'item in SequenceMatcher().bpopular'."
> -        warnings.warn("'SequenceMatcher().isbpopular(item)' is
> deprecated;\n"
> -                      "use 'item in SMinstance.bpopular' instead.",
> -                      DeprecationWarning, 2)
> -        return item in self.bpopular
> -
>      def find_longest_match(self, alo, ahi, blo, bhi):
>          """Find longest matching block in a[alo:ahi] and b[blo:bhi].
>
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/028af08b/attachment.html>

From eliben at gmail.com  Wed Mar 20 17:41:53 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 09:41:53 -0700
Subject: [Python-Dev] IDLE in the stdlib
Message-ID: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>

Interesting writeup about PyCon 2013 young coder education:
http://therealkatie.net/blog/2013/mar/19/pycon-2013-young-coders/

Quote:

"We used IDLE because it's already on Raspian's desktop. Personally, I like
IDLE as a teaching tool. It's included in the standard library, it does tab
completion and color coding, and it even has a text editor included so you
don't have to start your class off by teaching everyone about paths.

Too bad it's broke as hell."

Personally, I think that IDLE reflects badly on Python in more ways than
one. It's badly maintained, quirky and ugly. It serves a very narrow set of
uses, and does it badly.

Being part of Python *distributions* and being part of core Python standard
library are two different things. The former may make sense, the latter
IMHO makes no sense whatsoever. Outside the Python core IDLE can be
maintained more freely, with less restrictions on contributors and
hopefully become a better tool.


Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/865e1ae5/attachment.html>

From rovitotv at gmail.com  Wed Mar 20 18:11:18 2013
From: rovitotv at gmail.com (Todd Rovito)
Date: Wed, 20 Mar 2013 13:11:18 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
Message-ID: <CALZ1apxih96xYuLw_RQ0L+NWhZSAtDUzkAyTqegzCVLy54TL5Q@mail.gmail.com>

On Wed, Mar 20, 2013 at 12:41 PM, Eli Bendersky <eliben at gmail.com> wrote:
> Interesting writeup about PyCon 2013 young coder
> education:http://therealkatie.net/blog/2013/mar/19/pycon-2013-young-coders/
>
> Quote:
>
> "We used IDLE because it's already on Raspian's desktop. Personally, I like
> IDLE as a teaching tool. It's included in the standard library, it does tab
> completion and color coding, and it even has a text editor included so you
> don't have to start your class off by teaching everyone about paths.
>
> Too bad it's broke as hell."
>
> Personally, I think that IDLE reflects badly on Python in more ways than
> one. It's badly maintained, quirky and ugly. It serves a very narrow set of
> uses, and does it badly.
>
> Being part of Python *distributions* and being part of core Python standard
> library are two different things. The former may make sense, the latter IMHO
> makes no sense whatsoever. Outside the Python core IDLE can be maintained
> more freely, with less restrictions on contributors and hopefully become a
> better tool.
Eli,
   Thanks for sharing that article it was a fun read.  I think the
next paragraph from the article is important as well:
"I believe my first contribution to the Python Standard Library will
be fixes to IDLE. I really do like it that much. Happily, the kids
were flexible. If they needed to do a workaround, or ignore something
on our slides (they were written with the standard shell in mind),
they did so. They were total champs. My adult students would have been
much more upset."

Having an IDE that ships with Python is powerful and follows Python's
mantra "batteries included".  Personally I think removing IDLE from
the Python Standard Library is a mistake.  IDLE helps the novice get
started as demonstrated by this article.   What is frustrating is many
patches already exist for IDLE in the bug tracker they simply have not
been committed.  PEP-434 (http://www.python.org/dev/peps/pep-0434/) is
designed to make it easier to get these patches committed.  I would
ask that you give PEP-434 some time and let the process work before we
start a in-depth discussion on if IDLE should stay or go.

From eliben at gmail.com  Wed Mar 20 18:16:31 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 10:16:31 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CALZ1apxih96xYuLw_RQ0L+NWhZSAtDUzkAyTqegzCVLy54TL5Q@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<CALZ1apxih96xYuLw_RQ0L+NWhZSAtDUzkAyTqegzCVLy54TL5Q@mail.gmail.com>
Message-ID: <CAF-Rda_s0TrMswA9yp7KdeHDK4shQvnwf+DL1++ZZxGoMDrdXg@mail.gmail.com>

On Wed, Mar 20, 2013 at 10:11 AM, Todd Rovito <rovitotv at gmail.com> wrote:

> On Wed, Mar 20, 2013 at 12:41 PM, Eli Bendersky <eliben at gmail.com> wrote:
> > Interesting writeup about PyCon 2013 young coder
> > education:
> http://therealkatie.net/blog/2013/mar/19/pycon-2013-young-coders/
> >
> > Quote:
> >
> > "We used IDLE because it's already on Raspian's desktop. Personally, I
> like
> > IDLE as a teaching tool. It's included in the standard library, it does
> tab
> > completion and color coding, and it even has a text editor included so
> you
> > don't have to start your class off by teaching everyone about paths.
> >
> > Too bad it's broke as hell."
> >
> > Personally, I think that IDLE reflects badly on Python in more ways than
> > one. It's badly maintained, quirky and ugly. It serves a very narrow set
> of
> > uses, and does it badly.
> >
> > Being part of Python *distributions* and being part of core Python
> standard
> > library are two different things. The former may make sense, the latter
> IMHO
> > makes no sense whatsoever. Outside the Python core IDLE can be maintained
> > more freely, with less restrictions on contributors and hopefully become
> a
> > better tool.
> Eli,
>    Thanks for sharing that article it was a fun read.  I think the
> next paragraph from the article is important as well:
> "I believe my first contribution to the Python Standard Library will
> be fixes to IDLE. I really do like it that much. Happily, the kids
> were flexible. If they needed to do a workaround, or ignore something
> on our slides (they were written with the standard shell in mind),
> they did so. They were total champs. My adult students would have been
> much more upset."
>
> Having an IDE that ships with Python is powerful and follows Python's
> mantra "batteries included".  Personally I think removing IDLE from
> the Python Standard Library is a mistake.  IDLE helps the novice get
> started as demonstrated by this article.   What is frustrating is many
> patches already exist for IDLE in the bug tracker they simply have not
> been committed.  PEP-434 (http://www.python.org/dev/peps/pep-0434/) is
> designed to make it easier to get these patches committed.  I would
> ask that you give PEP-434 some time and let the process work before we
> start a in-depth discussion on if IDLE should stay or go.
>

Todd, note that I did not propose to remove IDLE from Python distributions,
just from the Python core (Mercurial repository, to be technically
precise). This is a big difference. Outside the Python core a more
free-moving community can be built around developing IDLE. I've seen PEP
434, but it's far from being enough. I just don't think there are enough
core devs with the time and desire to review IDLE patches (especially
non-trivial ones). Outside the Python code, this can be relaxed. And Python
distributions can still bundle some stable IDLE release.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/9a709781/attachment.html>

From ncoghlan at gmail.com  Wed Mar 20 18:18:48 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Mar 2013 10:18:48 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
Message-ID: <CADiSq7e55V91iRFSty5=BfmNhLyXZkZimdmFCqHsLsb2B-CgPg@mail.gmail.com>

On Wed, Mar 20, 2013 at 9:41 AM, Eli Bendersky <eliben at gmail.com> wrote:
> Interesting writeup about PyCon 2013 young coder
> education:http://therealkatie.net/blog/2013/mar/19/pycon-2013-young-coders/
>
> Quote:
>
> "We used IDLE because it's already on Raspian's desktop. Personally, I like
> IDLE as a teaching tool. It's included in the standard library, it does tab
> completion and color coding, and it even has a text editor included so you
> don't have to start your class off by teaching everyone about paths.
>
> Too bad it's broke as hell."
>
> Personally, I think that IDLE reflects badly on Python in more ways than
> one. It's badly maintained, quirky and ugly. It serves a very narrow set of
> uses, and does it badly.
>
> Being part of Python *distributions* and being part of core Python standard
> library are two different things. The former may make sense, the latter IMHO
> makes no sense whatsoever. Outside the Python core IDLE can be maintained
> more freely, with less restrictions on contributors and hopefully become a
> better tool.

Unfortunately, this cannot change until we have a usable installation
tool shipping with CPython. Thus, it can only be on the agenda for
serious consideration in Python 3.5 at the earliest. In the meantime,
any core developers concerned that IDLE reflects badly on Python could
go through the tracker issues on bugs.python.org and try to improve
the situation for 3.4 (and 3.3.2 and 2.7.5). Feedback on Terry's PEP
434 (explicitly pushing IDLE towards "application that ships with
Python that may receive minor enhancements in maintenance releases"
status, rather than "no new features whatsoever in maintenance
releases") would also be appreciated.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From rdmurray at bitdance.com  Wed Mar 20 19:09:41 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Mar 2013 14:09:41 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
Message-ID: <20130320180942.20A6A2500B3@webabinitio.net>

On Wed, 20 Mar 2013 09:41:53 -0700, Eli Bendersky <eliben at gmail.com> wrote:
> Personally, I think that IDLE reflects badly on Python in more ways than
> one. It's badly maintained, quirky and ugly. It serves a very narrow set of
> uses, and does it badly.
> 
> Being part of Python *distributions* and being part of core Python standard
> library are two different things. The former may make sense, the latter
> IMHO makes no sense whatsoever. Outside the Python core IDLE can be
> maintained more freely, with less restrictions on contributors and
> hopefully become a better tool.

On the other hand, after several years of almost complete neglect,
we have some people interested in and actively contributing to making
it better *in the stdib*.  Terry has proposed a PEP for allowing it
to see more rapid changes than a "normal" stdlib package, and I haven't
perceived a lot of opposition to this.  I think Terry's PEP represents
less of change to how we do things than bundling an externally maintained
IDLE would be, especially with respect to Linux.

FYI I talked to someone at PyCon who is not a current contributor to
IDLE but who is very interested in helping with it, and it sounded like
he had the backing of his organization to do this (it was a quick hall
conversation and unfortunately I did not get his name).  So we may be
approaching an inflection point where IDLE will start getting the love
that it needs.

That said, there is something important in the argument that more
contributors could be attracted to an external project.  I'm wondering,
however, if this is more a reflection of a general issue we might want to
look at, than anything specific to IDLE.  Python is a growing project,
and it may be time to start thinking about better ways to encourage
and coordinate more contributions to various pieces of Python and its
standard library.  But that is a much bigger conversation.

--David

From rdmurray at bitdance.com  Wed Mar 20 19:13:20 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Mar 2013 14:13:20 -0400
Subject: [Python-Dev] [Python-checkins] cpython: Issue #13248: removed
	deprecated and undocumented difflib.isbjunk, isbpopular.
In-Reply-To: <CAF-Rda_-rXSs2VVZM6eht=ZP2aq2rgqNj+M4q7Zw4h7Lu-hsew@mail.gmail.com>
References: <3ZVrTH23v4zPCm@mail.python.org>
	<CAF-Rda_-rXSs2VVZM6eht=ZP2aq2rgqNj+M4q7Zw4h7Lu-hsew@mail.gmail.com>
Message-ID: <20130320181320.5B45C2500B3@webabinitio.net>

On Wed, 20 Mar 2013 05:23:43 -0700, Eli Bendersky <eliben at gmail.com> wrote:
> A mention in Misc/NEWS can't hurt here, Terry. Even though it's
> undocumented, some old code could rely on it being there and this code will
> break with the transition to 3.4

Note that we also have a list of deprecated things that were removed in
What's New.

Aside: given the 3.3 experience, I think people should be thinking in
terms of always updating What's New when appropriate, at the time a
commit is made.

--David

From eliben at gmail.com  Wed Mar 20 19:22:12 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 11:22:12 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320180942.20A6A2500B3@webabinitio.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
Message-ID: <CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>

On Wed, Mar 20, 2013 at 11:09 AM, R. David Murray <rdmurray at bitdance.com>wrote:

> On Wed, 20 Mar 2013 09:41:53 -0700, Eli Bendersky <eliben at gmail.com>
> wrote:
> > Personally, I think that IDLE reflects badly on Python in more ways than
> > one. It's badly maintained, quirky and ugly. It serves a very narrow set
> of
> > uses, and does it badly.
> >
> > Being part of Python *distributions* and being part of core Python
> standard
> > library are two different things. The former may make sense, the latter
> > IMHO makes no sense whatsoever. Outside the Python core IDLE can be
> > maintained more freely, with less restrictions on contributors and
> > hopefully become a better tool.
>
> On the other hand, after several years of almost complete neglect,
> we have some people interested in and actively contributing to making
> it better *in the stdib*.  Terry has proposed a PEP for allowing it
> to see more rapid changes than a "normal" stdlib package, and I haven't
> perceived a lot of opposition to this.  I think Terry's PEP represents
> less of change to how we do things than bundling an externally maintained
> IDLE would be, especially with respect to Linux.
>
> FYI I talked to someone at PyCon who is not a current contributor to
> IDLE but who is very interested in helping with it, and it sounded like
> he had the backing of his organization to do this (it was a quick hall
> conversation and unfortunately I did not get his name).  So we may be
> approaching an inflection point where IDLE will start getting the love
> that it needs.
>

The "choke point" is going to be core devs with the time and desire to
review such contributions though. We have a relatively strict process in
the Python core, which makes a lot of since *because* it's Python core.
Getting things committed in Python is not easy, and even if we get a sudden
influx of good patches (which I doubt) these will take time to review and
get committed. In an outside project there's much less friction.

IDLE would be a great first foray into this "separate project" world,
because it is many ways a separate project.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/ee7b274d/attachment.html>

From barry at python.org  Wed Mar 20 18:09:07 2013
From: barry at python.org (Barry Warsaw)
Date: Wed, 20 Mar 2013 10:09:07 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
Message-ID: <20130320100907.22a24586@anarchist>

On Mar 20, 2013, at 11:22 AM, Eli Bendersky wrote:

>IDLE would be a great first foray into this "separate project" world,
>because it is many ways a separate project.

I really think that's true.  A separate project, occasionally sync'd back into
the stdlib by a core dev seems like the right way to manage IDLE.

-Barry

From guido at python.org  Wed Mar 20 19:47:03 2013
From: guido at python.org (Guido van Rossum)
Date: Wed, 20 Mar 2013 11:47:03 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
Message-ID: <CAP7+vJ+ABcfuW5wFOqMrYW=EcTmZC26rLTG8ejWJfes2GH=sOQ@mail.gmail.com>

On Wed, Mar 20, 2013 at 11:22 AM, Eli Bendersky <eliben at gmail.com> wrote:

>
>
>
> On Wed, Mar 20, 2013 at 11:09 AM, R. David Murray <rdmurray at bitdance.com>wrote:
>
>> On Wed, 20 Mar 2013 09:41:53 -0700, Eli Bendersky <eliben at gmail.com>
>> wrote:
>> > Personally, I think that IDLE reflects badly on Python in more ways than
>> > one. It's badly maintained, quirky and ugly. It serves a very narrow
>> set of
>> > uses, and does it badly.
>> >
>> > Being part of Python *distributions* and being part of core Python
>> standard
>> > library are two different things. The former may make sense, the latter
>> > IMHO makes no sense whatsoever. Outside the Python core IDLE can be
>> > maintained more freely, with less restrictions on contributors and
>> > hopefully become a better tool.
>>
>> On the other hand, after several years of almost complete neglect,
>> we have some people interested in and actively contributing to making
>> it better *in the stdib*.  Terry has proposed a PEP for allowing it
>> to see more rapid changes than a "normal" stdlib package, and I haven't
>> perceived a lot of opposition to this.  I think Terry's PEP represents
>> less of change to how we do things than bundling an externally maintained
>> IDLE would be, especially with respect to Linux.
>>
>> FYI I talked to someone at PyCon who is not a current contributor to
>> IDLE but who is very interested in helping with it, and it sounded like
>> he had the backing of his organization to do this (it was a quick hall
>> conversation and unfortunately I did not get his name).  So we may be
>> approaching an inflection point where IDLE will start getting the love
>> that it needs.
>>
>
> The "choke point" is going to be core devs with the time and desire to
> review such contributions though. We have a relatively strict process in
> the Python core, which makes a lot of since *because* it's Python core.
> Getting things committed in Python is not easy, and even if we get a sudden
> influx of good patches (which I doubt) these will take time to review and
> get committed. In an outside project there's much less friction.
>
> IDLE would be a great first foray into this "separate project" world,
> because it is many ways a separate project.
>

+1



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/bef487f7/attachment.html>

From benjamin at python.org  Wed Mar 20 20:14:09 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Wed, 20 Mar 2013 14:14:09 -0500
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320100907.22a24586@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
Message-ID: <CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>

2013/3/20 Barry Warsaw <barry at python.org>:
> On Mar 20, 2013, at 11:22 AM, Eli Bendersky wrote:
>
>>IDLE would be a great first foray into this "separate project" world,
>>because it is many ways a separate project.
>
> I really think that's true.  A separate project, occasionally sync'd back into
> the stdlib by a core dev seems like the right way to manage IDLE.

I would advise against this. Basically, every "externally-maintained"
package with have causes pain. For example, the stdlib now has some
long-diverged fork of simplejson. With xml.etree, it was not clear for
years whether core developers could touch it even though the external
project had died. Either the stdlib and IDLE should go separate ways
or development has to happen in the stdlib with CPython release
schedule and policies.



-- 
Regards,
Benjamin

From dan at woz.io  Wed Mar 20 20:04:19 2013
From: dan at woz.io (Daniel Wozniak)
Date: Wed, 20 Mar 2013 12:04:19 -0700
Subject: [Python-Dev] PyCon Sprints - Thank you
Message-ID: <514A0833.7000101@woz.io>

David and Senthil,

I won't make it to the sprints today because my ride wants to go into 
San Francisco to do touristy things. I'll be flying back to Arizona this 
evening. I still have a fair amount of code that has not been submitted 
to the issue tracker. I will sprint from my house tomorrow and will 
submit whatever else I have ready tomorrow night.

In the future, I'd also like to continue contributing by continuing to 
improve test coverage since that is helping me get familiar with both 
the code base, style, and general processes of contributing. Thank you 
both for all your help and patience. You have been very welcoming. I am 
sure Python will continue to grow and attract new contributors so long 
as there are mentors such as yourselves there to support n00bs like me.

Thanks again,

Daniel Wozniak

From rdmurray at bitdance.com  Wed Mar 20 20:24:47 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Wed, 20 Mar 2013 15:24:47 -0400
Subject: [Python-Dev] PyCon Sprints - Thank you
In-Reply-To: <514A0833.7000101@woz.io>
References: <514A0833.7000101@woz.io>
Message-ID: <20130320192447.4F188250069@webabinitio.net>

Thank you for your contributions, and we look forward to anything else
you may choose to contribute!

--David

From guido at python.org  Wed Mar 20 20:31:44 2013
From: guido at python.org (Guido van Rossum)
Date: Wed, 20 Mar 2013 12:31:44 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
Message-ID: <CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>

On Wed, Mar 20, 2013 at 12:14 PM, Benjamin Peterson <benjamin at python.org>wrote:

> 2013/3/20 Barry Warsaw <barry at python.org>:
> > On Mar 20, 2013, at 11:22 AM, Eli Bendersky wrote:
> >
> >>IDLE would be a great first foray into this "separate project" world,
> >>because it is many ways a separate project.
> >
> > I really think that's true.  A separate project, occasionally sync'd
> back into
> > the stdlib by a core dev seems like the right way to manage IDLE.
>
> I would advise against this. Basically, every "externally-maintained"
> package with have causes pain. For example, the stdlib now has some
> long-diverged fork of simplejson. With xml.etree, it was not clear for
> years whether core developers could touch it even though the external
> project had died. Either the stdlib and IDLE should go separate ways
> or development has to happen in the stdlib with CPython release
> schedule and policies.
>

Agreed that the "sync into stdlib" think should not happen, or should at
best be a temporary measure until we can remove idle from the source
tarball (maybe at the 3.4 release, otherwise at 3.5).

The main thing I like about the separate project idea is that, given that
only a small group of people care about IDLE, it is much more satisfying
for them to be able to release IDLE separately to their user community
regularly (every month if they want to) rather than being held to the core
Python release schedule and practices. We should deal with compatibility
obligations of the stdlib in the usual way, though maybe we can just delete
it in 3.4, since few people presumably use idlelib apart from IDLE itself.
Binary distributions from python.org should still include IDLE (and Tcl/Tk)
-- however we should switch to bundling the separate project's output
rather than bundling the increasingly broken version in the stdlib. What
other distributors do is outside our control, but we ought to recommend
them to do the same.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/9cecad30/attachment.html>

From barry at python.org  Wed Mar 20 20:38:33 2013
From: barry at python.org (Barry Warsaw)
Date: Wed, 20 Mar 2013 12:38:33 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
Message-ID: <20130320123833.14886a86@anarchist>

On Mar 20, 2013, at 12:31 PM, Guido van Rossum wrote:

>Agreed that the "sync into stdlib" think should not happen, or should at
>best be a temporary measure until we can remove idle from the source
>tarball (maybe at the 3.4 release, otherwise at 3.5).

Right.  Ultimately, I think IDLE should be a separate project entirely, but I
guess there's push back against that too.

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/345982a7/attachment.pgp>

From guido at python.org  Wed Mar 20 20:40:07 2013
From: guido at python.org (Guido van Rossum)
Date: Wed, 20 Mar 2013 12:40:07 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320123833.14886a86@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
Message-ID: <CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>

On Wed, Mar 20, 2013 at 12:38 PM, Barry Warsaw <barry at python.org> wrote:

> On Mar 20, 2013, at 12:31 PM, Guido van Rossum wrote:
>
> >Agreed that the "sync into stdlib" think should not happen, or should at
> >best be a temporary measure until we can remove idle from the source
> >tarball (maybe at the 3.4 release, otherwise at 3.5).
>
> Right.  Ultimately, I think IDLE should be a separate project entirely,
> but I
> guess there's push back against that too.


I didn't hear any at the sprint here.

-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/e24189fd/attachment.html>

From eliben at gmail.com  Wed Mar 20 20:46:25 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 12:46:25 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
Message-ID: <CAF-Rda-s1K6i9OG0fDgkR51-1qqs2mshSnX9tUDURkVFRr9i0g@mail.gmail.com>

On Wed, Mar 20, 2013 at 12:14 PM, Benjamin Peterson <benjamin at python.org>wrote:

> 2013/3/20 Barry Warsaw <barry at python.org>:
> > On Mar 20, 2013, at 11:22 AM, Eli Bendersky wrote:
> >
> >>IDLE would be a great first foray into this "separate project" world,
> >>because it is many ways a separate project.
> >
> > I really think that's true.  A separate project, occasionally sync'd
> back into
> > the stdlib by a core dev seems like the right way to manage IDLE.
>
> I would advise against this. Basically, every "externally-maintained"
> package with have causes pain. For example, the stdlib now has some
> long-diverged fork of simplejson. With xml.etree, it was not clear for
> years whether core developers could touch it even though the external
> project had died. Either the stdlib and IDLE should go separate ways
> or development has to happen in the stdlib with CPython release
> schedule and policies.
>

There are other dependencies like libffi, but I really think IDLE is
different. xml.etree and libffi are building blocks upon which a lot of
users' code depends. So we have to keep maintaining them (unless there's
some sort of agreed deprecation process). IDLE is really a stand-alone
project built on Python. It's unique in this respect.
Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/9638c570/attachment.html>

From python-dev at masklinn.net  Wed Mar 20 20:51:36 2013
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 20 Mar 2013 20:51:36 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320123833.14886a86@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
Message-ID: <BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>

On 2013-03-20, at 20:38 , Barry Warsaw wrote:

> On Mar 20, 2013, at 12:31 PM, Guido van Rossum wrote:
> 
>> Agreed that the "sync into stdlib" think should not happen, or should at
>> best be a temporary measure until we can remove idle from the source
>> tarball (maybe at the 3.4 release, otherwise at 3.5).
> 
> Right.  Ultimately, I think IDLE should be a separate project entirely, but I
> guess there's push back against that too.

The problem with it is, well, that it's a separate project so unless it
is still packaged in (in which case it's not quite separate project,
just a separate source tree) it's got to be downloaded and installed
separately.

That would be a blow to educators, but also Windows users: while the CLI
works very nicely in unices, that's not the case with the win32 console
which is as best as I can describe it a complete turd, making IDLE a
very nice proposition there (I never use IDLE on Linux or OSX, but do
all the time in Windows). It also provides a rather capable (and in many
case sufficient) code editor for a platform which lacks any form of
native text editor allowing sane edition of code.

Installing the Python windows packages and having everything "work" (in
the sense that you can immediately start writing and running python
code) is ? I think ? a pretty big feature.

From barry at python.org  Wed Mar 20 20:54:45 2013
From: barry at python.org (Barry Warsaw)
Date: Wed, 20 Mar 2013 12:54:45 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
Message-ID: <20130320125445.64676161@anarchist>

On Mar 20, 2013, at 12:40 PM, Guido van Rossum wrote:

>I didn't hear any at the sprint here.

JFDI! :)

-Barry
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 836 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/fadd6658/attachment.pgp>

From brian at python.org  Wed Mar 20 20:59:37 2013
From: brian at python.org (Brian Curtin)
Date: Wed, 20 Mar 2013 14:59:37 -0500
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
Message-ID: <CAD+XWwrhcH4Zi7yfW5My76m4CQtRgqnfi+AyL3M52cgceB+21g@mail.gmail.com>

On Wed, Mar 20, 2013 at 2:51 PM, Xavier Morel <python-dev at masklinn.net> wrote:
> That would be a blow to educators, but also Windows users: while the CLI
> works very nicely in unices, that's not the case with the win32 console
> which is as best as I can describe it a complete turd, making IDLE a
> very nice proposition there (I never use IDLE on Linux or OSX, but do
> all the time in Windows).

Can you explain this a bit more? I've been using the CLI python.exe on
Windows, Mac, and Linux for years and I don't know what you're talking
about.

From eliben at gmail.com  Wed Mar 20 21:00:35 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 13:00:35 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
Message-ID: <CAF-Rda96u5PyvLzBOJzzqM2LV7bfjCHB+-rNN4aMS1jOj6OWDA@mail.gmail.com>

On Wed, Mar 20, 2013 at 12:51 PM, Xavier Morel <python-dev at masklinn.net>wrote:

> On 2013-03-20, at 20:38 , Barry Warsaw wrote:
>
> > On Mar 20, 2013, at 12:31 PM, Guido van Rossum wrote:
> >
> >> Agreed that the "sync into stdlib" think should not happen, or should at
> >> best be a temporary measure until we can remove idle from the source
> >> tarball (maybe at the 3.4 release, otherwise at 3.5).
> >
> > Right.  Ultimately, I think IDLE should be a separate project entirely,
> but I
> > guess there's push back against that too.
>
> The problem with it is, well, that it's a separate project so unless it
> is still packaged in (in which case it's not quite separate project,
> just a separate source tree) it's got to be downloaded and installed
> separately.
>
> That would be a blow to educators, but also Windows users: while the CLI
> works very nicely in unices, that's not the case with the win32 console
> which is as best as I can describe it a complete turd, making IDLE a
> very nice proposition there (I never use IDLE on Linux or OSX, but do
> all the time in Windows). It also provides a rather capable (and in many
> case sufficient) code editor for a platform which lacks any form of
> native text editor allowing sane edition of code.
>
> Installing the Python windows packages and having everything "work" (in
> the sense that you can immediately start writing and running python
> code) is ? I think ? a pretty big feature.
> _____________________________
>

FWIW, I specifically suggested that IDLE still gets packaged with Python
releases for Windows. This shouldn't be hard, because IDLE depends on
Python rather than the other way around. Packaging is not what I'm against.
Maintaining this project's source within the Python core *is*.

I would be interested to hear Martin's opinion on this, as he's producing
the Windows installers.

Eli

P.S. other Python distributions like ActiveState already bundle additional
projects with their Python releases (pywin32 if I'm not mistaken).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/02c615f9/attachment.html>

From eliben at gmail.com  Wed Mar 20 21:14:00 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 13:14:00 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
Message-ID: <CAF-Rda-wTo_KF8swymc3FKjegVFcpsHfEKifsqE_r_KfLd1grg@mail.gmail.com>

>> Agreed that the "sync into stdlib" think should not happen, or should at

>  >> best be a temporary measure until we can remove idle from the source
> >> tarball (maybe at the 3.4 release, otherwise at 3.5).
> >
> > Right.  Ultimately, I think IDLE should be a separate project entirely,
> but I
> > guess there's push back against that too.
>
> The problem with it is, well, that it's a separate project so unless it
> is still packaged in (in which case it's not quite separate project,
> just a separate source tree) it's got to be downloaded and installed
> separately.
>
> That would be a blow to educators, but also Windows users: while the CLI
> works very nicely in unices, that's not the case with the win32 console
> which is as best as I can describe it a complete turd, making IDLE a
> very nice proposition there (I never use IDLE on Linux or OSX, but do
> all the time in Windows). It also provides a rather capable (and in many
> case sufficient) code editor for a platform which lacks any form of
> native text editor allowing sane edition of code.
>
> Installing the Python windows packages and having everything "work" (in
> the sense that you can immediately start writing and running python
> code) is ? I think ? a pretty big feature.


Oh, and another thing. If a Windows user wants a good Python shell, IDLE
should be his last choice. There's Spyder, there's IPython, there's
probably a bunch of others I'm not aware of.

This is for IDLE as a shell. The same can be said for IDLE as an editor.

This is precisely my main gripe with IDLE: it does a lot of things, but
neither of them it does well. It stomps in place while "competition" moves
fast forward. If someone cares about it, that someone should fix it and
improve it, quickly. The speed required for such improvements is
unrealistic for Python core.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/7fb26e57/attachment.html>

From dholth at gmail.com  Wed Mar 20 21:14:40 2013
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 20 Mar 2013 16:14:40 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320125445.64676161@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
	<20130320125445.64676161@anarchist>
Message-ID: <CAG8k2+47yo0Xsf3NjSsyXp47tGgrYTuzwGH5pdEU=1yLjZcw4Q@mail.gmail.com>

On Wed, Mar 20, 2013 at 3:54 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 20, 2013, at 12:40 PM, Guido van Rossum wrote:
>
>>I didn't hear any at the sprint here.
>
> JFDI! :)
>
> -Barry

+1 why are we still talking show me the patches

From doko at ubuntu.com  Wed Mar 20 21:18:19 2013
From: doko at ubuntu.com (Matthias Klose)
Date: Wed, 20 Mar 2013 13:18:19 -0700
Subject: [Python-Dev] How to fix the incorrect shared library extension on
 linux for 3.2 and newer?
Message-ID: <514A198B.500@ubuntu.com>

This is http://bugs.python.org/issue16754, affecting Linux systems only, and
only those which don't provide static libraries. PEP 3149 did change the SO
macro to include the ABI tag, although the SO macro is used to search for shared
system libraries too.  E.g. searching for the jpeg library search for a file
libjpeg.cpython3.3m.so, which is not found. If the static library libjpeg.a is
found, it is taken, and linked as -ljpeg, which then links with the shared library.

The patch in the issue now makes a distinction between EXT_SUFFIX and
SHLIB_SUFFIX, and restores the value of SO to SHLIB_SUFFIX.  Now this could
break users of sysconfig.get_config_var('SO'), however I don't see a better way
than to restore the original behaviour and advise people to use the new config
variables.

  Matthias

From python-dev at masklinn.net  Wed Mar 20 21:50:33 2013
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 20 Mar 2013 21:50:33 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAD+XWwrhcH4Zi7yfW5My76m4CQtRgqnfi+AyL3M52cgceB+21g@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
	<CAD+XWwrhcH4Zi7yfW5My76m4CQtRgqnfi+AyL3M52cgceB+21g@mail.gmail.com>
Message-ID: <AB3CC5B2-3E32-4491-AA8C-5CB05D67393B@masklinn.net>

On 2013-03-20, at 20:59 , Brian Curtin wrote:

> On Wed, Mar 20, 2013 at 2:51 PM, Xavier Morel <python-dev at masklinn.net> wrote:
>> That would be a blow to educators, but also Windows users: while the CLI
>> works very nicely in unices, that's not the case with the win32 console
>> which is as best as I can describe it a complete turd, making IDLE a
>> very nice proposition there (I never use IDLE on Linux or OSX, but do
>> all the time in Windows).
> 
> Can you explain this a bit more? I've been using the CLI python.exe on
> Windows, Mac, and Linux for years and I don't know what you're talking
> about.

Windows's terminal emulator (the "win32 console")'s deficiencies don't
break it for running existing script, but make using it interactively a
rather thankless task, at least as far as I'm concerned: no readline
keybinding (e.g. C-a & C-e), very limited scrollback, fixed width,
non-handling of signals (C-d will simply print ^D, a syntax error),
odd copy behavior (rectangle copies *only*), etc?

From python-dev at masklinn.net  Wed Mar 20 21:59:58 2013
From: python-dev at masklinn.net (Xavier Morel)
Date: Wed, 20 Mar 2013 21:59:58 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda-wTo_KF8swymc3FKjegVFcpsHfEKifsqE_r_KfLd1grg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
	<CAF-Rda-wTo_KF8swymc3FKjegVFcpsHfEKifsqE_r_KfLd1grg@mail.gmail.com>
Message-ID: <3EAD9253-17A5-45CD-AB32-0BE37BE72E48@masklinn.net>


On 2013-03-20, at 21:14 , Eli Bendersky wrote:

>>> Agreed that the "sync into stdlib" think should not happen, or should at
> 
>>>> best be a temporary measure until we can remove idle from the source
>>>> tarball (maybe at the 3.4 release, otherwise at 3.5).
>>> 
>>> Right.  Ultimately, I think IDLE should be a separate project entirely,
>> but I
>>> guess there's push back against that too.
>> 
>> The problem with it is, well, that it's a separate project so unless it
>> is still packaged in (in which case it's not quite separate project,
>> just a separate source tree) it's got to be downloaded and installed
>> separately.
>> 
>> That would be a blow to educators, but also Windows users: while the CLI
>> works very nicely in unices, that's not the case with the win32 console
>> which is as best as I can describe it a complete turd, making IDLE a
>> very nice proposition there (I never use IDLE on Linux or OSX, but do
>> all the time in Windows). It also provides a rather capable (and in many
>> case sufficient) code editor for a platform which lacks any form of
>> native text editor allowing sane edition of code.
>> 
>> Installing the Python windows packages and having everything "work" (in
>> the sense that you can immediately start writing and running python
>> code) is ? I think ? a pretty big feature.
> 
> 
> Oh, and another thing. If a Windows user wants a good Python shell, IDLE
> should be his last choice. There's Spyder, there's IPython, there's
> probably a bunch of others I'm not aware of.

Sure, there are plenty of tools for the experienced python developer
with reasons to invest time in a windows development setup, but IDLE
provides an acceptable low-cost and low-investment base which is
*there*: it does not require spending a day downloading, trying out and
getting familiar with a dozen different Python IDEs, it's simple and
for the most part it works.

I view it as an mg, not an emacs, if you see what I mean.

From brett at python.org  Wed Mar 20 22:05:10 2013
From: brett at python.org (Brett Cannon)
Date: Wed, 20 Mar 2013 17:05:10 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
Message-ID: <CAP1=2W7Gkr5_WKmzTJH757jgtrOGE7SPx3n7JPRu7XP4Qn4+yQ@mail.gmail.com>

On Wed, Mar 20, 2013 at 3:51 PM, Xavier Morel <python-dev at masklinn.net>wrote:

> On 2013-03-20, at 20:38 , Barry Warsaw wrote:
>
> > On Mar 20, 2013, at 12:31 PM, Guido van Rossum wrote:
> >
> >> Agreed that the "sync into stdlib" think should not happen, or should at
> >> best be a temporary measure until we can remove idle from the source
> >> tarball (maybe at the 3.4 release, otherwise at 3.5).
> >
> > Right.  Ultimately, I think IDLE should be a separate project entirely,
> but I
> > guess there's push back against that too.
>
> The problem with it is, well, that it's a separate project so unless it
> is still packaged in (in which case it's not quite separate project,
> just a separate source tree) it's got to be downloaded and installed
> separately.
>
> That would be a blow to educators, but also Windows users: while the CLI
> works very nicely in unices, that's not the case with the win32 console
> which is as best as I can describe it a complete turd, making IDLE a
> very nice proposition there (I never use IDLE on Linux or OSX, but do
> all the time in Windows). It also provides a rather capable (and in many
> case sufficient) code editor for a platform which lacks any form of
> native text editor allowing sane edition of code.
>
> Installing the Python windows packages and having everything "work" (in
> the sense that you can immediately start writing and running python
> code) is ? I think ? a pretty big feature.


First, a clarification since people seem to have missed it a couple of
times: both Eli and Guido said IDLE would continue to be bundled with
binary distributions from python.org, just developed independently. Now
Guido's comment may have just been to handle deprecation of IDLE from the
stdlib, but at least he wasn't saying "leave it out tomorrow", just "take
it out of the stdlib to be developed independently". That still allows it
to come with Python and alleviate any installation issues by shifting the
load to release managers.

Second, I hear the "it will hurt educators" argument every time this topic
comes up, and so I want to know exactly where the challenge comes from. Is
it from people coming to a class with their own laptop where they can't
install anything due to lack of knowledge but can bring up a shell and type
"idle"? If that were the case can't you also just as easily teach them to
type "pysetup pip", "pip install idle", "idle"? Or heck, if Nick's
dead-simple bootstrap installer could handle "pysetup idle" then you can
cut that down a step. Notice that none of this suggests the removal of
tkinter since that never changes and the hard work is already done
(although if we could get the binary wheel for the various OSs to just
include tkinter w/ IDLE then that could also potentially move out and
shrink the binary even further).

Is it lack of administrative access on machines in a computer lab? Then I
would ask if IDLE is even included in Linux distributions by default
typically or can be removed separately?

Is it lack of administrative access on personal laptops? In that case you
should be able to ask them to find out the password before they come. You
can also ask them to install into their user-level site-packages directory
(we have PEP 370 for a reason).

Regardless of the answers to the questions above, I support the idea of at
least releasing IDLE more often and that mostly means developing it
externally from the stdlib. If we want to continue to bundle it in the
binary, then we should pull in the latest stable release when we cut a new
version of Python. But regardless the current situation should not continue.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/7cbf37a7/attachment.html>

From eliben at gmail.com  Wed Mar 20 22:15:38 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 14:15:38 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <3EAD9253-17A5-45CD-AB32-0BE37BE72E48@masklinn.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
	<CAF-Rda-wTo_KF8swymc3FKjegVFcpsHfEKifsqE_r_KfLd1grg@mail.gmail.com>
	<3EAD9253-17A5-45CD-AB32-0BE37BE72E48@masklinn.net>
Message-ID: <CAF-Rda9T1TjsdhQ2ejGMMYKftbrJsOTADp2o38Kbs0+Q5F_qkA@mail.gmail.com>

On Wed, Mar 20, 2013 at 1:59 PM, Xavier Morel <python-dev at masklinn.net>wrote:

>
> On 2013-03-20, at 21:14 , Eli Bendersky wrote:
>
> >>> Agreed that the "sync into stdlib" think should not happen, or should
> at
> >
> >>>> best be a temporary measure until we can remove idle from the source
> >>>> tarball (maybe at the 3.4 release, otherwise at 3.5).
> >>>
> >>> Right.  Ultimately, I think IDLE should be a separate project entirely,
> >> but I
> >>> guess there's push back against that too.
> >>
> >> The problem with it is, well, that it's a separate project so unless it
> >> is still packaged in (in which case it's not quite separate project,
> >> just a separate source tree) it's got to be downloaded and installed
> >> separately.
> >>
> >> That would be a blow to educators, but also Windows users: while the CLI
> >> works very nicely in unices, that's not the case with the win32 console
> >> which is as best as I can describe it a complete turd, making IDLE a
> >> very nice proposition there (I never use IDLE on Linux or OSX, but do
> >> all the time in Windows). It also provides a rather capable (and in many
> >> case sufficient) code editor for a platform which lacks any form of
> >> native text editor allowing sane edition of code.
> >>
> >> Installing the Python windows packages and having everything "work" (in
> >> the sense that you can immediately start writing and running python
> >> code) is ? I think ? a pretty big feature.
> >
> >
> > Oh, and another thing. If a Windows user wants a good Python shell, IDLE
> > should be his last choice. There's Spyder, there's IPython, there's
> > probably a bunch of others I'm not aware of.
>
> Sure, there are plenty of tools for the experienced python developer
> with reasons to invest time in a windows development setup, but IDLE
> provides an acceptable low-cost and low-investment base which is
> *there*: it does not require spending a day downloading, trying out and
> getting familiar with a dozen different Python IDEs, it's simple and
> for the most part it works.
>
> I view it as an mg, not an emacs, if you see what I mean.
> _______________________________________________
>

This seems more like an education & documentation issue than a technical
problem. We can explicitly recommend Python Windows users to install IDLE
or Spyder or IPython in some friendly "get started on Windows" guide.

But we seem to be talking about different things, really. I'm not saying we
shouldn't distribute IDLE with Python on Windows, at this point (I think
this will be a good idea in the future, but let's make it gradual). All I'm
saying is that IDLE should be developed outside the CPython core project.
This has the potential of making  both CPython and IDLE better.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/bb77e9ea/attachment-0001.html>

From senthil at uthcode.com  Wed Mar 20 22:23:06 2013
From: senthil at uthcode.com (Senthil Kumaran)
Date: Wed, 20 Mar 2013 14:23:06 -0700
Subject: [Python-Dev] PyCon Sprints - Thank you
In-Reply-To: <20130320192447.4F188250069@webabinitio.net>
References: <514A0833.7000101@woz.io>
	<20130320192447.4F188250069@webabinitio.net>
Message-ID: <CAPOVWOTdeLiuv=GQgd=-NWRedkuTksnOkEYGM5QFMgtv=o=3OQ@mail.gmail.com>

Thanks Daniel, for all the patches and improving the test coverage.
Hope you had a good time and will you enjoy contributing further.

Happy touring SF.

-- 
Senthil

On Wed, Mar 20, 2013 at 12:24 PM, R. David Murray <rdmurray at bitdance.com> wrote:
> Thank you for your contributions, and we look forward to anything else
> you may choose to contribute!
>
> --David

From barry at python.org  Wed Mar 20 22:36:48 2013
From: barry at python.org (Barry Warsaw)
Date: Wed, 20 Mar 2013 14:36:48 -0700
Subject: [Python-Dev] How to fix the incorrect shared library extension
 on linux for 3.2 and newer?
In-Reply-To: <514A198B.500@ubuntu.com>
References: <514A198B.500@ubuntu.com>
Message-ID: <20130320143648.6df30a2f@anarchist>

On Mar 20, 2013, at 01:18 PM, Matthias Klose wrote:

>The patch in the issue now makes a distinction between EXT_SUFFIX and
>SHLIB_SUFFIX, and restores the value of SO to SHLIB_SUFFIX.  Now this could
>break users of sysconfig.get_config_var('SO'), however I don't see a better
>way than to restore the original behaviour and advise people to use the new
>config variables.

It should probably be considered a bug that we changed the meaning of SO in
PEP 3149, but I don't think anybody realized it was being used for both
purposes (otherwise I'm sure we wouldn't have done it that way).  I suppose
Georg should make the final determination for 3.2 and 3.3, but the solution
you propose seems about the best you can do.

As we discussed at Pycon, you'll post a diff to the PEP in the tracker issue
and I'll commit that when I figure out the best way to indicate that a PEP has
been updated post-Final status.

-Barry

From ncoghlan at gmail.com  Wed Mar 20 23:05:40 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 20 Mar 2013 15:05:40 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320125445.64676161@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
	<20130320125445.64676161@anarchist>
Message-ID: <CADiSq7cuJwrzWKSTm-KtC6L-Jk44aWTGk3zG5gEp8Fivz1OOxQ@mail.gmail.com>

On Wed, Mar 20, 2013 at 12:54 PM, Barry Warsaw <barry at python.org> wrote:
> On Mar 20, 2013, at 12:40 PM, Guido van Rossum wrote:
>
>>I didn't hear any at the sprint here.
>
> JFDI! :)

Please don't rush this. We have Roger Serwy being given commit
privileges specifically to work on Idle, Terry's PEP proposing to make
it explicit that we consider IDLE an application bundled with Python
that can receive new features in maintenance releases and several
people expressing interest in helping to make IDLE better (primarily
educators, including Katie Cunningham, one of the teachers who ran the
Raspberry Pi based Young Coders sessions for teens and pre-teens here
at PyCon).

These are the people who care about Idle, we should be recruiting them
to work on it *as it is now*, and then letting them decide if they
wish to continue working on it as it is now, or if they prefer to move
to a more inclusive development platform which allows them to accept
pull requests rather than requiring patches to be generated locally
and uploaded to our tracker.

It's not as simple as saying "let's split it out to a separate repo
and then bundle it", because bundling still means python-dev is
placing it's stamp of approval on the application, which means we
should be satisfied that the developers leading the project are people
we trust as stewards of software we distribute.

Yes, the status quo of Idle is not something we should allow to
continue indefinitely, but decisions about its future development
should be made by active maintainers that are already trusted to make
changes to it (such as Terry and Roger), rather than those of us that
don't use it, and aren't interested in maintaining it.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From "ja...py" at farowl.co.uk  Wed Mar 20 22:27:05 2013
From: "ja...py" at farowl.co.uk (Jeff Allen)
Date: Wed, 20 Mar 2013 21:27:05 +0000
Subject: [Python-Dev] Recent changes to TextIOWrapper and its tests
In-Reply-To: <ki964o$evp$1@ger.gmane.org>
References: <5147785D.3070100@farowl.co.uk> <ki964o$evp$1@ger.gmane.org>
Message-ID: <514A29A9.3020500@farowl.co.uk>

On 19/03/2013 08:03, Serhiy Storchaka wrote:
> On 18.03.13 22:26, Jeff Allen wrote:
>> The puzzle is that it requires t.read() to succeed.
>>
>> When I insert a check for bytes type in all the places it seems
>> necessary in my code, I pass the first two conditions, but since
>> t.read() also raises TypeError, the overall test fails. Is reading the
>> stream with read() intended to succeed? Why is this desired?
>
> An alternative option is to change C implementation of 
> TextIOWrapper.read() to raise an exception in this case. However I 
> worry that it can break backward compatibility.
Thanks for this and the previous note. It is good to get it from the 
horse's mouth. I was surprised that the Python 3 version of the test was 
different here. I'd looked at the source of textio.c and found no test 
for bytes type in the n<0 branch of textiowrapper_read. Having tested it 
just now, I see that the TypeError is raised by the decoder in Py3k, 
because the input (when it is a unicode string) does not bear the buffer 
API, and not by a type test in TextIOWrapper.read() at all.

For Jython, I shall make TextIOWrapper raise TypeError and our version 
of test_io will check for it. Any incompatibility relates only to 
whether a particular mistake sometimes goes undetected, so I feel pretty 
compatible. Added to which, this is the behaviour of Python 3 and we 
feel safe anticipating Py3k in small ways.

>   Are there other tests (in other test files) which fail with a new 
> Jython TextIOWrapper?

I don't think there is anything else specific to TextIOWrapper, but if 
there is I will first treat that as a fault in our implementation. This 
is the general approach, to emulate the CPython implementation unless a 
test is clearly specific to arbitrary implementation choices. (There's a 
general exclusion for garbage collection.) In this case the test 
appeared to reflect an accident of implementation, but might just have 
been deliberate.

Parts of the Jython implementation of io not yet implemented in Java are 
supplied by a Python module _jyio. This is essentially a copy of the 
corresponding parts of _pyio, except that it has to pass the C* tests, 
not the Py* tests. In places _jyio is therefore closer to _io than is 
_pyio. For example, it makes the type tests just discussed, and passes 
CTextIOWrapperTest.test_illegal_decoder and test_initialization. 
_jyio.StringIO has getstate and setstate methods lacking in _pyio 
counterparts to pass pickling tests in test_memoryio. This might be of 
interest to CPython for _pyio.

Jeff Allen


From solipsis at pitrou.net  Wed Mar 20 23:11:47 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Wed, 20 Mar 2013 23:11:47 +0100
Subject: [Python-Dev] IDLE in the stdlib
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
	<20130320125445.64676161@anarchist>
	<CADiSq7cuJwrzWKSTm-KtC6L-Jk44aWTGk3zG5gEp8Fivz1OOxQ@mail.gmail.com>
Message-ID: <20130320231147.665b3adc@pitrou.net>

On Wed, 20 Mar 2013 15:05:40 -0700
Nick Coghlan <ncoghlan at gmail.com> wrote:
> 
> Yes, the status quo of Idle is not something we should allow to
> continue indefinitely, but decisions about its future development
> should be made by active maintainers that are already trusted to make
> changes to it (such as Terry and Roger), rather than those of us that
> don't use it, and aren't interested in maintaining it.

Definitely. People shouldn't remain quiescently torpid about the idle
status quo.

Regards

Antoine.



From kbk at shore.net  Wed Mar 20 23:48:15 2013
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 20 Mar 2013 18:48:15 -0400
Subject: [Python-Dev] IDLE in the stdlib
References: <E1UIRP3-0004SF-PR@smtp3.shore.net>
Message-ID: <1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>

[Barry]
> On Mar 20, 2013, at 11:22 AM, Eli Bendersky wrote:
>>IDLE would be a great first foray into this "separate project" world,
because it is many ways a separate project.
> I really think that's true.  A separate project, occasionally sync'd
back
> into
> the stdlib by a core dev seems like the right way to manage IDLE.

It seems to me that we are seeing increasing use of IDLE for beginner
training. I've seen several recent Python books that use IDLE as their
programming environment, and which include IDLE screen captures in the
text.

I've always felt that IDLE should be targeted to an eight year old
beginner, and should work uniformly across the major platforms.  That
now
includes the Raspberry Pi!!

I believe it's very important that Python come with an IDE as part of
the
"batteries"  - it's very awkward for a beginner to write code in
something
like Notepad and then run and debug it in a Windows command shell. Just
getting the paths right is problematic (and I'm not talking about
backslashes).

It's very helpful for an instructor to be able to deal with a single
application that runs on all the major platforms, and not have to spend
a
lot of time getting the tools up to speed before the actual Python
training can begin.

And, while an instructor can walk a student through downloading and
installing some IDE, it's very helpful IMHO for a beginner working alone
on Windows or Mac to be able to just click on IDLE.

A Raspberry Pi might not even have a web connection!

IDLE has a single keystroke round trip - it's an IDE, not just an editor
like Sublime Text or Notepad.  In the 21st century, people expect some
sort of IDE.  Or, they should!

IDLE forked nearly a decade ago to introduce subprocess execution and
the
configuration dialog.  Subsequently, I merged it back into core and it
played a useful role in Python 3 development.

Scheme hackers write new Scheme implementations.  Python hackers tend to
write editors and IDEs, it seems.  I think, considering all the
competition, that IDLE would have died if it hadn't been merged back. 
Instead, many of its competitors died.

So, although I'm pretty agnostic regarding where development is done, I
think Python should continue to release a simple native IDE in its
binaries, and I worry that IDLE will be eventually dropped from the
binaries if it's separate. Right now, Apple is delivering IDLE along
with
Python (though there are issues with the current installation) and I
hope
that will continue.

OTOH, development is likely to be more vigorous if it's separate.

I'd also like to make a plea to keep IDLE's interface clean and basic. 
There are lots of complex IDEs available for those who want them.  It's
natural for developers to add features, that's what they do :-), but you
don't hand a novice a Ferrari (or emacs) and expect good results.  IMHO
some of the feature patches on the tracker should be rejected on that
basis.

It's sometimes said that IDLE is "ugly" or "broken".  These terms are
subjective!  If it's truly broken, then we should fix it.  If it's
"broken" because a feature is missing, maybe that's an intentional part
of
Guido's design of a simple Python IDE.

-- 
KBK

From kbk at shore.net  Thu Mar 21 00:10:32 2013
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed, 20 Mar 2013 19:10:32 -0400
Subject: [Python-Dev] IDLE in the stdlib
Message-ID: <1363821032.7658.140661207069089.6C4712FB@webmail.messagingengine.com>

I apologize for that formatting mess - the Barracuda rejected my
original email for some reason (squirrelmail directly from shore.net)
and I had to resend it from rejects - it got re-wrapped on transmission.

-- 
KBK

From barry at python.org  Thu Mar 21 00:47:29 2013
From: barry at python.org (Barry Warsaw)
Date: Wed, 20 Mar 2013 16:47:29 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320231147.665b3adc@pitrou.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
	<20130320125445.64676161@anarchist>
	<CADiSq7cuJwrzWKSTm-KtC6L-Jk44aWTGk3zG5gEp8Fivz1OOxQ@mail.gmail.com>
	<20130320231147.665b3adc@pitrou.net>
Message-ID: <20130320164729.715c9e10@anarchist>

On Mar 20, 2013, at 11:11 PM, Antoine Pitrou wrote:

>On Wed, 20 Mar 2013 15:05:40 -0700
>Nick Coghlan <ncoghlan at gmail.com> wrote:
>> 
>> Yes, the status quo of Idle is not something we should allow to
>> continue indefinitely, but decisions about its future development
>> should be made by active maintainers that are already trusted to make
>> changes to it (such as Terry and Roger), rather than those of us that
>> don't use it, and aren't interested in maintaining it.
>
>Definitely. People shouldn't remain quiescently torpid about the idle
>status quo.

The release managers should have a say in the matter, since it does cause some
amount of pain there.

-Barry

From tjreedy at udel.edu  Thu Mar 21 01:15:56 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Mar 2013 20:15:56 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
Message-ID: <kidjg2$u22$1@ger.gmane.org>

On 3/20/2013 2:22 PM, Eli Bendersky wrote:
> On Wed, Mar 20, 2013 at 11:09 AM, R. David Murray <rdmurray at bitdance.com
> <mailto:rdmurray at bitdance.com>> wrote:
>
>     On Wed, 20 Mar 2013 09:41:53 -0700, Eli Bendersky <eliben at gmail.com
>     <mailto:eliben at gmail.com>> wrote:
>> Personally, I think that IDLE reflects badly on Python in more ways than
>> one. It's badly maintained, quirky and ugly. It serves a very narrow set of
>> uses, and does it badly.

Personally, I think running Python with the imitation-MSDOS Windows 
Command Prompt (CP) reflects badly on Python in more ways than one. It's 
anti-maintained, quirky and ugly, and for some uses is badly broken.

I am serious and am not just being sarcastic. I suggested years ago that 
we try to find an alternative. Some details:

Ugly (and quirky): it imitates ancient white type on black background 
CRT monitors. I do not know of anything else on Windows that looks like 
that.

Broken (and quirky): it has an absurdly limited output buffer (under a 
thousand lines) that is over-flowed, for instance, by "python -m test". 
In other words, by the time the test suite has finished, the early 
results and error messages have scrolled off the top. Same can be true 
of verbose repository pulls, doc builds, external dependency fetch and 
compile, help messages, and any other substantial printing. For example, 
start the standard interpreter, enter 'help(str)', page down to the 
bottom, scroll back up, and the top of the help message is gone. *That* 
is real 'broken'.

Quirky: Windows uses cntl-C to copy selected text to the clipboard and 
(where appropriate) cntl-V to insert clipboard text at the cursor pretty 
much everywhere.

(Anti-maintained: I think Microsoft wants people to abandon unix-like 
command processing and that they are intentionally keeping CP ugly and 
disfunctional.)

Now lets look at IDLE: Standard black (+ other colors) on white and 
close to standard gui. *Much* prettier than CP 'Unlimited' lines of code 
and output in windows. *Much* more functional in this respect. I cannot 
think of *anything* in IDLE that compares to the brokeness of CP in 
throwing output away. Standard ^C,^V behavior. To me, IDLE is much less 
quirky than CP.

Oh, but it did have one quirk -- the right context menus lacked the 
standard Copy and Paste entries of Windows applications. When that was 
added for all versions, a couple of people challenged it as a 
default-only 'enhancement' rather than all-versions 'bugfix'. That 
challenge directly lead to PEP434.  In last two months since, IDLE work 
has mostly stopped.  I will say more about maintenance problems, but 
getting back to CP versus IDLE...

 From IDLE:
 >>> print('\x80')
?
 >>> print('\xc8')
?

Impressed? You should be. Open Start menu / Python33 / Python (command 
line) and both of those result (modulo the specific character) in
   UnicodeEncodeError: 'charmap' codec can't encode character
   '\xc8' in position 0: character maps to <undefined>

and we have not even gotten beyond latin-1, into the 'real' unicode char 
set, which IDLE supports MUCH better. For display, default United States 
CP is close to ascii-only.

In summary: IDLE makes Python interactive mode tolerable and useful on 
Windows in ways where CP completely fails. If you are worried about 
something making Python look bad on Windows, target CP before IDLE.

> Getting things committed in Python is not easy, and even if we get a
> sudden influx of good patches (which I doubt)

Given recent history, this is silly. A year and a few months ago, (Dec 
2011, I think), I asked Roger Serwy to start submitting patches, with 
the promise to review and possibly commit some. Consequently, in the 
last year, we have *already* gotten a 'sudden influx of good patches'.

It is true that I have not yet kept up my end of the bargain. One 
problem for me has been an inability to test IDLE patches on Windows 
with repository builds, due to the need for a working tkinter. It turns 
out that the pc build files are buggy and the devguide and the (to me 
confusing) pcbuild/readme do not have the workaround that was 
communicated to me only about 10 days ago.

 > these will take time to review and get committed.

In spite of my slowness, others have been active. Searching 
cpython/Misc/NEWS for ' IDLE ' (case-insensitive) returns 31 hits. For 4 
months (Oct-Jan), that is pretty active, certainly much more active than 
in the years prior to 2012. As soon as PEP434 is resolved and Roger is 
fully on board, the pace will pick up again.

At least a few people have been given commit privileges but never (or 
essentially never) exercised them. I think a couple were specifically 
for working on IDLE. I have not asked any of these people why, but I can 
imagine from my own experience that uncertainty over what is and is not 
allowed is one reason.

I will discuss repository separation in another response, but request 
here that the still vague idea of doing something in the *future* not be 
used to stop PEP434, in whatever form, and current IDLE development *now*.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Thu Mar 21 01:30:23 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Mar 2013 20:30:23 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAD+XWwrhcH4Zi7yfW5My76m4CQtRgqnfi+AyL3M52cgceB+21g@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
	<CAD+XWwrhcH4Zi7yfW5My76m4CQtRgqnfi+AyL3M52cgceB+21g@mail.gmail.com>
Message-ID: <kidkb4$5i6$1@ger.gmane.org>

On 3/20/2013 3:59 PM, Brian Curtin wrote:
> On Wed, Mar 20, 2013 at 2:51 PM, Xavier Morel <python-dev at masklinn.net> wrote:
>> That would be a blow to educators, but also Windows users: while the CLI
>> works very nicely in unices, that's not the case with the win32 console
>> which is as best as I can describe it a complete turd, making IDLE a
>> very nice proposition there (I never use IDLE on Linux or OSX, but do
>> all the time in Windows).
>
> Can you explain this a bit more? I've been using the CLI python.exe on
> Windows, Mac, and Linux for years and I don't know what you're talking
> about.

I gave examples in my response to the original post: ^C,^V do not work 
(but do in IDLE); the output buffer cannot even hold the entire 
'help(str)' response (IDLE can, and much more) ; by default, print() 
cannot even print all latin-1 chars, let alone the BMP (IDLE will at 
least print a box for the entire BMP); CP is frozen in time 30 years 
ago, while IDLE is being maintained and modernized.

-- 
Terry Jan Reedy



From tjreedy at udel.edu  Thu Mar 21 02:17:08 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Mar 2013 21:17:08 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>
References: <E1UIRP3-0004SF-PR@smtp3.shore.net>
	<1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>
Message-ID: <kidn2q$t5s$1@ger.gmane.org>

On 3/20/2013 6:48 PM, Kurt B. Kaiser wrote:

> It seems to me that we are seeing increasing use of IDLE for beginner
> training. I've seen several recent Python books that use IDLE as their
> programming environment, and which include IDLE screen captures in the
> text.

Well, one can hardly use Command Prompt captures, unless one were to 
flip black and white within the window (but not its frame).

> I've always felt that IDLE should be targeted to an eight year old
> beginner, and should work uniformly across the major platforms.  That
> now includes the Raspberry Pi!!

I think it should also work uniformly across Python versions. That is 
the gist of PEP434.

> I believe it's very important that Python come with an IDE as part of
> the "batteries"  - it's very awkward for a beginner to write code in
> something like Notepad and then run and debug it in a Windows command shell.

I cut and pasted when I began ;-).

> IDLE has a single keystroke round trip - it's an IDE, not just an editor
> like Sublime Text or Notepad.  In the 21st century, people expect some
> sort of IDE.  Or, they should!

I have never understood those who suggest that an editor, even a super 
editor, can replace IDLE's one key F5-run, with one click return to the 
spot of the foul on syntax errors.

> OTOH, development is likely to be more vigorous if it's separate.

Perhaps, perhaps not, or perhaps it would become 'too' vigorous if too 
many developers pushed multiple 'kitchen sinks'.

> I'd also like to make a plea to keep IDLE's interface clean and basic.
> There are lots of complex IDEs available for those who want them.  It's
> natural for developers to add features, that's what they do :-), but you
> don't hand a novice a Ferrari (or emacs) and expect good results.  IMHO
> some of the feature patches on the tracker should be rejected on that
> basis.

Have you commented on those issues? I so far have mostly concentrated on 
fixing current features. I agree that major new features should be 
considered carefully and perhaps discussed on a revived idle-sig list. I 
have never used some of the existing features, like breakpoints, that 
seem pretty advanced. I first opened a debugger window only recently, in 
order to comment on a issue about a possible bug. We should document how 
to use that before adding anything else comparable.

> It's sometimes said that IDLE is "ugly" or "broken".  These terms are
> subjective!

When IDLE-closing bugs are all fixed, I would like to see how much 
difference themed widgets would make to appearance. Then we could debate 
whether IDLE should look 'native' on each platform or have a common 
'Python' theme -- or have both and let users choose.

> If it's truly broken, then we should fix it.  If it's
> "broken" because a feature is missing, maybe that's an intentional part
> of Guido's design of a simple Python IDE.

Without a vision and design document, it is sometimes hard for someone 
like me to know which is which.

-- 
Terry Jan Reedy


From nyamatongwe at me.com  Thu Mar 21 01:38:37 2013
From: nyamatongwe at me.com (Neil Hodgson)
Date: Thu, 21 Mar 2013 11:38:37 +1100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kidjg2$u22$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
Message-ID: <0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>

Terry Reedy:

> Broken (and quirky): it has an absurdly limited output buffer (under a thousand lines)

   The limit is actually 9999 lines.

> Quirky: Windows uses cntl-C to copy selected text to the clipboard and (where appropriate) cntl-V to insert clipboard text at the cursor pretty much everywhere.

   CP uses Ctrl+C to interrupt programs similar to Unix. Therefore it moves copy to a different key in a similar way to Unix consoles like GNOME Terminal and MATE Terminal which use Shift+Ctrl+C for copy despite Ctrl+C being the standard for other applications.

   Neil


From cben at users.sf.net  Thu Mar 21 03:30:10 2013
From: cben at users.sf.net (Beni Paskin-Cherniavsky)
Date: Thu, 21 Mar 2013 02:30:10 +0000 (UTC)
Subject: [Python-Dev] IDLE in the stdlib
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<BB1D6024-1AEC-4C7F-844E-D7699D29B598@masklinn.net>
	<CAF-Rda-wTo_KF8swymc3FKjegVFcpsHfEKifsqE_r_KfLd1grg@mail.gmail.com>
Message-ID: <loom.20130321T015147-345@post.gmane.org>

Eli Bendersky <eliben <at> gmail.com> writes:

> Oh, and another thing. If a Windows user wants a good Python shell,
> IDLE should be his last choice. There's Spyder, there's IPython,
> there's probably a bunch of others I'm not aware of.This is for IDLE
> as a shell. The same can be said for IDLE as an editor.This is
> precisely my main gripe with IDLE: it does a lot of things, but
> neither of them it does well.

Actually, there are surprisingly little competition to IDLE as a shell!
IDLE has mostly working multi-line editing and history, while most 
"sophisticated" environments (including Spyder) work line-by-line, which makes  
defining a function (let alone a class) in the shell prohibitively painful.

The only other shells I could recommend to a beginner are:
1. IPython, which of course does multi-line editing superbly.
   Its non-standard extensions are a distraction and it's too far into 
   power-user end of the spectrum to ever become a fits-all recommendation.
   The notebook is enough of a win to tip the scales for some educators,
   but the jury is still out if that's a smooth beginner experience.
2. Dreampie, designed by an ex-IDLE-contributor for the sole purpose of 
   being a better shell than IDLE.
   However, lack of an editor makes it less practical as an introductory tool.



From tjreedy at udel.edu  Thu Mar 21 03:30:33 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Mar 2013 22:30:33 -0400
Subject: [Python-Dev] A 'common' respository? (was Re: IDLE in the stdlib)
In-Reply-To: <kidjg2$u22$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
Message-ID: <kidrce$u7k$1@ger.gmane.org>

On 3/20/2013 8:15 PM, Terry Reedy wrote:

> I will discuss repository separation in another response

Here is a radical idea I have been toying with: set up a 'common' 
repository to 'factor out' files that are, could be, or should be the 
same across versions. The 'common' files would be declared (especially 
to packagers, when relevant) to be a part of each branch. Each release 
would (somehow - not my department) incorporate the latest version of 
everything in 'common'.

What would go here?

Misc/ACKS: the sensible idea that there should only be one copy of this 
file has been discussed before.

LICENSE: I believe this is the same across current versions and must be 
edited in parallel for all future branches.

xxx: others that I have not thought of.

Doc/tools (sphinx and dependencies): setting this up separately but 
identically for each branch is a bit silly if it could be avoided. The 
sphinx versions should, of course, be the new one that runs on both 
python 2 and 3.

idlelib: already discussed. Having only one IDLE version would partially 
speed up development.

(surely controvesial) tkinter and _tkinter: I think the _tkinter and 
tkinter for each release should work with and be tested with the most 
recent tcl/tk release. Having only one tkinter version might make having 
one version of IDLE even easier.

(probably even more controversial) tcl/tk (or at least the files needed 
to fetch and build - but as long as the sources are on python.org 
anyway, the sources could also be moved here from svn): For IDLE to 
really work the same across versions, it needs to run on the same tcl/tk 
version with the same bugfixes. For example, over a year ago, a French 
professor wrote python-list or idle-sig or maybe both saying that he 
would like to use IDLE in a class in Sept 2012, but there was a bug 
keeping it from working properly with French keyboards. He wanted to 
know if we were likely to fix it. The first answer (provided by Kevin 
Walzer) was that it was a tcl/tk bug that he (Kevin) was working on. The 
fix made it into 8.5.9 a year ago and hence into 3.3 but 2.7.3 or 3.2.3, 
released a month after the fix. So I later told him he could use IDLE, 
but, at least on Windows, only with the then upcoming 3.3. (I don't know 
the tcl/tk version policy for the non-Apple builds.)

I do not know if tcl/tk 8.5.z releases have added many features or are 
primarily bugfixes like our micro releases. If the latter, the case for 
distributing at least the most recent 8.5.z with windows would seem 
pretty strong. I also do not know what 8.6.z adds. But an tcl/tk 
'enhancement' of supporting astral characters might look like a bugfix 
for IDLE. (Running from IDLE, print(astral_char) raises, but I believe 
the same code works in some Linux interpreters.)

yyy: any other external dependencies that we update on all versions.

---
Terry Jan Reedy


From raymond.hettinger at gmail.com  Thu Mar 21 03:57:54 2013
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Wed, 20 Mar 2013 19:57:54 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320123833.14886a86@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
Message-ID: <8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>


On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org> wrote:

> Right.  Ultimately, I think IDLE should be a separate project entirely, but I
> guess there's push back against that too.

The most important feature of IDLE is that it ships with the standard library.
Everyone who clicks on the Windows MSI on the python.org webpage
automatically has IDLE.   That is why I frequently teach Python with IDLE.

If this thread results in IDLE being ripped out of the standard distribution,
then I would likely never use it again.

> JFDI! :)


That is a comment from a person who uses Emacs every day.
For those of us who have to support people with basic installs,
it is essential that they have some Python aware editor on their
machine.   Without IDLE, a shocking number of people would
create Python files using notepad.


Raymond



-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/5b2f4148/attachment.html>

From steve at pearwood.info  Thu Mar 21 03:53:44 2013
From: steve at pearwood.info (Steven D'Aprano)
Date: Thu, 21 Mar 2013 13:53:44 +1100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kidjg2$u22$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
Message-ID: <514A7638.4090205@pearwood.info>

On 21/03/13 11:15, Terry Reedy wrote:

> getting back to CP versus IDLE...
>
>  From IDLE:
>>>> print('\x80')
> ?
>>>> print('\xc8')
> ?
>
> Impressed? You should be. Open Start menu / Python33 / Python (command line) and both of those result (modulo the specific character) in
>    UnicodeEncodeError: 'charmap' codec can't encode character
>    '\xc8' in position 0: character maps to <undefined>


Terry, you have just done something I didn't think was possible: you've changed my personal opinion about IDLE. On the rare, rare occasions where I've had to use Python interactively on Windows, I use the standard python.exe command prompt, which I thought was easier than learning the (to me) quirks of IDLE's UI.

You've just given me a reason to use IDLE.

I also note that in the last few weeks, I've seen at least two instances that I recall of a beginner on the tutor at python.org mailing list being utterly confused by Python's Unicode handling because the Windows command prompt is unable to print Unicode strings.


Thanks Terry.


-- 
Steven

From dreamingforward at gmail.com  Thu Mar 21 04:03:28 2013
From: dreamingforward at gmail.com (Mark Janssen)
Date: Wed, 20 Mar 2013 20:03:28 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
Message-ID: <CAMjeLr8q6R3X0xkuTdmv3QFgogNpQxPKDuVR6VVaxx3bKo4tKQ@mail.gmail.com>

On Wed, Mar 20, 2013 at 7:57 PM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:

>
> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org> wrote:
>
> Right.  Ultimately, I think IDLE should be a separate project entirely,
> but I
> guess there's push back against that too.
>
>
> The most important feature of IDLE is that it ships with the standard
> library.
> Everyone who clicks on the Windows MSI on the python.org webpage
> automatically has IDLE.   That is why I frequently teach Python with IDLE.
>
> If this thread results in IDLE being ripped out of the standard
> distribution,
> then I would likely never use it again.
>
>
+1, FWIW

MarkJ
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/8fe2d6c9/attachment.html>

From tjreedy at udel.edu  Thu Mar 21 04:32:38 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed, 20 Mar 2013 23:32:38 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
Message-ID: <kidv0r$tcv$1@ger.gmane.org>

On 3/20/2013 12:41 PM, Eli Bendersky wrote:

> Personally, I think that IDLE reflects badly on Python in more ways than
> one. It's badly maintained, quirky and ugly.

Ugly is subjective: by what standard and compared to what?

I suggested in my previous response why I think 'badly maintained' is 
untrue and/or unfair. Dismissing the recent work that has been done does 
not help.

There are 20 open issues with smtp(lib) in the title. It is 37 kb, 
making .54 issues per kb. For idlelib, with 786 kb, there are 104 
issues, or .13 issues per kb, which is one fourth as many. I could claim 
that smtplib, based on 1990s RFCs is much worse maintained. It certainly 
could use somee positive attention.

What current quirks, not already the subject of a tracker issue, are you 
thinking of?

 > It serves a very narrow set of uses,

Relative to the computing universe, yes. It focuses on editing and 
running Python code.

 > and does it badly.

As a user, I rate it at least 'good'. Most of the tracker issues hardly 
affect me, and many or most of the worst problems for me have already 
been fixed. What IDE would you suggest as a simple, install and go, 
alternative? It should have the following features or something close:
* One-key saves the file and runs it with the -i option (enter 
interactive mode after running the file) so one can enter additional 
statements interactively.
* Syntax errors cause a message display; one click returns to the spot 
the error was detected.
* Error tracebacks are displayed unmodified, without extra garbage or 
censorship.
# Right click on a line like
  File "C:\Programs\Python33\lib\difflib.py", line 1759, ...
and then left click on the goto popup to go to that line in that file, 
opening the file if necessary.

As of 3.3.0, this last feature was not documented, at least not in the 
Idle Help file. Since then, it has been.

-- 
Terry Jan Reedy


From eliben at gmail.com  Thu Mar 21 04:36:39 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 20:36:39 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
Message-ID: <CAF-Rda8n9GLkCifPtO2HR-405fkucJxZKXtSqhuieFy=AYNfHA@mail.gmail.com>

On Wed, Mar 20, 2013 at 7:57 PM, Raymond Hettinger <
raymond.hettinger at gmail.com> wrote:

>
>
> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org> wrote:
>
> Right.  Ultimately, I think IDLE should be a separate project entirely,
> but I
> guess there's push back against that too.
>
>
> The most important feature of IDLE is that it ships with the standard
> library.
> Everyone who clicks on the Windows MSI on the python.org webpage
> automatically has IDLE.   That is why I frequently teach Python with IDLE.
>
> If this thread results in IDLE being ripped out of the standard
> distribution,
> then I would likely never use it again.
>

Why is it necessary to conflate distribution and development. "standard
library" != "Python distribution".

Take the ActivePython distribution for example. They ship with extra
packages for Windows (pywin32, etc) and our Python installer doesn't. This
is a reason many Windows people prefer ActivePython. That's their right,
but this preference is not the point. The point is that it's perfectly
conceivable to ship IDLE with Python releases on Windows, while managing it
as a separate project outside the CPython core Mercurial repository.

This seems to me to combine benefits from both worlds:

1. IDLE keeps being shipped to end users. I have to admit the reasons made
in favor of this in the thread so far are convincing.
2. IDLE is developed as a standalone project. As such, it's much easier to
contribute to, which will hopefully result in a quicker pace of
improvement. The only demand is that it keeps working with a release
version of Python, and this is pretty easy. It's even possible and easy to
have a single IDLE version for Python 3.x instead of contributors having to
propose patches for 3.2, 3.3 and 3.4 simultaneously.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/1eb4766c/attachment.html>

From eliben at gmail.com  Thu Mar 21 04:54:25 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Wed, 20 Mar 2013 20:54:25 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kidv0r$tcv$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
Message-ID: <CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>

On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 3/20/2013 12:41 PM, Eli Bendersky wrote:
>
>  Personally, I think that IDLE reflects badly on Python in more ways than
>> one. It's badly maintained, quirky and ugly.
>>
>
> Ugly is subjective: by what standard and compared to what?
>
>
Compared to other existing Python IDEs and shells which are layered on top
of modern GUI toolkits that are actively developed to keep with modern
standards, unlike Tk which is frozen in the 1990s.


> I suggested in my previous response why I think 'badly maintained' is
> untrue and/or unfair. Dismissing the recent work that has been done does
> not help.
>

I did not intend to dismiss your work Terry, and I'm sorry if it came out
this way. You know that I also contributed bug fixes to IDLE in the past so
I'm not a complete outsider. I see the value of IDLE being distributed with
Python. However, especially in view of the recent developments in the area
of alternative Python implementations, I think it's important to clearly
mark the boundaries between things that belong in the core CPython code
repository and things that don't.

>
> There are 20 open issues with smtp(lib) in the title. It is 37 kb, making
> .54 issues per kb. For idlelib, with 786 kb, there are 104 issues, or .13
> issues per kb, which is one fourth as many. I could claim that smtplib,
> based on 1990s RFCs is much worse maintained. It certainly could use somee
> positive attention.
>

You know better than I do that the number of open issues is not really the
only factor for determining the quality of a module.



Eli

--


> What current quirks, not already the subject of a tracker issue, are you
> thinking of?
>
>
> > It serves a very narrow set of uses,
>
> Relative to the computing universe, yes. It focuses on editing and running
> Python code.
>
> > and does it badly.
>
> As a user, I rate it at least 'good'. Most of the tracker issues hardly
> affect me, and many or most of the worst problems for me have already been
> fixed. What IDE would you suggest as a simple, install and go, alternative?
> It should have the following features or something close:
> * One-key saves the file and runs it with the -i option (enter interactive
> mode after running the file) so one can enter additional statements
> interactively.
> * Syntax errors cause a message display; one click returns to the spot the
> error was detected.
> * Error tracebacks are displayed unmodified, without extra garbage or
> censorship.
> # Right click on a line like
>  File "C:\Programs\Python33\lib\**difflib.py", line 1759, ...
> and then left click on the goto popup to go to that line in that file,
> opening the file if necessary.
>
> As of 3.3.0, this last feature was not documented, at least not in the
> Idle Help file. Since then, it has been.
>
>


> --
> Terry Jan Reedy
>
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> eliben%40gmail.com<http://mail.python.org/mailman/options/python-dev/eliben%40gmail.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/50470ab9/attachment-0001.html>

From kbk at shore.net  Thu Mar 21 05:32:50 2013
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu, 21 Mar 2013 00:32:50 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kidn2q$t5s$1@ger.gmane.org>
References: <E1UIRP3-0004SF-PR@smtp3.shore.net>
	<1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>
	<kidn2q$t5s$1@ger.gmane.org>
Message-ID: <1363840370.26832.140661207112409.77D53C31@webmail.messagingengine.com>



On Wed, Mar 20, 2013, at 09:17 PM, Terry Reedy wrote:
> On 3/20/2013 6:48 PM, Kurt B. Kaiser wrote:
>
> Well, one can hardly use Command Prompt captures, unless one were to
> flip black and white within the window (but not its frame).

And yet many books do - it's really ugly.  When I see a book with a
bunch of DOS windows, white on black, printed on cheap paper, 900 pages,
I just put it back on the shelf.  Quickly.


> > I've always felt that IDLE should be targeted to an eight year old
> > beginner, and should work uniformly across the major platforms. That
> > now includes the Raspberry Pi!!
>
> I think it should also work uniformly across Python versions. That is
> the gist of PEP434.

Well, spending a lot of time backporting new features is not my idea of
fun. OTOH, I have no objection.  Maybe we could automate checkpoints of
IDLE into PyPI or somewhere for those who don't care to pull from hg,
yet want to be on the cutting  edge?

Along those lines, I've thought that IDLE should refrain from using the
newest features in Python, to allow people running released versions of
Python to access the newest IDLE.  i.e. Python 3.4 innovations would not
be used in IDLE 3.4.  Only Python 3.3 and earlier innovations.

[...]

> > OTOH, development is likely to be more vigorous if it's separate.
>
> Perhaps, perhaps not, or perhaps it would become 'too' vigorous if too
> many developers pushed multiple 'kitchen sinks'.

Oh, I agree with that.  Distributed version control can lead to chaos
and dilution of effort.

I don't know if this is the place to comment on PEP 434 (is it?), but
I've always taken the approach that IDLE development should be less
formal than the rest of stdlib, since it's not a dependency.  Big
changes should be reserved for the tip, but I don't see why something
like the right click menu change shouldn't be backported.

I think we should try a more relaxed idlelib development process inside
core before we move it out, and should be generous about adding checkin
permissions for that purpose.  Rietveld will help.  It's a good way to
habilitate new developers.

>
> > I'd also like to make a plea to keep IDLE's interface clean and
> > basic. There are lots of complex IDEs available for those who want
> > them.  It's natural for developers to add features, that's what they
> > do :-), but you don't hand a novice a Ferrari (or emacs) and expect
> > good results.  IMHO some of the feature patches on the tracker
> > should be rejected on that basis.
>
> Have you commented on those issues? I so far have mostly concentrated
> on fixing current features. I agree that major new features should be
> considered carefully and perhaps discussed on a revived idle-sig list.
> I have never used some of the existing features, like breakpoints,
> that seem pretty advanced. I first opened a debugger window only
> recently, in order to comment on a issue about a possible bug. We
> should document how to use that before adding anything else
> comparable.

I have commented over the years, but lately I've been so distracted by
the Treasurer job that I haven't found much time for IDLE.  And as I was
telling Ned here at PyCon, when you step off the train, it can be hard
to get back on.

I've tried to channel Guido over the years.  I looked at what he did,
and tried to project forward.

IDLE-dev is still active.  Would anyone else like to be a moderator?


> > It's sometimes said that IDLE is "ugly" or "broken".  These terms
> > are subjective!
>
> When IDLE-closing bugs are all fixed, I would like to see how much
> difference themed widgets would make to appearance. Then we could
> debate whether IDLE should look 'native' on each platform or have a
> common 'Python' theme -- or have both and let users choose.
>
> > If it's truly broken, then we should fix it.  If it's "broken"
> > because a feature is missing, maybe that's an intentional part of
> > Guido's design of a simple Python IDE.
>
> Without a vision and design document, it is sometimes hard for someone
> like me to know which is which.

IDLE development has always been organic, as opposed to the formal
approach of the PEPs.  What we need is a Zen of IDLE.  When I look at
it, I see a simple IDE.  I try to adhere to the principle of least
surprise, and to maintain an uncluttered interface suitable for
beginners.  Expert features can be there, but somewhat hidden (though
they should be documented!) or implemented as disabled extensions.

IDLE tries to promote Pythonic style.  For example, the lack of a
horizontal scroll bar was deliberate, I think.

We should implement the patch that adds an extension selector to the
options dialog, and keep the expert features as disabled extensions.

That way, an instructor could distribute an idlerc file which would set
IDLE up exactly as desired, including links to course specific help
files on the web.

BTW, I'll take this chance to promote the use of idlelib/NEWS.txt for
IDLE news, instead of Misc/NEWS.  That way, if IDLE is used outside of
the main release, the NEWS.txt will go with it.


Stability is paramount.

No surprises beats cool.

KISS is better than full featured.

Uniform trumps native look.

Experts go to the rear.

Where they can find what they want if they know where to look.

IDLE tests tkinter.

-- 
KBK

From v+python at g.nevcal.com  Thu Mar 21 05:36:05 2013
From: v+python at g.nevcal.com (Glenn Linderman)
Date: Wed, 20 Mar 2013 21:36:05 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kidjg2$u22$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
Message-ID: <514A8E35.8090204@g.nevcal.com>

On 3/20/2013 5:15 PM, Terry Reedy wrote:
> Broken (and quirky): it has an absurdly limited output buffer (under a 
> thousand lines)

People keep claiming that Windows CMD has a limited output buffer. It is 
configurable, at least to 9999 lines, which is where I have mine set. 
That is far too much to actually scroll back through for most practical 
purposes, although sometimes I do :)

I'm not trying to claim that Windows CMD is wonderful, perfect, or has 
large numbers of redeeming values, but let's keep to the facts.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/902aca85/attachment.html>

From tjreedy at udel.edu  Thu Mar 21 06:19:42 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 01:19:42 -0400
Subject: [Python-Dev] cpython: Issue #13248: removed deprecated and
 undocumented difflib.isbjunk, isbpopular.
In-Reply-To: <20130320181320.5B45C2500B3@webabinitio.net>
References: <3ZVrTH23v4zPCm@mail.python.org>
	<CAF-Rda_-rXSs2VVZM6eht=ZP2aq2rgqNj+M4q7Zw4h7Lu-hsew@mail.gmail.com>
	<20130320181320.5B45C2500B3@webabinitio.net>
Message-ID: <kie59j$cpc$1@ger.gmane.org>

On 3/20/2013 2:13 PM, R. David Murray wrote:
> On Wed, 20 Mar 2013 05:23:43 -0700, Eli Bendersky <eliben at gmail.com> wrote:
>> A mention in Misc/NEWS can't hurt here, Terry. Even though it's
>> undocumented, some old code could rely on it being there and this code will
>> break with the transition to 3.4

Will do.

> Note that we also have a list of deprecated things that were removed in
> What's New.
>
> Aside: given the 3.3 experience, I think people should be thinking in
> terms of always updating What's New when appropriate, at the time a
> commit is made.

How does this look? Is ``replacement`` right? Should the subsequent sm 
by itself be marked? If so, how?

* :meth:`difflib.SequenceMatcher.isbjunk` and
   :meth:`difflib.SequenceMatcher.isbpopular`: use ``x in sm.bjunk`` and
   ``x in sm.bpopular``, where sm is a SequenceMatcher object.

-- 
Terry Jan Reedy


From pjj at philipjohnjames.com  Thu Mar 21 07:06:31 2013
From: pjj at philipjohnjames.com (Philip James)
Date: Wed, 20 Mar 2013 23:06:31 -0700
Subject: [Python-Dev] A 'common' respository? (was Re: IDLE in the
	stdlib)
In-Reply-To: <kidrce$u7k$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org> <kidrce$u7k$1@ger.gmane.org>
Message-ID: <CACXM1yFG4_4K4cFQN0SMU51+Zj9q+9PDJvW5EDczte-jR7AG=A@mail.gmail.com>

I hope I'm not coming across as pedantic, because I think you have some
good arguments listed above, but shouldn't discussion like this go in
python-ideas rather than python-dev? I'm very new to these lists, so
forgive me if I'm stepping on any toes, I'm just trying to grok what kind
of content should go in each list.

PJJ
http://philipjohnjames.com


On Wed, Mar 20, 2013 at 7:30 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 3/20/2013 8:15 PM, Terry Reedy wrote:
>
>  I will discuss repository separation in another response
>>
>
> Here is a radical idea I have been toying with: set up a 'common'
> repository to 'factor out' files that are, could be, or should be the same
> across versions. The 'common' files would be declared (especially to
> packagers, when relevant) to be a part of each branch. Each release would
> (somehow - not my department) incorporate the latest version of everything
> in 'common'.
>
> What would go here?
>
> Misc/ACKS: the sensible idea that there should only be one copy of this
> file has been discussed before.
>
> LICENSE: I believe this is the same across current versions and must be
> edited in parallel for all future branches.
>
> xxx: others that I have not thought of.
>
> Doc/tools (sphinx and dependencies): setting this up separately but
> identically for each branch is a bit silly if it could be avoided. The
> sphinx versions should, of course, be the new one that runs on both python
> 2 and 3.
>
> idlelib: already discussed. Having only one IDLE version would partially
> speed up development.
>
> (surely controvesial) tkinter and _tkinter: I think the _tkinter and
> tkinter for each release should work with and be tested with the most
> recent tcl/tk release. Having only one tkinter version might make having
> one version of IDLE even easier.
>
> (probably even more controversial) tcl/tk (or at least the files needed to
> fetch and build - but as long as the sources are on python.org anyway,
> the sources could also be moved here from svn): For IDLE to really work the
> same across versions, it needs to run on the same tcl/tk version with the
> same bugfixes. For example, over a year ago, a French professor wrote
> python-list or idle-sig or maybe both saying that he would like to use IDLE
> in a class in Sept 2012, but there was a bug keeping it from working
> properly with French keyboards. He wanted to know if we were likely to fix
> it. The first answer (provided by Kevin Walzer) was that it was a tcl/tk
> bug that he (Kevin) was working on. The fix made it into 8.5.9 a year ago
> and hence into 3.3 but 2.7.3 or 3.2.3, released a month after the fix. So I
> later told him he could use IDLE, but, at least on Windows, only with the
> then upcoming 3.3. (I don't know the tcl/tk version policy for the
> non-Apple builds.)
>
> I do not know if tcl/tk 8.5.z releases have added many features or are
> primarily bugfixes like our micro releases. If the latter, the case for
> distributing at least the most recent 8.5.z with windows would seem pretty
> strong. I also do not know what 8.6.z adds. But an tcl/tk 'enhancement' of
> supporting astral characters might look like a bugfix for IDLE. (Running
> from IDLE, print(astral_char) raises, but I believe the same code works in
> some Linux interpreters.)
>
> yyy: any other external dependencies that we update on all versions.
>
> ---
> Terry Jan Reedy
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> pjj%40philipjohnjames.com<http://mail.python.org/mailman/options/python-dev/pjj%40philipjohnjames.com>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130320/2836720a/attachment.html>

From tjreedy at udel.edu  Thu Mar 21 07:18:05 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 02:18:05 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
	<0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
Message-ID: <kie8n3$774$1@ger.gmane.org>

On 3/20/2013 8:38 PM, Neil Hodgson wrote:
> Terry Reedy:
>
>> Broken (and quirky): it has an absurdly limited output buffer
>> (under a thousand lines)
>
> The limit is actually 9999 lines.

I clicked Start / All programs / Python 3.3 / Python (command line)
 >>> help(str)
<space> (several times)
and scrolled back up and the result was as I described. Help was gone 
above the 'c' methods. That is not 9999 lines.

I believe I later specified 'as installed'. But you are right. If one 
knows to right click on the blue and yellow Python snake, select 
Properties, Layout, and find Screen Buffer Size and Height, then one can 
increase the miserly default of 300 to 9999, at least on Win 7. 'Under a 
thousand lines' may be a vague memory from XP. I am also sure that with 
XP, the settings would revert for non-admin users after closing. Maybe 
MS did upgrade Command Prompt a bit.

Oh, but we are not done with the stupidity of Command Prompt. If one 
does set the buffer to 9999 lines, it pads the output to 9999 lines and 
shrinks the movable scroll bar down to an eighth inch. If you move it. 
say, 1/20 of the screen, you jump 500 lines, which initially is way past 
your actually output. The standard 'modern' convenience of dynamically 
resized buffers and bars, such as found in the nearly 20 year old 
Notepad, is not for CP. (There is an idea for MS: junk CP and re-build a 
modern version on top of Notepad.)

Setting properties by right clicking the icon is not standard on 
Windows. There is no help available from the window that I could find. I 
also could not find anything about the properties dialog in Windows 
help. If you can find an official entry for 'QuickEdit Mode' and 'Insert 
Mode', please let me know. Python Setup and Usage also says nothing 
about using the Command Prompt interpreter.

-- 
Terry Jan Reedy


From g.brandl at gmx.net  Thu Mar 21 07:34:37 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 21 Mar 2013 07:34:37 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130320164729.715c9e10@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
	<20130320125445.64676161@anarchist>
	<CADiSq7cuJwrzWKSTm-KtC6L-Jk44aWTGk3zG5gEp8Fivz1OOxQ@mail.gmail.com>
	<20130320231147.665b3adc@pitrou.net>
	<20130320164729.715c9e10@anarchist>
Message-ID: <kie9lb$e6t$1@ger.gmane.org>

Am 21.03.2013 00:47, schrieb Barry Warsaw:
> On Mar 20, 2013, at 11:11 PM, Antoine Pitrou wrote:
> 
>>On Wed, 20 Mar 2013 15:05:40 -0700
>>Nick Coghlan <ncoghlan at gmail.com> wrote:
>>> 
>>> Yes, the status quo of Idle is not something we should allow to
>>> continue indefinitely, but decisions about its future development
>>> should be made by active maintainers that are already trusted to make
>>> changes to it (such as Terry and Roger), rather than those of us that
>>> don't use it, and aren't interested in maintaining it.
>>
>>Definitely. People shouldn't remain quiescently torpid about the idle
>>status quo.
> 
> The release managers should have a say in the matter, since it does cause some
> amount of pain there.

I don't really understand what Antoine's "quiescently torpid" means, but
splitting IDLE out to a separate repo and then merging it back every time
a release rolls around sounds stupid.

Either split it off completely or develop it here (my preferred solution).
It's really not that hard to get CPython commit bits.

Georg


From tjreedy at udel.edu  Thu Mar 21 07:42:33 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 02:42:33 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
Message-ID: <kiea4v$iar$1@ger.gmane.org>

On 3/20/2013 11:54 PM, Eli Bendersky wrote:
> On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu

>     Ugly is subjective: by what standard and compared to what?
>
> Compared to other existing Python IDEs and shells which are layered on
> top of modern GUI toolkits that are actively developed to keep with
> modern standards, unlike Tk which is frozen in the 1990s.

I think being frozen in the late 1990s is better than being frozen in 
the early 1980s, like Command Prompt is. In fact, I think we should 
'deprecate' the Command Prompt interpreter as the standard interactive 
interpreter and finish polishing and de-glitching IDLE's Python Shell, 
which runs on top of the windowless version of CP with a true GUI. Then 
we can promote and present the latter as the preferred interface, which 
for many people, it already is.

>     There are 20 open issues with smtp(lib) in the title. It is 37 kb,
>     making .54 issues per kb. For idlelib, with 786 kb, there are 104
>     issues, or .13 issues per kb, which is one fourth as many. I could
>     claim that smtplib, based on 1990s RFCs is much worse maintained. It
>     certainly could use somee positive attention.

Repeat: based on the 1990s RFCs, needing to be updated to the 2008 RFC, 
itself in the process of being superseded by a more unicode aware RFC.

> You know better than I do that the number of open issues is not really
> the only factor for determining the quality of a module.

And you should notice that I did not present that as the only factor for 
what I said I *could* claim. Actually, I think the comparison would be 
fairer if enhancements were not counted. I am pretty sure this would 
favor IDLE even more (depending on what one counted as a bug).

Let me repeat this question.

What IDE might be a simple, install and go, alternative to IDLE that I
might investigate, even if just as a source of ideas for IDLE?

 >     It should have the following features or something close:
>     * One-key saves the file and runs it with the -i option (enter
>     interactive mode after running the file) so one can enter additional
>     statements interactively.
>     * Syntax errors cause a message display; one click returns to the
>     spot the error was detected.
>     * Error tracebacks are displayed unmodified, without extra garbage
>     or censorship.
>     # Right click on a line like
>       File "C:\Programs\Python33\lib\__difflib.py", line 1759, ...
>     and then left click on the goto popup to go to that line in that
>     file, opening the file if necessary.

-- 
Terry Jan Reedy


From jeanpierreda at gmail.com  Thu Mar 21 07:54:01 2013
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Thu, 21 Mar 2013 02:54:01 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kiea4v$iar$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
Message-ID: <CABicbJ+uRE+u8trzu8NsFroHxSnStMum-P7-yV3_dDjekhTK-g@mail.gmail.com>

On Thu, Mar 21, 2013 at 2:42 AM, Terry Reedy <tjreedy at udel.edu> wrote:
> I think being frozen in the late 1990s is better than being frozen in the
> early 1980s, like Command Prompt is. In fact, I think we should 'deprecate'
> the Command Prompt interpreter as the standard interactive interpreter and
> finish polishing and de-glitching IDLE's Python Shell, which runs on top of
> the windowless version of CP with a true GUI. Then we can promote and
> present the latter as the preferred interface, which for many people, it
> already is.

Please don't cease supporting the command line interface. I use the
command line interactive interpreter plenty. That way I can use git,
grep, the unit test suite, etc. ... and the interactive interpreter,
all from one place: the console.

That can't happen with IDLE, by design.

-- Devin

From tjreedy at udel.edu  Thu Mar 21 08:03:34 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 03:03:34 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <514A8E35.8090204@g.nevcal.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org> <514A8E35.8090204@g.nevcal.com>
Message-ID: <kiebcb$sc4$1@ger.gmane.org>

On 3/21/2013 12:36 AM, Glenn Linderman wrote:
> On 3/20/2013 5:15 PM, Terry Reedy wrote:
>> Broken (and quirky): it has an absurdly limited output buffer (under a
>> thousand lines)
>
> People keep claiming that Windows CMD has a limited output buffer. It is
> configurable, at least to 9999 lines, which is where I have mine set.
> That is far too much to actually scroll back through for most practical
> purposes, although sometimes I do :)

See my response to the same point by Neil Hodgson, where I noticed that 
setting to 9999 lines somewhat disables easy scrolling. (I checked and 
it was 9999 also on XP.)

> I'm not trying to claim that Windows CMD is wonderful, perfect, or has
> large numbers of redeeming values, but let's keep to the facts.

Yes, lets do. I tried to. A person who installs Python on Windows and 
runs Python (command prompt) instead of IDLE is confronted with a 300 
line default. Someone not familiar with Command Prompt will not know 
that the limit can be increased. I use CP so seldom, until recently, 
that I had forgotten how to do so. I fiddled with the history buffer 
setting on the first page. That did not work so I gave up.

-- 
Terry Jan Reedy


From larry at hastings.org  Thu Mar 21 08:42:14 2013
From: larry at hastings.org (Larry Hastings)
Date: Thu, 21 Mar 2013 00:42:14 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kie9lb$e6t$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<CAP7+vJLdgtbZEFAAhc5u+3b8u4gW+f2Ke9QuBsGq+9p_YjnAzw@mail.gmail.com>
	<20130320125445.64676161@anarchist>
	<CADiSq7cuJwrzWKSTm-KtC6L-Jk44aWTGk3zG5gEp8Fivz1OOxQ@mail.gmail.com>
	<20130320231147.665b3adc@pitrou.net>
	<20130320164729.715c9e10@anarchist> <kie9lb$e6t$1@ger.gmane.org>
Message-ID: <514AB9D6.3070406@hastings.org>

On 03/20/2013 11:34 PM, Georg Brandl wrote:
> I don't really understand what Antoine's "quiescently torpid" means,

"quiescent" = "peaceful, quiet, still"
"torpid" = "lethargic, not moving"
"antoine" = "thesaurus owner"


//arry/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/a3fce199/attachment.html>

From tjreedy at udel.edu  Thu Mar 21 08:59:52 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 03:59:52 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <1363840370.26832.140661207112409.77D53C31@webmail.messagingengine.com>
References: <E1UIRP3-0004SF-PR@smtp3.shore.net>
	<1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>
	<kidn2q$t5s$1@ger.gmane.org>
	<1363840370.26832.140661207112409.77D53C31@webmail.messagingengine.com>
Message-ID: <kieelu$p1c$1@ger.gmane.org>

On 3/21/2013 12:32 AM, Kurt B. Kaiser wrote:

> Well, spending a lot of time backporting new features is not my idea of
> fun. OTOH, I have no objection.

I intentionally did not say in the PEP that it should be mandatory.

> Along those lines, I've thought that IDLE should refrain from using the
> newest features in Python, to allow people running released versions of
> Python to access the newest IDLE.  i.e. Python 3.4 innovations would not
> be used in IDLE 3.4.  Only Python 3.3 and earlier innovations.

So far, since 3.0/1, that has been a moot point. 3.2 was syntax frozen. 
3.3 has 'yield from', but I do not know if there is much of anywhere 
that *could* be used, and for simple uses, the old 'for i in it: yield 
i; is good enough.

> I don't know if this is the place to comment on PEP 434 (is it?), but
> I've always taken the approach that IDLE development should be less
> formal than the rest of stdlib, since it's not a dependency.  Big
> changes should be reserved for the tip, but I don't see why something
> like the right click menu change shouldn't be backported.

Thank you for confirming my impression of your approach. When the right 
click changed was challenged, I claimed that an *informal* relaxed 
approach was the defacto rule. Antoine said that that was not OK and 
Nick agreed and said that a relaxed approach would have to be 
*formalized* in a PEP and approved. Then there would be no (less ;-) 
arguments about IDLE pushes. Todd took the initiative to write a first 
draft and I revised it.

> I think we should try a more relaxed idlelib development process inside
> core before we move it out, and should be generous about adding checkin
> permissions for that purpose.  Rietveld will help.  It's a good way to
> habilitate new developers.

> I have commented over the years, but lately I've been so distracted by
> the Treasurer job that I haven't found much time for IDLE.

So I should be selective in specifically asking for comment.
>
> I've tried to channel Guido over the years.  I looked at what he did,
> and tried to project forward.
>
> IDLE-dev is still active.  Would anyone else like to be a moderator?

Yes. I presume that is the one mirrored as gmane.comp.pythong.idle. It 
has been mostly quiet since September. When we have an accepted ground 
rule for making decisions, I expect more discussions. For instance, I 
think it might be the appropriate place to get input from interested 
users about preferred behavior on a specific issue.

http://mail.python.org/mailman/listinfo/idle-dev
is out of date in places.

>> Without a vision and design document, it is sometimes hard for someone
>> like me to know which is which.
>
> IDLE development has always been organic, as opposed to the formal
> approach of the PEPs.  What we need is a Zen of IDLE.  When I look at
> it, I see a simple IDE.  I try to adhere to the principle of least
> surprise, and to maintain an uncluttered interface suitable for
> beginners.  Expert features can be there, but somewhat hidden (though
> they should be documented!) or implemented as disabled extensions.
>
> IDLE tries to promote Pythonic style.  For example, the lack of a
> horizontal scroll bar was deliberate, I think.
>
> We should implement the patch that adds an extension selector to the
> options dialog, and keep the expert features as disabled extensions.

In all versions, I presume ;-).

I have been wondering if there are any beginner features implemented as 
extensions that should be directly incorporated and planned to ask Roger 
on idle-dev. Anyway, I like that idea as it lets us somewhat have the 
cake of simplicity and also eat the cupcakes of expert features as desired.

> That way, an instructor could distribute an idlerc file which would set
> IDLE up exactly as desired, including links to course specific help
> files on the web.

That would be excellent.

> BTW, I'll take this chance to promote the use of idlelib/NEWS.txt for
> IDLE news, instead of Misc/NEWS.  That way, if IDLE is used outside of
> the main release, the NEWS.txt will go with it.

Thanks for the reminder. That came up here on pydev recently and it was
agreed that IDLE should indeed go in the idlelib/NEWS and that the 
release manager or someone could then *copy* it into Misc/NEWS as a 
separate *** IDLE *** section. Or maybe the idea was vice versa. Either 
was, this would avoid NEWS conficts for IDLE patches (unless multiple 
IDLE developers pushed nearly simultaneously).
http://bugs.python.org/issue17506

> Stability is paramount.
>
> No surprises beats cool.
>
> KISS is better than full featured.
>
> Uniform trumps native look.
>
> Experts go to the rear.
>
> Where they can find what they want if they know where to look.
>
> IDLE tests tkinter.

Thanks. I am glad I asked.

-- 
Terry Jan Reedy


From solipsis at pitrou.net  Thu Mar 21 10:20:23 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 21 Mar 2013 10:20:23 +0100
Subject: [Python-Dev] IDLE in the stdlib
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
Message-ID: <20130321102023.31b78c7f@pitrou.net>

Le Thu, 21 Mar 2013 02:42:33 -0400,
Terry Reedy <tjreedy at udel.edu> a ?crit :
> On 3/20/2013 11:54 PM, Eli Bendersky wrote:
> > On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu
> 
> >     Ugly is subjective: by what standard and compared to what?
> >
> > Compared to other existing Python IDEs and shells which are layered
> > on top of modern GUI toolkits that are actively developed to keep
> > with modern standards, unlike Tk which is frozen in the 1990s.
> 
> I think being frozen in the late 1990s is better than being frozen in 
> the early 1980s, like Command Prompt is. In fact, I think we should 
> 'deprecate' the Command Prompt interpreter as the standard
> interactive interpreter and finish polishing and de-glitching IDLE's
> Python Shell, which runs on top of the windowless version of CP with
> a true GUI.

And this may indeed be reasonable under Windows, where the command-line
is a PITA! But the Linux command-line is actually quite very usable
these days, especially if you configure your Python interpreter to use
readline for tab-completion of identifiers (which should be done by
default, see http://bugs.python.org/issue5845).

Regards

Antoine.



From p.f.moore at gmail.com  Thu Mar 21 10:27:40 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 21 Mar 2013 09:27:40 +0000
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
	<0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
Message-ID: <CACac1F-Tg8m0qCJeyBfLi-v+R11MHkpRfB3wSD5w_PZmfiJCog@mail.gmail.com>

On 21 March 2013 00:38, Neil Hodgson <nyamatongwe at me.com> wrote:
> Terry Reedy:
>
>> Broken (and quirky): it has an absurdly limited output buffer (under a thousand lines)
>
>    The limit is actually 9999 lines.
>
>> Quirky: Windows uses cntl-C to copy selected text to the clipboard and (where appropriate) cntl-V to insert clipboard text at the cursor pretty much everywhere.
>
>    CP uses Ctrl+C to interrupt programs similar to Unix. Therefore it moves copy to a different key in a similar way to Unix consoles like GNOME Terminal and MATE Terminal which use Shift+Ctrl+C for copy despite Ctrl+C being the standard for other applications.

Can I suggest that debates about the capability of Windows command
line programming are off-topic here? Whether it is good or bad (and in
my view, it is perfectly adequate, and in some ways better than Unix)
it is what Windows users who use the command line are used to. The
experience with Python is *identical* to what people see with other
scripting languages like Perl, Ruby, etc. It's even similar to
something like Java (I know everyone uses something like Eclipse for
Java, but that's a 3rd party download). And there's Python Tools for
Visual Studio if people want a "real Windows IDE"...

If people teaching Python have problems with the current environment
(and I know we've had some very good feedback on that score) then
that's fine, let's address it. But simply saying "Windows users have
no usable command line so they need GUI support" is neither productive
nor true.

(Apologies if this sounds grumpy. I'll go and get my first cup of tea
of the day now...)
Paul.

From solipsis at pitrou.net  Thu Mar 21 10:37:50 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 21 Mar 2013 10:37:50 +0100
Subject: [Python-Dev] IDLE in the stdlib
References: <E1UIRP3-0004SF-PR@smtp3.shore.net>
	<1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>
Message-ID: <20130321103750.311cb6f2@pitrou.net>

Le Wed, 20 Mar 2013 18:48:15 -0400,
"Kurt B. Kaiser" <kbk at shore.net> a ?crit :
> 
> IDLE has a single keystroke round trip - it's an IDE, not just an
> editor like Sublime Text or Notepad.  In the 21st century, people
> expect some sort of IDE.  Or, they should!

I don't think I've used an IDE in years (not seriously anyway).
I also don't think beginners "expect some sort of IDE", since they
don't know what it is. They probably don't even expect a text editor at
first.

> I'd also like to make a plea to keep IDLE's interface clean and
> basic. There are lots of complex IDEs available for those who want
> them.  It's natural for developers to add features, that's what they
> do :-), but you don't hand a novice a Ferrari (or emacs) and expect
> good results.

What is the point of an IDE without features?

Also, this is touching another issue: IDLE needs active maintainers,
who will obviously be experienced Python developers. But if they are
experienced Python developers, they will certainly want the additional
features, otherwise's they'll stop using and maintaining IDLE.

In other words, if IDLE were actually usable *and* pleasant for
experienced developers, I'm sure more developers would be motivated
to improve and maintain it.

> It's sometimes said that IDLE is "ugly" or "broken".  These terms are
> subjective!

Subjective statements are not baseless and idiotic. They come from the
experience of people actually wanting to like a piece of software, you
shouldn't discard them at face value.

Regards

Antoine.



From p.f.moore at gmail.com  Thu Mar 21 10:41:43 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 21 Mar 2013 09:41:43 +0000
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CABicbJ+uRE+u8trzu8NsFroHxSnStMum-P7-yV3_dDjekhTK-g@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CABicbJ+uRE+u8trzu8NsFroHxSnStMum-P7-yV3_dDjekhTK-g@mail.gmail.com>
Message-ID: <CACac1F83g0cKT9ON+1rRQ0UXrTQosRu9V-zU+jyFOOtyjGsqcQ@mail.gmail.com>

On 21 March 2013 06:54, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
> On Thu, Mar 21, 2013 at 2:42 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>> I think being frozen in the late 1990s is better than being frozen in the
>> early 1980s, like Command Prompt is. In fact, I think we should 'deprecate'
>> the Command Prompt interpreter as the standard interactive interpreter and
>> finish polishing and de-glitching IDLE's Python Shell, which runs on top of
>> the windowless version of CP with a true GUI. Then we can promote and
>> present the latter as the preferred interface, which for many people, it
>> already is.
>
> Please don't cease supporting the command line interface. I use the
> command line interactive interpreter plenty. That way I can use git,
> grep, the unit test suite, etc. ... and the interactive interpreter,
> all from one place: the console.
>
> That can't happen with IDLE, by design.

Agreed. Command line Python is 100% of my usage, and removing it would
make Python unusable for me.

If what is being suggested is removing the "Python Command Line"
*shortcuts* that are installed by default, but leaving the console
"python.exe" program, then I have no view on that, as I don't use
those shortcuts (and if I did, I could set them up myself). But before
removing them, why not consider setting the defaults to be more
helpful (larger scrollback buffer, things like quick edit set on, etc)
if that's the real issue here? I'm not saying it is, but some of the
complaints about "the default experience" *can* be fixed simply by
changing the defaults.

Paul.

From tjreedy at udel.edu  Thu Mar 21 11:07:03 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 06:07:03 -0400
Subject: [Python-Dev] A 'common' respository? (was Re: IDLE in the
	stdlib)
In-Reply-To: <CACXM1yFG4_4K4cFQN0SMU51+Zj9q+9PDJvW5EDczte-jR7AG=A@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org> <kidrce$u7k$1@ger.gmane.org>
	<CACXM1yFG4_4K4cFQN0SMU51+Zj9q+9PDJvW5EDczte-jR7AG=A@mail.gmail.com>
Message-ID: <kiem4c$v9k$1@ger.gmane.org>

On 3/21/2013 2:06 AM, Philip James wrote:
> I hope I'm not coming across as pedantic, because I think you have some
> good arguments listed above, but shouldn't discussion like this go in
> python-ideas rather than python-dev?

Normally yes. But since this is a counter-proposal or an alternate 
proposal to proposals already made in this thread, it belongs as an 
offshoot of this thread. This thread in turn is a continuation of 
similar threads here in the past, and involves some people who are less 
active in python-ideas. Also, it is more about technical development 
matters than about future *language* changes.

 > I'm very new to these lists, so
> forgive me if I'm stepping on any toes, I'm just trying to grok what
> kind of content should go in each list.

You are not stepping on my toes, and that is a good question to think about.

-- 
Terry Jan Reedy


From jsbueno at python.org.br  Thu Mar 21 11:08:33 2013
From: jsbueno at python.org.br (Joao S. O. Bueno)
Date: Thu, 21 Mar 2013 07:08:33 -0300
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <514A7638.4090205@pearwood.info>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org> <514A7638.4090205@pearwood.info>
Message-ID: <CAH0mxTSzG-4z4Aq36V+arrsLwYBX2zAj7C9ve0SmEdj8=db2Yw@mail.gmail.com>

On 20 March 2013 23:53, Steven D'Aprano <steve at pearwood.info> wrote:
> I also note that in the last few weeks, I've seen at least two instances
> that I recall of a beginner on the tutor at python.org mailing list being
> utterly confused by Python's Unicode handling because the Windows command
> prompt is unable to print Unicode strings.

It can be worst than that - in i18nes Windows installs, the DOS Prompt sometimes
have a different encoding than the Windows running -  for example, for
pt_BR Windows, all UI apps run using latin1, but the CP uses a CP850 encoding
which generates _different_ characters for the same codes.

As someone who form times to times lecture a introductory workshop of
Python to people running Windows, I second Terry's long message - and
I highlight
Raymond's """ Without IDLE, a shocking number of people would
create Python files using notepad. """


   js
  -><-
>
>
> Thanks Terry.

From tjreedy at udel.edu  Thu Mar 21 11:32:46 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 06:32:46 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CACac1F-Tg8m0qCJeyBfLi-v+R11MHkpRfB3wSD5w_PZmfiJCog@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
	<0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
	<CACac1F-Tg8m0qCJeyBfLi-v+R11MHkpRfB3wSD5w_PZmfiJCog@mail.gmail.com>
Message-ID: <kienkk$eaa$1@ger.gmane.org>

On 3/21/2013 5:27 AM, Paul Moore wrote:

> Can I suggest that debates about the capability of Windows command
> line programming are off-topic here?

I respectfully disagree, unless you say that the whole thread is off 
topic. If it is okay for people to say that IDLE, including the IDLE 
interactive interpreter shell is ugly, quirky, broken, badly maintained, 
and disfunctional, without giving hardly any facts or details to back up 
or explain the claims, and then claim it is so bad that it should be 
banished, then to me it is perfectly on topic for me to point out that 
the alternative, the CP shell, is objectively far worse in multiple 
respects. Yes, I gave some facts for the benefit of those who were 
willing to consider my claim that IDLE is better.

> it is what Windows users who use the command line are used to.

I bet that 'Windows users who use the command line' are less than 10% of 
Windows users.

I am willing to accept that you find it adequate. Why can't you accept 
that I find it wretched, especially when I explained some of why rather 
than just throwing it out as an opinion, and consider IDLE to be 
wonderfully better?

-- 
Terry Jan Reedy


From p.f.moore at gmail.com  Thu Mar 21 12:02:38 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 21 Mar 2013 11:02:38 +0000
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kienkk$eaa$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
	<0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
	<CACac1F-Tg8m0qCJeyBfLi-v+R11MHkpRfB3wSD5w_PZmfiJCog@mail.gmail.com>
	<kienkk$eaa$1@ger.gmane.org>
Message-ID: <CACac1F8uzTk+w6izCvMY=w4Y2Zv=wW0z1iU8Ht5PJk308NyUPQ@mail.gmail.com>

On 21 March 2013 10:32, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/21/2013 5:27 AM, Paul Moore wrote:
>
>> Can I suggest that debates about the capability of Windows command
>> line programming are off-topic here?
>
>
> I respectfully disagree, unless you say that the whole thread is off topic.
> If it is okay for people to say that IDLE, including the IDLE interactive
> interpreter shell is ugly, quirky, broken, badly maintained, and
> disfunctional, without giving hardly any facts or details to back up or
> explain the claims, and then claim it is so bad that it should be banished,
> then to me it is perfectly on topic for me to point out that the
> alternative, the CP shell, is objectively far worse in multiple respects.
> Yes, I gave some facts for the benefit of those who were willing to consider
> my claim that IDLE is better.

I agree entirely that unsubstantiated claims of "it's dreadful" are
just as unacceptable when referring to IDLE. And I'm happy to accept
that you find IDLE better - although I'd take issue with a claim that
it's objectively better for everyone. I'd rather that we focus on
dealing with genuine issues as reported by users (e.g., the comments
from people working with IDLE in training courses).

One difference, though, is that the quality of IDLE is within our
control, so comments about how it can be improved are valid - whereas
comments about the Windows console are simply statements of a reality
we have no means of addressing.

>> it is what Windows users who use the command line are used to.
>
> I bet that 'Windows users who use the command line' are less than 10% of
> Windows users.

I have no figures one way or the other on that. You may well be right.
Are we aiming at "all Windows users" here? All I can say is that my
experience (in a corporate Windows-based environment) is that people
who have any interest in learning or using Python are nearly always
*also* command line users, and comfortable with it. They are often
looking at Python as a step up from Windows batch files - and that is
such a huge improvement that trivia like the way you select text in
the console are completely irrelevant by comparison. My experience may
well be atypical, I can't say.

> I am willing to accept that you find it adequate. Why can't you accept that
> I find it wretched, especially when I explained some of why rather than just
> throwing it out as an opinion, and consider IDLE to be wonderfully better?

I'm happy to accept that. What I don't know (no criticism here, but
it's something I mentioned at the start of my post) is whether you use
Windows as your main platform, and so what you're implying in terms of
how to generalise the fact that you prefer IDLE (I assume you want me
to generalise, and not just take your comments as a statement of your
personal preference). Much of the comments about the Windows
experience *seem to me* to come from Unix users who only occasionally
use Windows and find the experience unpleasant. A bit more clarity as
to where the people advocating particular actions are coming from
would help (because I, and possibly others, may be wrong in that
impression).

Anyway, I was the one who claimed things were getting off-topic, so
I'll stop at this point. I hope I haven't offended anyone - I didn't
mean to.

Paul

From tjreedy at udel.edu  Thu Mar 21 12:03:32 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 07:03:32 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CACac1F83g0cKT9ON+1rRQ0UXrTQosRu9V-zU+jyFOOtyjGsqcQ@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CABicbJ+uRE+u8trzu8NsFroHxSnStMum-P7-yV3_dDjekhTK-g@mail.gmail.com>
	<CACac1F83g0cKT9ON+1rRQ0UXrTQosRu9V-zU+jyFOOtyjGsqcQ@mail.gmail.com>
Message-ID: <kiepea$v2$1@ger.gmane.org>

On 3/21/2013 5:41 AM, Paul Moore wrote:
> On 21 March 2013 06:54, Devin Jeanpierre <jeanpierreda at gmail.com> wrote:
>> On Thu, Mar 21, 2013 at 2:42 AM, Terry Reedy <tjreedy at udel.edu> wrote:
>>> I think being frozen in the late 1990s is better than being frozen in the
>>> early 1980s, like Command Prompt is. In fact, I think we should 'deprecate'
>>> the Command Prompt interpreter as the standard interactive interpreter and
>>> finish polishing and de-glitching IDLE's Python Shell, which runs on top of
>>> the windowless version of CP with a true GUI. Then we can promote and
>>> present the latter as the preferred interface, which for many people, it
>>> already is.

I should have prefaced that with 'on Windows'. I presume the command 
line on *nix is better with most of the issues I discussed than on Windows.

>> Please don't cease supporting the command line interface.

My one person opinion and counter-proposal is not going to change 
anything. Anyway, my two points were this: if late 1990s is bad, isn't 
early 1980s worse? And *if* we are going to one downplay/demote of the 
two interactive shells, should not it be the worse one?

 >> I use the
>> command line interactive interpreter plenty. That way I can use git,
>> grep, the unit test suite, etc. ... and the interactive interpreter,
>> all from one place: the console.
>>
>> That can't happen with IDLE, by design.

It is Microsoft, not me, that is a threat to Command Prompt. I have the 
impression that it is not part of the Win 8 not-Metro tablet interface 
that they would like everyone to use even on the desktop. To push 
beginners away from the desktop to the pane interface, they were 
initially going to limit the free Visual Express IDE and compilers to 
the new interface.

You can use idle from the command line almost as easily as the CP 
interpreter: 'python -m idlelib' instead of just 'python' (I just tried 
it to verify). Unlike bare 'python', IDLE includes a grep. Right click 
on any 'hit' and it opens the file at the specified line. Unlike bare 
'python', you can run tests and collect the all the output, from as many 
tests as you want, in a dynamically right-sized buffer.

> Agreed. Command line Python is 100% of my usage, and removing it would
> make Python unusable for me.

Whereas others would say the same about removing IDLE.

> If what is being suggested is removing the "Python Command Line"
> *shortcuts*

I did not suggest that.

> those shortcuts (and if I did, I could set them up myself). But before
> removing them, why not consider setting the defaults to be more
> helpful (larger scrollback buffer, things like quick edit set on, etc)
> if that's the real issue here? I'm not saying it is, but some of the
> complaints about "the default experience" *can* be fixed simply by
> changing the defaults.

If that is so easily possible, then it should have been done already.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Thu Mar 21 12:19:09 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 07:19:09 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130321102023.31b78c7f@pitrou.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org> <20130321102023.31b78c7f@pitrou.net>
Message-ID: <kieqbj$a9e$1@ger.gmane.org>

On 3/21/2013 5:20 AM, Antoine Pitrou wrote:
> Le Thu, 21 Mar 2013 02:42:33 -0400,
> Terry Reedy <tjreedy at udel.edu> a ?crit :
>> On 3/20/2013 11:54 PM, Eli Bendersky wrote:
>>> On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu
>>
>>>      Ugly is subjective: by what standard and compared to what?
>>>
>>> Compared to other existing Python IDEs and shells which are layered
>>> on top of modern GUI toolkits that are actively developed to keep
>>> with modern standards, unlike Tk which is frozen in the 1990s.
>>
>> I think being frozen in the late 1990s is better than being frozen in
>> the early 1980s, like Command Prompt is.  In fact, I think we should
>> 'deprecate' the Command Prompt interpreter as the standard
>> interactive interpreter and finish polishing and de-glitching IDLE's
>> Python Shell, which runs on top of the windowless version of CP with
>> a true GUI.
>
> And this may indeed be reasonable under Windows, where the command-line
> is a PITA!

Which is the only context I was talking about.

> But the Linux command-line is actually quite very usable
> these days, especially if you configure your Python interpreter to use
> readline for tab-completion of identifiers (which should be done by
> default, see http://bugs.python.org/issue5845).

IDLE has tab-completion for both identifiers and attributes, in both 
shell and editor windows. It is probably under-documented; I am still 
learning to use it effectively. I am curious if the readline version 
works better in any way that IDLE could imitate.

-- 
Terry Jan Reedy



From eliben at gmail.com  Thu Mar 21 13:22:40 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 21 Mar 2013 05:22:40 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <514AAAF0.7050303@gmx.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
	<CAF-Rda8n9GLkCifPtO2HR-405fkucJxZKXtSqhuieFy=AYNfHA@mail.gmail.com>
	<514AAAF0.7050303@gmx.net>
Message-ID: <CAF-Rda-+1gX27UzADnUFABUUOaPAHEBjzPz1J+DVSF8tgBz4hg@mail.gmail.com>

> >     On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org
> >     <mailto:barry at python.org>> wrote:
> >
> >>     Right.  Ultimately, I think IDLE should be a separate project
> entirely, but I
> >>     guess there's push back against that too.
> >
> >     The most important feature of IDLE is that it ships with the
> standard library.
> >     Everyone who clicks on the Windows MSI on the python.org <
> http://python.org>
> >     webpage
> >     automatically has IDLE.   That is why I frequently teach Python with
> IDLE.
> >
> >     If this thread results in IDLE being ripped out of the standard
> distribution,
> >     then I would likely never use it again.
> >
> >
> > Why is it necessary to conflate distribution and development. "standard
> library"
> > != "Python distribution".
>
> Because that's how CPython defines its distribution.  We distribute things
> that
> are in the CPython/standard library repo, and nothing else.
>

Yes, I realize this is the case. I was wondering whether it's hard to
change.


>
> > Take the ActivePython distribution for example. They ship with extra
> packages
> > for Windows (pywin32, etc) and our Python installer doesn't. This is a
> reason
> > many Windows people prefer ActivePython. That's their right, but this
> preference
> > is not the point. The point is that it's perfectly conceivable to ship
> IDLE with
> > Python releases on Windows, while managing it as a separate project
> outside the
> > CPython core Mercurial repository.
>
> And what's the benefit?  I just don't see it.  It just makes it harder to
> create
> a Python release.
>
>
This is the feedback I was looking for. If this will make Python
distribution non-trivially harder, then it's a point against the proposal.


> > This seems to me to combine benefits from both worlds:
> >
> > 1. IDLE keeps being shipped to end users. I have to admit the reasons
> made in
> > favor of this in the thread so far are convincing.
> > 2. IDLE is developed as a standalone project. As such, it's much easier
> to
> > contribute to, which will hopefully result in a quicker pace of
> improvement.
>
> Why?  You won't magically gather more people that are interested in IDLE
> development.
>

But that's the point - If there are not enough people interested in it, it
should then die. Right now it's a burden of Python core developers to keep
it functional even if no one else cares (and if anything, the low amount of
open issues Terry quoted elsewhere may be a sign that indeed not many care).


>
> > The
> > only demand is that it keeps working with a release version of Python,
> and this
> > is pretty easy. It's even possible and easy to have a single IDLE
> version for
> > Python 3.x instead of contributors having to propose patches for 3.2,
> 3.3 and
> > 3.4 simultaneously.
>
> They don't anyway.
>


But we know perfectly well that a core dev is expected to backport
reasonably. In an outside repo, it can have a single-code base. It's not
hard to avoid the new features of 3.3 and 3.4 and be compatible with all
active Python 3 versions. Note that even if the same is done in our
Mercurial repo, each commit still needs to be triplicated in the push dance.

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/20ee37ec/attachment.html>

From eliben at gmail.com  Thu Mar 21 13:25:57 2013
From: eliben at gmail.com (Eli Bendersky)
Date: Thu, 21 Mar 2013 05:25:57 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kiea4v$iar$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
Message-ID: <CAF-Rda__2rFEAvUtaDMU+tfaasPKkBNkSsRNMGPfeWvqpW0qgw@mail.gmail.com>

On Wed, Mar 20, 2013 at 11:42 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 3/20/2013 11:54 PM, Eli Bendersky wrote:
>
>> On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu
>>
>
>      Ugly is subjective: by what standard and compared to what?
>>
>> Compared to other existing Python IDEs and shells which are layered on
>> top of modern GUI toolkits that are actively developed to keep with
>> modern standards, unlike Tk which is frozen in the 1990s.
>>
>
> I think being frozen in the late 1990s is better than being frozen in the
> early 1980s, like Command Prompt is. In fact, I think we should 'deprecate'
> the Command Prompt interpreter as the standard interactive interpreter and
> finish polishing and de-glitching IDLE's Python Shell, which runs on top of
> the windowless version of CP with a true GUI. Then we can promote and
> present the latter as the preferred interface, which for many people, it
> already is.


There are two discussions being held in parallel here:

1. Whether IDLE should be developed separately from the core Python
repository (while still being shipped).
2. Whether IDLE is bad in a general sense and should die.

I've stated several times now that it's (1) that I'm interested in. (2) is
too subjective, and frankly I'm not in a good position to argue from an
objective point of view. As Antoine mentioned, "feeling" should not be
dismissed (especially if it resonates from a number of developers). That
said, I'm not going to continue to talk about (2). I really want to
constructively focus on (1).

Eli
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/fa8791db/attachment.html>

From dholth at gmail.com  Thu Mar 21 13:26:35 2013
From: dholth at gmail.com (Daniel Holth)
Date: Thu, 21 Mar 2013 08:26:35 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kieqbj$a9e$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org> <20130321102023.31b78c7f@pitrou.net>
	<kieqbj$a9e$1@ger.gmane.org>
Message-ID: <CAG8k2+5uA4F7C=o1y09QtH1LNjAYhH8y2pML5EcZZzF-K+xkAQ@mail.gmail.com>

I showed IDLE to my 6-year-old on the Raspberry Pi and I'm convinced
it is cool. Gave up on trying to (slowly) install bpython. We were
multiplying large numbers and counting to 325,000 in no time. It might
not be for *me* but I'm not going to teach my daughter a large IDE any
time soon.

From tjreedy at udel.edu  Thu Mar 21 13:34:24 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 08:34:24 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
Message-ID: <kieuom$mn8$1@ger.gmane.org>

On 3/20/2013 12:41 PM, Eli Bendersky wrote:
> Interesting writeup about PyCon 2013 young coder
> education:http://therealkatie.net/blog/2013/mar/19/pycon-2013-young-coders/
>
> Quote:
>
> "We used IDLE because it's already on Raspian's desktop. Personally, I
> like IDLE as a teaching tool. It's included in the standard library, it
> does tab completion and color coding, and it even has a text editor
> included so you don't have to start your class off by teaching everyone
> about paths.
>
> Too bad it's broke as hell."

A typical blog exaggeration, which is not a good basis for serious 
decision making. He continued "I believe my first contribution to the 
Python Standard Library will be fixes to IDLE. I really do like it that 
much. "

-- 
Terry Jan Reedy


From sturla at molden.no  Thu Mar 21 13:53:06 2013
From: sturla at molden.no (Sturla Molden)
Date: Thu, 21 Mar 2013 13:53:06 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314222337.GG24307@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<20130314222337.GG24307@snakebite.org>
Message-ID: <6BE748A8-4F59-43CF-BD18-AF116A764A60@molden.no>

Den 14. mars 2013 kl. 23:23 skrev Trent Nelson <trent at snakebite.org>:

> 
>    For the record, here are all the Windows calls I'm using that have
>    no *direct* POSIX equivalent:
> 
>        Interlocked singly-linked lists:
>            - InitializeSListHead()
>            - InterlockedFlushSList()
>            - QueryDepthSList()
>            - InterlockedPushEntrySList()
>            - InterlockedPushListSList()
>            - InterlockedPopEntrySlist()
> 
>        Synchronisation and concurrency primitives:
>            - Critical sections
>                - InitializeCriticalSectionAndSpinCount()
>                - EnterCriticalSection()
>                - LeaveCriticalSection()
>                - TryEnterCriticalSection()
>            - Slim read/writer locks (some pthread implements have
>              rwlocks)*:
>                - InitializeSRWLock()
>                - AcquireSRWLockShared()
>                - AcquireSRWLockExclusive()
>                - ReleaseSRWLockShared()
>                - ReleaseSRWLockExclusive()
>                - TryAcquireSRWLockExclusive()
>                - TryAcquireSRWLockShared()
>            - One-time initialization:
>                - InitOnceBeginInitialize()
>                - InitOnceComplete()
>            - Generic event, signalling and wait facilities:
>                - CreateEvent()
>                - SetEvent()
>                - WaitForSingleObject()
>                - WaitForMultipleObjects()
>                - SignalObjectAndWait()
> 
>        Native thread pool facilities:
>            - TrySubmitThreadpoolCallback()
>            - StartThreadpoolIo()
>            - CloseThreadpoolIo()
>            - CancelThreadpoolIo()
>            - DisassociateCurrentThreadFromCallback()
>            - CallbackMayRunLong()
>            - CreateThreadpoolWait()
>            - SetThreadpoolWait()
> 
>        Memory management:
>            - HeapCreate()
>            - HeapAlloc()
>            - HeapDestroy()
> 
>        Structured Exception Handling (#ifdef Py_DEBUG):
>            - __try/__except
> 
>        Sockets:
>            - ConnectEx()
>            - AcceptEx()
>            - WSAEventSelect(FD_ACCEPT)
>            - DisconnectEx(TF_REUSE_SOCKET)
>            - Overlapped WSASend()
>            - Overlapped WSARecv()
> 
> 
>    Don't get me wrong, I grew up with UNIX and love it as much as the
>    next guy, but you can't deny the usefulness of Windows' facilities
>    for writing high-performance, multi-threaded IO code.  It's decades
>    ahead of POSIX.  (Which is also why it bugs me when I see select()
>    being used on Windows, or IOCP being used as if it were a poll-type
>    "generic IO multiplexor" -- that's like having a Ferrari and speed
>    limiting it to 5mph!)
> 
>    So, before any of this has a chance of working on Linux/BSD, a lot
>    more scaffolding will need to be written to provide the things we
>    get for free on Windows (threadpools being the biggest freebie).
> 
> 
> 


Have you considered using OpenMP instead of Windows API or POSIX threads directly? OpenMP gives you a thread pool and synchronization primitives for free as well, with no special code needed for Windows or POSIX. 

OpenBLAS (and GotoBLAS2) uses OpenMP to produce a thread pool on POSIX systems (and actually Windows API on Windows). The OpenMP portion of the C code is wrapped so it looks like sending an asynch task to a thread pool; the C code is not littered with OpenMP pragmas. If you need something like Windows threadpools on POSIX, just look at the BSD licensed OpenBLAS code. It is written to be scalable for the world's largest supercomputers (but also beautifully written and very easy to read).

Cython has code to register OpenMP threads as Python threads, in case that is needed. So that problem is also solved.


Sturla









From solipsis at pitrou.net  Thu Mar 21 14:17:25 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 21 Mar 2013 14:17:25 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<20130314222337.GG24307@snakebite.org>
Message-ID: <20130321141725.5a1e9d0d@pitrou.net>

Le Thu, 14 Mar 2013 15:23:37 -0700,
Trent Nelson <trent at snakebite.org> a ?crit :
> 
>     Don't get me wrong, I grew up with UNIX and love it as much as the
>     next guy, but you can't deny the usefulness of Windows' facilities
>     for writing high-performance, multi-threaded IO code.  It's
> decades ahead of POSIX.

I suppose that's why all high-performance servers run under Windows.

Regards

Antoine.



From stephen at xemacs.org  Thu Mar 21 14:18:34 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 21 Mar 2013 22:18:34 +0900
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CACac1F8uzTk+w6izCvMY=w4Y2Zv=wW0z1iU8Ht5PJk308NyUPQ@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
	<0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
	<CACac1F-Tg8m0qCJeyBfLi-v+R11MHkpRfB3wSD5w_PZmfiJCog@mail.gmail.com>
	<kienkk$eaa$1@ger.gmane.org>
	<CACac1F8uzTk+w6izCvMY=w4Y2Zv=wW0z1iU8Ht5PJk308NyUPQ@mail.gmail.com>
Message-ID: <87y5dgdh79.fsf@uwakimon.sk.tsukuba.ac.jp>

Paul Moore writes:

 > I have no figures one way or the other on that. You may well be
 > right.  Are we aiming at "all Windows users" here?

We need to be careful about this.  ISTM that IDLE is aiming at the
subset of users on any platform who for some reason need/want a simple
development environment that is consistent across Python versions and
platforms and immediately available when they install Python, but
don't have one yet.

I think that there's been sufficient testimony to demonstrate that
there are a fair number of folks in that boat.  Educators (acting as
proxies for a couple of orders of magnitude more students) are one
identifiable group.  Beginning Python users on Windows who don't use
English in their daily lives and therefore need an environment that
deals with the nightmare of "code pages" and "POSIX locales" are
another.

 > My experience may well be atypical, I can't say.

I suppose it is reasonably typical.  I'm sure everybody (by now,
including Guido!)  have many parts of the stdlib they just never need
to use, nor does anybody around them.  Most of us rarely to never want
IDLE.  That's not the point.

The "batteries included" slogan is inaccurate in the sense that
everybody needs batteries or their toys won't run.  The batteries that
slogan refers to aren't for everyone, rather the hope is that there
are enough different batteries in the kit that everyone can get
started.  They can always get Twisted later.<wink/>


From rosuav at gmail.com  Thu Mar 21 14:31:29 2013
From: rosuav at gmail.com (Chris Angelico)
Date: Fri, 22 Mar 2013 00:31:29 +1100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <87y5dgdh79.fsf@uwakimon.sk.tsukuba.ac.jp>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<kidjg2$u22$1@ger.gmane.org>
	<0FD3F001-CA03-4FE8-A088-736A819BA252@me.com>
	<CACac1F-Tg8m0qCJeyBfLi-v+R11MHkpRfB3wSD5w_PZmfiJCog@mail.gmail.com>
	<kienkk$eaa$1@ger.gmane.org>
	<CACac1F8uzTk+w6izCvMY=w4Y2Zv=wW0z1iU8Ht5PJk308NyUPQ@mail.gmail.com>
	<87y5dgdh79.fsf@uwakimon.sk.tsukuba.ac.jp>
Message-ID: <CAPTjJmqb=dYiGkC_bhQXvGND9fhcxE8Uod4=xknCoMp-S0JQ5g@mail.gmail.com>

On Fri, Mar 22, 2013 at 12:18 AM, Stephen J. Turnbull
<stephen at xemacs.org> wrote:
> Paul Moore writes:
>
>  > I have no figures one way or the other on that. You may well be
>  > right.  Are we aiming at "all Windows users" here?
>
> We need to be careful about this.  ISTM that IDLE is aiming at the
> subset of users on any platform who for some reason need/want a simple
> development environment that is consistent across Python versions and
> platforms and immediately available when they install Python, but
> don't have one yet.

When I'm on Windows, I use IDLE as my interactive interpreter, but
SciTE for actual development. Even on Linux, there's one feature that
CLI interactive mode lacks: multi-line command recall. To tinker with
a function definition in IDLE, just recall it and edit it. To tinker
with it in command-line Python, fetch back each of its lines, or edit
it elsewhere and paste it. IDLE isn't a program editor for me, it's
the face of Python.

ChrisA

From barry at python.org  Thu Mar 21 14:22:55 2013
From: barry at python.org (Barry Warsaw)
Date: Thu, 21 Mar 2013 06:22:55 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAF-Rda__2rFEAvUtaDMU+tfaasPKkBNkSsRNMGPfeWvqpW0qgw@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CAF-Rda__2rFEAvUtaDMU+tfaasPKkBNkSsRNMGPfeWvqpW0qgw@mail.gmail.com>
Message-ID: <20130321062255.5da0dadf@anarchist>

On Mar 21, 2013, at 05:25 AM, Eli Bendersky wrote:

>1. Whether IDLE should be developed separately from the core Python
>repository (while still being shipped).
>
>I really want to constructively focus on (1).

In fact, solving (1) should help move along the discussions about separating
the stdlib into a separate repo, for the benefit of alternative
implementations.  Agreed that there are many other issues to resolve in that
latter discussion.

But I think Eli is right that we should be thinking about how to develop code
in separate repos and still ship a combined release.

-Barry

From ncoghlan at gmail.com  Thu Mar 21 16:21:44 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Mar 2013 08:21:44 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAG8k2+5uA4F7C=o1y09QtH1LNjAYhH8y2pML5EcZZzF-K+xkAQ@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org> <20130321102023.31b78c7f@pitrou.net>
	<kieqbj$a9e$1@ger.gmane.org>
	<CAG8k2+5uA4F7C=o1y09QtH1LNjAYhH8y2pML5EcZZzF-K+xkAQ@mail.gmail.com>
Message-ID: <CADiSq7dFRJdVnYaaK69LgYwQj9kpDZdkmbY6f8LRE23Ec0k4xw@mail.gmail.com>

On Thu, Mar 21, 2013 at 5:26 AM, Daniel Holth <dholth at gmail.com> wrote:
> I showed IDLE to my 6-year-old on the Raspberry Pi and I'm convinced
> it is cool. Gave up on trying to (slowly) install bpython. We were
> multiplying large numbers and counting to 325,000 in no time. It might
> not be for *me* but I'm not going to teach my daughter a large IDE any
> time soon.

This, 1000x this.

It was helping out at the Young Coders tutorials that convinced me we
need to continue shipping IDLE, or something like it, for use by
*people learning to use computers as more than just passive consumers
for the first time*. This means running well on Windows and the
Raspberry Pi at this point.

Keeping IDLE in the core represents a commitment to the use of Python
as a teaching language both inside and outside of formal educational
settings.

We can refactor IDLE to make aspects of it easier to test with the
buildbots, especially now that we have unittest.mock in the standard
library to mock out some of the UI interaction in the test suite. (I'm
happy to help coach the IDLE devs on that if they want to start
improving the test suite coverage for the IDLE code)

I think we should commit to making "start with IDLE" the recommended
teaching experience, and then focus on *making that experience
awesome*. Once people are already familiar with the language and what
it can do for them, they may choose to move on to other tools, or they
may decide to stick with IDLE. But deciding on "What is IDLE?" and
"Why is it part of the CPython development repo?" is a necessary step
to revitalising it and stopping the recurring discussions about taking
it out.

If Terry is willing to recast his PEP in that light, I think that
would be a wonderful thing to do.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Thu Mar 21 16:23:59 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 21 Mar 2013 08:23:59 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130321062255.5da0dadf@anarchist>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CAF-Rda__2rFEAvUtaDMU+tfaasPKkBNkSsRNMGPfeWvqpW0qgw@mail.gmail.com>
	<20130321062255.5da0dadf@anarchist>
Message-ID: <CADiSq7eOPGBUxcvKSX9p0EmN3vjHL2wf-t8=Qvx4WgyTWtGZ9w@mail.gmail.com>

On Thu, Mar 21, 2013 at 6:22 AM, Barry Warsaw <barry at python.org> wrote:
> On Mar 21, 2013, at 05:25 AM, Eli Bendersky wrote:
>
>>1. Whether IDLE should be developed separately from the core Python
>>repository (while still being shipped).
>>
>>I really want to constructively focus on (1).
>
> In fact, solving (1) should help move along the discussions about separating
> the stdlib into a separate repo, for the benefit of alternative
> implementations.  Agreed that there are many other issues to resolve in that
> latter discussion.
>
> But I think Eli is right that we should be thinking about how to develop code
> in separate repos and still ship a combined release.

I think a federated repo model in general is something we need to
consider, it's not something we should consider IDLE specific.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From baptiste.lepilleur at gmail.com  Thu Mar 21 17:30:34 2013
From: baptiste.lepilleur at gmail.com (Baptiste Lepilleur)
Date: Thu, 21 Mar 2013 17:30:34 +0100
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130314230007.GA24799@snakebite.org>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org>
	<20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<51425433.1090700@v.loewis.de> <20130314230007.GA24799@snakebite.org>
Message-ID: <CAGyWicDFSXmV+0FDtsjYF4ZNPCkR=NBe7arUbKihGOA0eOJWwA@mail.gmail.com>

2013/3/15 Trent Nelson <trent at snakebite.org>

> On Thu, Mar 14, 2013 at 03:50:27PM -0700, "Martin v. L?wis" wrote:
> > Am 14.03.13 12:59, schrieb Stefan Ring:
> > > I think you should be able to just take the address of a static
> > > __thread variable to achieve the same thing in a more portable way.
> >
> > That assumes that the compiler supports __thread variables, which
> > isn't that portable in the first place.
>
>     FWIW, I make extensive use of __declspec(thread).  I'm aware of GCC
>     and Clang's __thread alternative.  No idea what IBM xlC, Sun Studio
>     and others offer, if anything.
>

IBM xlC and Sun Studio also support this feature. From memory, it's also
__thread keyword. This features is also supported by the new C11/C++11
standards.

Baptiste.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/7b910e56/attachment.html>

From p.f.moore at gmail.com  Thu Mar 21 17:32:14 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 21 Mar 2013 16:32:14 +0000
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on Windows
Message-ID: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>

PEP 405 has this note:

"""
On Windows, it is necessary to also copy or symlink DLLs and pyd files
from compiled stdlib modules into the env, because if the venv is
created from a non-system-wide Python installation, Windows won't be
able to find the Python installation's copies of those files when
Python is run from the venv.
"""

I don't understand what this is saying - can someone clarify the
reason behind this statement? What is different about a
"non-system-wide installation" that causes this issue (I assume
"non-system-wide" means "not All Users")? The reason I ask is that
virtualenv doesn't do this, and I'm not clear if this is because of a
potential bug lurking in virtualenv (in which case, I'd like to find
out how to reproduce it) or because virtualenv takes a different
approach which avoids this issue somehow.

Thanks,
Paul.

From trent at snakebite.org  Thu Mar 21 17:55:51 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 21 Mar 2013 09:55:51 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <CAGyWicDFSXmV+0FDtsjYF4ZNPCkR=NBe7arUbKihGOA0eOJWwA@mail.gmail.com>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<51425433.1090700@v.loewis.de> <20130314230007.GA24799@snakebite.org>
	<CAGyWicDFSXmV+0FDtsjYF4ZNPCkR=NBe7arUbKihGOA0eOJWwA@mail.gmail.com>
Message-ID: <906BF18E-918D-4018-9840-27A2AD0A83EF@snakebite.org>

That's good to hear :-)

(It's a fantastic facility, I couldn't imagine having to go back to manual TLS API stuff after using __thread/__declspec(thread).)

This e-mail was sent from a wireless device.

On 21 Mar 2013, at 09:30, "Baptiste Lepilleur" <baptiste.lepilleur at gmail.com<mailto:baptiste.lepilleur at gmail.com>> wrote:



2013/3/15 Trent Nelson <trent at snakebite.org<mailto:trent at snakebite.org>>
On Thu, Mar 14, 2013 at 03:50:27PM -0700, "Martin v. L?wis" wrote:
> Am 14.03.13 12:59, schrieb Stefan Ring:
> > I think you should be able to just take the address of a static
> > __thread variable to achieve the same thing in a more portable way.
>
> That assumes that the compiler supports __thread variables, which
> isn't that portable in the first place.

    FWIW, I make extensive use of __declspec(thread).  I'm aware of GCC
    and Clang's __thread alternative.  No idea what IBM xlC, Sun Studio
    and others offer, if anything.

IBM xlC and Sun Studio also support this feature. From memory, it's also __thread keyword. This features is also supported by the new C11/C++11 standards.

Baptiste.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/9b56e1b0/attachment.html>

From g.brandl at gmx.net  Thu Mar 21 18:39:00 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 21 Mar 2013 18:39:00 +0100
Subject: [Python-Dev] How to fix the incorrect shared library extension
 on linux for 3.2 and newer?
In-Reply-To: <20130320143648.6df30a2f@anarchist>
References: <514A198B.500@ubuntu.com> <20130320143648.6df30a2f@anarchist>
Message-ID: <kifgj1$tsq$1@ger.gmane.org>

Am 20.03.2013 22:36, schrieb Barry Warsaw:
> On Mar 20, 2013, at 01:18 PM, Matthias Klose wrote:
> 
>>The patch in the issue now makes a distinction between EXT_SUFFIX and
>>SHLIB_SUFFIX, and restores the value of SO to SHLIB_SUFFIX.  Now this could
>>break users of sysconfig.get_config_var('SO'), however I don't see a better
>>way than to restore the original behaviour and advise people to use the new
>>config variables.

I agree. This looks like a seriously broken behavior to me.

> It should probably be considered a bug that we changed the meaning of SO in
> PEP 3149, but I don't think anybody realized it was being used for both
> purposes (otherwise I'm sure we wouldn't have done it that way).  I suppose
> Georg should make the final determination for 3.2 and 3.3, but the solution
> you propose seems about the best you can do.
>
> As we discussed at Pycon, you'll post a diff to the PEP in the tracker issue
> and I'll commit that when I figure out the best way to indicate that a PEP has
> been updated post-Final status.

Sounds good.

Georg


From g.brandl at gmx.net  Thu Mar 21 18:40:27 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 21 Mar 2013 18:40:27 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CADiSq7eOPGBUxcvKSX9p0EmN3vjHL2wf-t8=Qvx4WgyTWtGZ9w@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CAF-Rda__2rFEAvUtaDMU+tfaasPKkBNkSsRNMGPfeWvqpW0qgw@mail.gmail.com>
	<20130321062255.5da0dadf@anarchist>
	<CADiSq7eOPGBUxcvKSX9p0EmN3vjHL2wf-t8=Qvx4WgyTWtGZ9w@mail.gmail.com>
Message-ID: <kifglo$tsq$2@ger.gmane.org>

Am 21.03.2013 16:23, schrieb Nick Coghlan:
> On Thu, Mar 21, 2013 at 6:22 AM, Barry Warsaw <barry at python.org> wrote:
>> On Mar 21, 2013, at 05:25 AM, Eli Bendersky wrote:
>>
>>>1. Whether IDLE should be developed separately from the core Python
>>>repository (while still being shipped).
>>>
>>>I really want to constructively focus on (1).
>>
>> In fact, solving (1) should help move along the discussions about separating
>> the stdlib into a separate repo, for the benefit of alternative
>> implementations.  Agreed that there are many other issues to resolve in that
>> latter discussion.
>>
>> But I think Eli is right that we should be thinking about how to develop code
>> in separate repos and still ship a combined release.
> 
> I think a federated repo model in general is something we need to
> consider, it's not something we should consider IDLE specific.

Right.  Without a coordinated plan this will go the road of elementtree or
simplejson.

Georg


From fwierzbicki at gmail.com  Thu Mar 21 18:44:16 2013
From: fwierzbicki at gmail.com (fwierzbicki at gmail.com)
Date: Thu, 21 Mar 2013 10:44:16 -0700
Subject: [Python-Dev] Federated repo model [was: IDLE in the stdlib]
Message-ID: <CADrh4z+fK+N5JagCbV--3jDHv8-of5aVXxLpJM2Bu_nvgwaUjQ@mail.gmail.com>

On Thu, Mar 21, 2013 at 8:23 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I think a federated repo model in general is something we need to
> consider, it's not something we should consider IDLE specific.
I would love to have a federated repo model. I have recently made the
attempt to port the devguide for CPython to Jython with some
reasonable success. Part of that success has come because the devguide
is in its own repo and so forking it and continuing to merge
improvements from the original has been very easy.

I'd love to be able to do the same for the Doc/ directory at the root
of the CPython repo, but currently would have to fork the entire code
and doc repository etc. This would mean that merging the Doc/
improvements would be a big pain, with lots and lots of useless merges
where it would be hard to pick out the Doc changes.

To a lesser extent the same would hold for the Lib/ area - though in
that case I don't mind just pushing our changes to the CPython Lib/
(through the tracker and with code reviews of course) in the medium
term. Still, a separate repo for Lib would definitely be nice down the
road.

-Frank

From solipsis at pitrou.net  Thu Mar 21 19:13:00 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Thu, 21 Mar 2013 19:13:00 +0100
Subject: [Python-Dev] IDLE in the stdlib
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
Message-ID: <20130321191300.4ffe831f@pitrou.net>

On Wed, 20 Mar 2013 19:57:54 -0700
Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> 
> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org> wrote:
> 
> > Right.  Ultimately, I think IDLE should be a separate project entirely, but I
> > guess there's push back against that too.
> 
> The most important feature of IDLE is that it ships with the standard library.
> Everyone who clicks on the Windows MSI on the python.org webpage
> automatically has IDLE.   That is why I frequently teach Python with IDLE.
> 
> If this thread results in IDLE being ripped out of the standard distribution,
> then I would likely never use it again.

Which says a lot about its usefulness, if the only reason you use it is
that it's bundled with the standard distribution.

Regards

Antoine.



From rdmurray at bitdance.com  Thu Mar 21 19:23:49 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Thu, 21 Mar 2013 14:23:49 -0400
Subject: [Python-Dev] cpython: Issue #13248: removed deprecated and
	undocumented difflib.isbjunk, isbpopular.
In-Reply-To: <kie59j$cpc$1@ger.gmane.org>
References: <3ZVrTH23v4zPCm@mail.python.org>
	<CAF-Rda_-rXSs2VVZM6eht=ZP2aq2rgqNj+M4q7Zw4h7Lu-hsew@mail.gmail.com>
	<20130320181320.5B45C2500B3@webabinitio.net>
	<kie59j$cpc$1@ger.gmane.org>
Message-ID: <20130321182350.4240A250069@webabinitio.net>

On Thu, 21 Mar 2013 01:19:42 -0400, Terry Reedy <tjreedy at udel.edu> wrote:
> On 3/20/2013 2:13 PM, R. David Murray wrote:
> > On Wed, 20 Mar 2013 05:23:43 -0700, Eli Bendersky <eliben at gmail.com> wrote:
> >> A mention in Misc/NEWS can't hurt here, Terry. Even though it's
> >> undocumented, some old code could rely on it being there and this code will
> >> break with the transition to 3.4
> 
> Will do.
> 
> > Note that we also have a list of deprecated things that were removed in
> > What's New.
> >
> > Aside: given the 3.3 experience, I think people should be thinking in
> > terms of always updating What's New when appropriate, at the time a
> > commit is made.
> 
> How does this look? Is ``replacement`` right? Should the subsequent sm 
> by itself be marked? If so, how?
> 
> * :meth:`difflib.SequenceMatcher.isbjunk` and
>    :meth:`difflib.SequenceMatcher.isbpopular`: use ``x in sm.bjunk`` and
>    ``x in sm.bpopular``, where sm is a SequenceMatcher object.

Looks fine to me.  A link on SequenceMatcher probably isn't necessary,
but might be handy.  If you want to add it it would be
:class:`~difflib.SequenceMatcher`.

--David

From doko at ubuntu.com  Thu Mar 21 21:14:59 2013
From: doko at ubuntu.com (Matthias Klose)
Date: Thu, 21 Mar 2013 13:14:59 -0700
Subject: [Python-Dev] backporting the _sysconfigdata.py module to 2.7
Message-ID: <514B6A43.20102@ubuntu.com>

I'd like to backport issue13150, the _sysconfigdata.py module to 2.7. My
motivation is not the improved startup time, but the ability to cross-build
extension modules using distutils/setuptools.  The basic idea is to use the
python interpreter of the build machine (the machine you build on), and the
_sysconfigdata.py for the host machine (the machine you build for).  This kind
of setup works fine as long as the setup.py for a third party package gets all
build related information from the sysconfig.py module, and not directly from os
or sys (e.g. sys.platform).

The patch for issue13150 doesn't change any API's, but only moves the
computation of the config vars from runtime to build time.  I'd like to avoid
backporting this to 3.2 as well, because the cross-build support is currently
only found in 2.7, 3.3 and the trunk.

  Matthias

From g.brandl at gmx.net  Thu Mar 21 21:38:41 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Thu, 21 Mar 2013 21:38:41 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130321191300.4ffe831f@pitrou.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
	<20130321191300.4ffe831f@pitrou.net>
Message-ID: <kifr3u$hqc$1@ger.gmane.org>

Am 21.03.2013 19:13, schrieb Antoine Pitrou:
> On Wed, 20 Mar 2013 19:57:54 -0700
> Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
>> 
>> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org> wrote:
>> 
>> > Right.  Ultimately, I think IDLE should be a separate project entirely, but I
>> > guess there's push back against that too.
>> 
>> The most important feature of IDLE is that it ships with the standard library.
>> Everyone who clicks on the Windows MSI on the python.org webpage
>> automatically has IDLE.   That is why I frequently teach Python with IDLE.
>> 
>> If this thread results in IDLE being ripped out of the standard distribution,
>> then I would likely never use it again.
> 
> Which says a lot about its usefulness, if the only reason you use it is
> that it's bundled with the standard distribution.

Just like a lot of the stdlib, it *gets* a lot of usefulness from being a
battery.  But just because there are better/more comprehensive/prettier
replacements out there is not reason enough to remove standard libraries.

Georg


From dreamingforward at gmail.com  Thu Mar 21 22:19:33 2013
From: dreamingforward at gmail.com (Mark Janssen)
Date: Thu, 21 Mar 2013 14:19:33 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kidv0r$tcv$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
Message-ID: <CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>

On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu> wrote:

> On 3/20/2013 12:41 PM, Eli Bendersky wrote:
>
>  Personally, I think that IDLE reflects badly on Python in more ways than
>> one. It's badly maintained, quirky and ugly.
>>
>
> Ugly is subjective: by what standard and compared to what?
>

I might be jumping in late here, but...

The *only* thing I find "ugly" about it is that it doesn't have a
white-on-black color scheme.  Look at any hacker console and you won't find
a white screen.  Otherwise its fine.  Fixing that issue is simple, I can
upload my color scheme if anyone wants.

> It serves a very narrow set of uses,
>
> IDLE serves a very important "narrow use" purpose -- helping the plethora
of beginning programmers.  Anyone who wants to criticize it can slap
themselves.  Python attracts many beginners, and if you don't remember,
installing a separate "fancy" editor was never on the priority list until
several years later.  Give me a break.


> > and does it badly.
>
> Come on.  It gets even a strong programmer 80% of the way to what he/she
needs.

And in any case, I think the interpreter environment is the place to keep
the programmer's focus.  That is the arena where the community has been and
it's what has kept programming in Python fun.  And although this goes
against decades(?) of programming history, the future of programming, is
not in the editor.  The "editor-centric paradigm" has not created a
community of re-usable code, despite all the promises.

I'll argue that the *interpreter environment* will be the future and the
editor will be relegated to a simple memory-saving device.

Mark
Tacoma, Washington
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/32c614f7/attachment.html>

From phd at phdru.name  Thu Mar 21 22:31:28 2013
From: phd at phdru.name (Oleg Broytman)
Date: Fri, 22 Mar 2013 01:31:28 +0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
Message-ID: <20130321213128.GA6934@iskra.aviel.ru>

On Thu, Mar 21, 2013 at 02:19:33PM -0700, Mark Janssen <dreamingforward at gmail.com> wrote:
> The *only* thing I find "ugly" about it is that it doesn't have a
> white-on-black color scheme.  Look at any hacker console and you won't find
> a white screen.

   Call me a bad hacker or not hacker at all -- I hate black
backgrounds. My windows are always black-on-lightgrey, sometimes on dark
grey, never black. I have been spending 16 hours a day at the screen for
last 25 years -- and never understood black background.

Oleg.
-- 
     Oleg Broytman            http://phdru.name/            phd at phdru.name
           Programmers don't die, they just GOSUB without RETURN.

From benjamin at python.org  Thu Mar 21 22:48:50 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Thu, 21 Mar 2013 16:48:50 -0500
Subject: [Python-Dev] backporting the _sysconfigdata.py module to 2.7
In-Reply-To: <514B6A43.20102@ubuntu.com>
References: <514B6A43.20102@ubuntu.com>
Message-ID: <CAPZV6o8HKmKuiZ_nLfkftdA5A7bq1oA2NwTjmfqroYs3DdCidQ@mail.gmail.com>

2013/3/21 Matthias Klose <doko at ubuntu.com>:
> I'd like to backport issue13150, the _sysconfigdata.py module to 2.7. My
> motivation is not the improved startup time, but the ability to cross-build
> extension modules using distutils/setuptools.  The basic idea is to use the
> python interpreter of the build machine (the machine you build on), and the
> _sysconfigdata.py for the host machine (the machine you build for).  This kind
> of setup works fine as long as the setup.py for a third party package gets all
> build related information from the sysconfig.py module, and not directly from os
> or sys (e.g. sys.platform).
>
> The patch for issue13150 doesn't change any API's, but only moves the
> computation of the config vars from runtime to build time.  I'd like to avoid
> backporting this to 3.2 as well, because the cross-build support is currently
> only found in 2.7, 3.3 and the trunk.

This is a fairly small non-userfacing change, so okay.



-- 
Regards,
Benjamin

From dreamingforward at gmail.com  Thu Mar 21 23:36:58 2013
From: dreamingforward at gmail.com (Mark Janssen)
Date: Thu, 21 Mar 2013 15:36:58 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130321213128.GA6934@iskra.aviel.ru>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
	<20130321213128.GA6934@iskra.aviel.ru>
Message-ID: <CAMjeLr_=Ti0GqgD5bci-PXwGx-ALFX4sfzBKtTK_drvcS6uWSw@mail.gmail.com>

On Thu, Mar 21, 2013 at 2:31 PM, Oleg Broytman <phd at phdru.name> wrote:

> On Thu, Mar 21, 2013 at 02:19:33PM -0700, Mark Janssen <
> dreamingforward at gmail.com> wrote:
> > The *only* thing I find "ugly" about it is that it doesn't have a
> > white-on-black color scheme.  Look at any hacker console and you won't
> find
> > a white screen.
>
>    Call me a bad hacker or not hacker at all -- I hate black
> backgrounds. My windows are always black-on-lightgrey, sometimes on dark
> grey, never black. I have been spending 16 hours a day at the screen for
> last 25 years -- and never understood black background.


Lol, funny.  It takes energy to display a phosphor, but none for black.  So
I don't know how it could be harder for the eyes.  Plus, it's "hacker
nostalgia" for me, going back to assembler and BASIC on an Apple II.  But I
think this thread discussion happened decades ago.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/46fc2110/attachment.html>

From thomas at python.org  Thu Mar 21 23:49:39 2013
From: thomas at python.org (Thomas Wouters)
Date: Thu, 21 Mar 2013 23:49:39 +0100
Subject: [Python-Dev] backporting the _sysconfigdata.py module to 2.7
In-Reply-To: <CAPZV6o8HKmKuiZ_nLfkftdA5A7bq1oA2NwTjmfqroYs3DdCidQ@mail.gmail.com>
References: <514B6A43.20102@ubuntu.com>
	<CAPZV6o8HKmKuiZ_nLfkftdA5A7bq1oA2NwTjmfqroYs3DdCidQ@mail.gmail.com>
Message-ID: <CAPdQG2q_K6Lt0yDt37-H+8Tz-GYaGaiHmNnMGedd96nebwz_6w@mail.gmail.com>

On Thu, Mar 21, 2013 at 10:48 PM, Benjamin Peterson <benjamin at python.org>wrote:

> 2013/3/21 Matthias Klose <doko at ubuntu.com>:
> > I'd like to backport issue13150, the _sysconfigdata.py module to 2.7. My
> > motivation is not the improved startup time, but the ability to
> cross-build
> > extension modules using distutils/setuptools.  The basic idea is to use
> the
> > python interpreter of the build machine (the machine you build on), and
> the
> > _sysconfigdata.py for the host machine (the machine you build for).
>  This kind
> > of setup works fine as long as the setup.py for a third party package
> gets all
> > build related information from the sysconfig.py module, and not directly
> from os
> > or sys (e.g. sys.platform).
> >
> > The patch for issue13150 doesn't change any API's, but only moves the
> > computation of the config vars from runtime to build time.  I'd like to
> avoid
> > backporting this to 3.2 as well, because the cross-build support is
> currently
> > only found in 2.7, 3.3 and the trunk.
>
> This is a fairly small non-userfacing change, so okay.
>

FWIW, we do the exact same thing in our (internal) Google Python 2.7 builds
(because we're forbidden from having files named 'Makefile' and '*.h' in
our production environment -- and also because of the startup time) and
while we've seen the most obscure, internal changes cause failures in the
most unexpected ways, we haven't seen anything broken or failing in any way
by that change. (Not that I was expecting it, I'm just saying even I think
this is a good idea ;)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm a .signature virus! copy me into your .signature file to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130321/89a5dfc5/attachment.html>

From thomas at python.org  Fri Mar 22 00:17:38 2013
From: thomas at python.org (Thomas Wouters)
Date: Fri, 22 Mar 2013 00:17:38 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAMjeLr_=Ti0GqgD5bci-PXwGx-ALFX4sfzBKtTK_drvcS6uWSw@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
	<20130321213128.GA6934@iskra.aviel.ru>
	<CAMjeLr_=Ti0GqgD5bci-PXwGx-ALFX4sfzBKtTK_drvcS6uWSw@mail.gmail.com>
Message-ID: <CAPdQG2oNxX3HDS9__wsRR4dAaGB+1=orHEX_+3nK1AmCBN_qtw@mail.gmail.com>

I expressed this opinion at the sprints (right before I left) in the group
discussion with Guido and Nick, but I'm not sure if it's been represented
in this thread yet (I'm jetlagged and talk about Windows command prompts
depresses me) -- so I'll just rehash it: distributing IDLE in the binary
packages people download from python.org means *python-dev is still
responsible IDLE*. We can't distribute something that we don't support.
Even for the third-party libraries we're wrapping we're taking
responsibility for updating them, fixing specific bugs or working around
the bugs in the wrappers. Removing IDLE from the source tarballs isn't a
way to disown it, or shed responsibility. The benefits of having IDLE in a
separate repository, as I see it, would be that we can have separate access
control for the repositories, and possibly make it more approachable for
new developers, and easier to re-use by other Python implementations. We
couldn't even sensibly stop accepting bugs for it on bugs.python.org.

It may well be that moving IDLE to a separate repository is the right
thing, but only if there's an active team of people working on it that
would prefer it that way. And only if we realize that if IDLE languishes
again, python-dev is *still* on the hook for it, even in the separate
repository. I don't know if excluding it from the source tarball gains us
anything on top of that -- although I do think we should move 'idlelib' out
of the standard library :)

-- 
Thomas Wouters <thomas at python.org>

Hi! I'm an email virus! Think twice before sending your email to help me
spread!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130322/eb50d6ed/attachment.html>

From tjreedy at udel.edu  Fri Mar 22 00:37:57 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 19:37:57 -0400
Subject: [Python-Dev] cpython: Issue #13248: removed deprecated and
 undocumented difflib.isbjunk, isbpopular.
In-Reply-To: <20130321182350.4240A250069@webabinitio.net>
References: <3ZVrTH23v4zPCm@mail.python.org>
	<CAF-Rda_-rXSs2VVZM6eht=ZP2aq2rgqNj+M4q7Zw4h7Lu-hsew@mail.gmail.com>
	<20130320181320.5B45C2500B3@webabinitio.net>
	<kie59j$cpc$1@ger.gmane.org>
	<20130321182350.4240A250069@webabinitio.net>
Message-ID: <kig5ks$qig$1@ger.gmane.org>

On 3/21/2013 2:23 PM, R. David Murray wrote:
> On Thu, 21 Mar 2013 01:19:42 -0400, Terry Reedy <tjreedy at udel.edu> wrote:

>> How does this look? Is ``replacement`` right? Should the subsequent sm
>> by itself be marked? If so, how?
>>
>> * :meth:`difflib.SequenceMatcher.isbjunk` and
>>     :meth:`difflib.SequenceMatcher.isbpopular`: use ``x in sm.bjunk`` and
>>     ``x in sm.bpopular``, where sm is a SequenceMatcher object.
>
> Looks fine to me.  A link on SequenceMatcher probably isn't necessary,
> but might be handy.  If you want to add it it would be
> :class:`~difflib.SequenceMatcher`.

Committed and pushed.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Fri Mar 22 01:26:59 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 20:26:59 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
Message-ID: <kig8gp$jle$1@ger.gmane.org>

On 3/21/2013 5:19 PM, Mark Janssen wrote:
> On Wed, Mar 20, 2013 at 8:32 PM, Terry Reedy <tjreedy at udel.edu
> <mailto:tjreedy at udel.edu>> wrote:

> I might be jumping in late here, but...

Not at all. Thank you for the enlightening post.

> The *only* thing I find "ugly" about it is that it doesn't have a
> white-on-black color scheme.

Oh.

The wonderful thing about computers is the possibility of customizing to 
taste. The think I do not like about .pdf is the motivation to 'control 
the user experience'.

> Look at any hacker console and you won't
> find a white screen.  Otherwise its fine.  Fixing that issue is simple,
> I can upload my color scheme if anyone wants.

If you want, open an IDLE enhancement issue "IDLE: add alternate 
high-light schemes", select Componemts: IDLE, and attach your 'Console' 
scheme. Many skinnable apps either have a place for users to share their 
custom skins or come with alternatives builtin. There might be other 
emulation schemes worth adding.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Fri Mar 22 01:27:26 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 20:27:26 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CADiSq7dFRJdVnYaaK69LgYwQj9kpDZdkmbY6f8LRE23Ec0k4xw@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org> <20130321102023.31b78c7f@pitrou.net>
	<kieqbj$a9e$1@ger.gmane.org>
	<CAG8k2+5uA4F7C=o1y09QtH1LNjAYhH8y2pML5EcZZzF-K+xkAQ@mail.gmail.com>
	<CADiSq7dFRJdVnYaaK69LgYwQj9kpDZdkmbY6f8LRE23Ec0k4xw@mail.gmail.com>
Message-ID: <kig8hk$jle$2@ger.gmane.org>

On 3/21/2013 11:21 AM, Nick Coghlan wrote:
> On Thu, Mar 21, 2013 at 5:26 AM, Daniel Holth <dholth at gmail.com> wrote:
>> I showed IDLE to my 6-year-old on the Raspberry Pi and I'm convinced
>> it is cool. Gave up on trying to (slowly) install bpython. We were
>> multiplying large numbers and counting to 325,000 in no time. It might
>> not be for *me* but I'm not going to teach my daughter a large IDE any
>> time soon.
>
> This, 1000x this.
>
> It was helping out at the Young Coders tutorials that convinced me we
> need to continue shipping IDLE, or something like it, for use by
> *people learning to use computers as more than just passive consumers
> for the first time*. This means running well on Windows and the
> Raspberry Pi at this point.
>
> Keeping IDLE in the core represents a commitment to the use of Python
> as a teaching language both inside and outside of formal educational
> settings.
>
> We can refactor IDLE to make aspects of it easier to test with the
> buildbots, especially now that we have unittest.mock in the standard
> library to mock out some of the UI interaction in the test suite. (I'm
> happy to help coach the IDLE devs on that if they want to start
> improving the test suite coverage for the IDLE code)

Thank for for the offer to help. I added you to the IDLE test issue.
http://bugs.python.org/issue15392
Improving tests is one of the main things I personally want to do. Roger 
is expert at tkinter code, so I will focus on other things. I want to 
work toward IDLE patches following the standard rule of adding at least 
one test with every patch. A permanent exemption from that rule is *not* 
part of the PEP.

> I think we should commit to making "start with IDLE" the recommended
> teaching experience, and then focus on *making that experience
> awesome*. Once people are already familiar with the language and what
> it can do for them, they may choose to move on to other tools, or they
> may decide to stick with IDLE. But deciding on "What is IDLE?" and
> "Why is it part of the CPython development repo?" is a necessary step
> to revitalising it and stopping the recurring discussions about taking
> it out.

> If Terry is willing to recast his PEP in that light, I think that
> would be a wonderful thing to do.

I completely agree ;-). I asked Todd to help with this, and perhaps you 
can give me some more concrete hints as to what you would like to see where.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Fri Mar 22 02:24:58 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu, 21 Mar 2013 21:24:58 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CAPdQG2oNxX3HDS9__wsRR4dAaGB+1=orHEX_+3nK1AmCBN_qtw@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAMjeLr-KaaksosAnRqfHUkamO9i7Y2F=T41=C+vpXtVLVcXLLg@mail.gmail.com>
	<20130321213128.GA6934@iskra.aviel.ru>
	<CAMjeLr_=Ti0GqgD5bci-PXwGx-ALFX4sfzBKtTK_drvcS6uWSw@mail.gmail.com>
	<CAPdQG2oNxX3HDS9__wsRR4dAaGB+1=orHEX_+3nK1AmCBN_qtw@mail.gmail.com>
Message-ID: <kigbth$hf2$1@ger.gmane.org>

On 3/21/2013 7:17 PM, Thomas Wouters wrote:

> although I do think we should move 'idlelib' out of the standard library :)

Currently, 'python -m idlelib' start idle from the command line. If 
idlelib/ were moved out of /Lib, idle.py should be added so 'python -m 
idle' would work. I may suggest that anyway.

-- 
Terry Jan Reedy


From trent at snakebite.org  Fri Mar 22 04:00:18 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 21 Mar 2013 20:00:18 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <6BE748A8-4F59-43CF-BD18-AF116A764A60@molden.no>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<20130314222337.GG24307@snakebite.org>
	<6BE748A8-4F59-43CF-BD18-AF116A764A60@molden.no>
Message-ID: <81018940-7EEA-48DA-879B-DEBE158B1338@snakebite.org>

No, I haven't.  I'd lose the excellent Windows pairing of thread pool IO and overlapped IO facilities if I did that.

Not saying it isn't an option down the track for the generic "submit work" API though; that stuff will work against any thread pool without too much effort.

But for now, the fact that all I need to call is TrySubmitThreadpoolCallback and Windows does *everything* else is pretty handy.  Lets me concentrate on the problem instead of getting distracted by scaffolding.

This e-mail was sent from a wireless device. 

On 21 Mar 2013, at 05:53, "Sturla Molden" <sturla at molden.no> wrote:

> Den 14. mars 2013 kl. 23:23 skrev Trent Nelson <trent at snakebite.org>:
> 
>> 
>>   For the record, here are all the Windows calls I'm using that have
>>   no *direct* POSIX equivalent:
>> 
>>       Interlocked singly-linked lists:
>>           - InitializeSListHead()
>>           - InterlockedFlushSList()
>>           - QueryDepthSList()
>>           - InterlockedPushEntrySList()
>>           - InterlockedPushListSList()
>>           - InterlockedPopEntrySlist()
>> 
>>       Synchronisation and concurrency primitives:
>>           - Critical sections
>>               - InitializeCriticalSectionAndSpinCount()
>>               - EnterCriticalSection()
>>               - LeaveCriticalSection()
>>               - TryEnterCriticalSection()
>>           - Slim read/writer locks (some pthread implements have
>>             rwlocks)*:
>>               - InitializeSRWLock()
>>               - AcquireSRWLockShared()
>>               - AcquireSRWLockExclusive()
>>               - ReleaseSRWLockShared()
>>               - ReleaseSRWLockExclusive()
>>               - TryAcquireSRWLockExclusive()
>>               - TryAcquireSRWLockShared()
>>           - One-time initialization:
>>               - InitOnceBeginInitialize()
>>               - InitOnceComplete()
>>           - Generic event, signalling and wait facilities:
>>               - CreateEvent()
>>               - SetEvent()
>>               - WaitForSingleObject()
>>               - WaitForMultipleObjects()
>>               - SignalObjectAndWait()
>> 
>>       Native thread pool facilities:
>>           - TrySubmitThreadpoolCallback()
>>           - StartThreadpoolIo()
>>           - CloseThreadpoolIo()
>>           - CancelThreadpoolIo()
>>           - DisassociateCurrentThreadFromCallback()
>>           - CallbackMayRunLong()
>>           - CreateThreadpoolWait()
>>           - SetThreadpoolWait()
>> 
>>       Memory management:
>>           - HeapCreate()
>>           - HeapAlloc()
>>           - HeapDestroy()
>> 
>>       Structured Exception Handling (#ifdef Py_DEBUG):
>>           - __try/__except
>> 
>>       Sockets:
>>           - ConnectEx()
>>           - AcceptEx()
>>           - WSAEventSelect(FD_ACCEPT)
>>           - DisconnectEx(TF_REUSE_SOCKET)
>>           - Overlapped WSASend()
>>           - Overlapped WSARecv()
>> 
>> 
>>   Don't get me wrong, I grew up with UNIX and love it as much as the
>>   next guy, but you can't deny the usefulness of Windows' facilities
>>   for writing high-performance, multi-threaded IO code.  It's decades
>>   ahead of POSIX.  (Which is also why it bugs me when I see select()
>>   being used on Windows, or IOCP being used as if it were a poll-type
>>   "generic IO multiplexor" -- that's like having a Ferrari and speed
>>   limiting it to 5mph!)
>> 
>>   So, before any of this has a chance of working on Linux/BSD, a lot
>>   more scaffolding will need to be written to provide the things we
>>   get for free on Windows (threadpools being the biggest freebie).
>> 
>> 
>> 
> 
> 
> Have you considered using OpenMP instead of Windows API or POSIX threads directly? OpenMP gives you a thread pool and synchronization primitives for free as well, with no special code needed for Windows or POSIX. 
> 
> OpenBLAS (and GotoBLAS2) uses OpenMP to produce a thread pool on POSIX systems (and actually Windows API on Windows). The OpenMP portion of the C code is wrapped so it looks like sending an asynch task to a thread pool; the C code is not littered with OpenMP pragmas. If you need something like Windows threadpools on POSIX, just look at the BSD licensed OpenBLAS code. It is written to be scalable for the world's largest supercomputers (but also beautifully written and very easy to read).
> 
> Cython has code to register OpenMP threads as Python threads, in case that is needed. So that problem is also solved.
> 
> 
> Sturla
> 
> 
> 
> 
> 
> 
> 
> 

From trent at snakebite.org  Fri Mar 22 04:11:02 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 21 Mar 2013 20:11:02 -0700
Subject: [Python-Dev] Slides from today's parallel/async Python talk
In-Reply-To: <20130321141725.5a1e9d0d@pitrou.net>
References: <20130314020540.GB22505@snakebite.org>
	<5141C0B5.6060904@python.org> <20130314182352.GC24307@snakebite.org>
	<CAAxjCEwJpZXWN4b0XLvHaK6U4UcH44y2Vai2t1QAU-VGkYSbuQ@mail.gmail.com>
	<20130314222337.GG24307@snakebite.org>
	<20130321141725.5a1e9d0d@pitrou.net>
Message-ID: <0FE526DD-12B6-4A30-A68C-CBDB390B54C5@snakebite.org>

http://c2.com/cgi/wiki?BlubParadox

;-)

Sent from my iPhone

On 21 Mar 2013, at 06:18, "Antoine Pitrou" <solipsis at pitrou.net> wrote:

> Le Thu, 14 Mar 2013 15:23:37 -0700,
> Trent Nelson <trent at snakebite.org> a ?crit :
>> 
>>    Don't get me wrong, I grew up with UNIX and love it as much as the
>>    next guy, but you can't deny the usefulness of Windows' facilities
>>    for writing high-performance, multi-threaded IO code.  It's
>> decades ahead of POSIX.
> 
> I suppose that's why all high-performance servers run under Windows.
> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/trent%40snakebite.org

From rovitotv at gmail.com  Fri Mar 22 05:26:36 2013
From: rovitotv at gmail.com (Todd Rovito)
Date: Fri, 22 Mar 2013 00:26:36 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <CADiSq7dFRJdVnYaaK69LgYwQj9kpDZdkmbY6f8LRE23Ec0k4xw@mail.gmail.com>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org> <20130321102023.31b78c7f@pitrou.net>
	<kieqbj$a9e$1@ger.gmane.org>
	<CAG8k2+5uA4F7C=o1y09QtH1LNjAYhH8y2pML5EcZZzF-K+xkAQ@mail.gmail.com>
	<CADiSq7dFRJdVnYaaK69LgYwQj9kpDZdkmbY6f8LRE23Ec0k4xw@mail.gmail.com>
Message-ID: <CALZ1apysoHkxD40tkfrPRJq=oQ1BhGMEMs9ecBsFR3CUgGyQ=g@mail.gmail.com>

On Thu, Mar 21, 2013 at 11:21 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> We can refactor IDLE to make aspects of it easier to test with the
> buildbots, especially now that we have unittest.mock in the standard
> library to mock out some of the UI interaction in the test suite. (I'm
> happy to help coach the IDLE devs on that if they want to start
> improving the test suite coverage for the IDLE code)
Count me in where/when do we start?  Can we perhaps setup a IRC chat with you?

From andrew.svetlov at gmail.com  Fri Mar 22 05:54:30 2013
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Thu, 21 Mar 2013 21:54:30 -0700
Subject: [Python-Dev] [Python-checkins] cpython (2.7): - Issue #13150:
 sysconfig no longer parses the Makefile and config.h files
In-Reply-To: <3ZX26R49RDzSn4@mail.python.org>
References: <3ZX26R49RDzSn4@mail.python.org>
Message-ID: <CAL3CFcVbu8LDwcHM+_vrPKZuafPUCC1HWX2CBZkRyNEAjGgocA@mail.gmail.com>

Great!

On Thu, Mar 21, 2013 at 3:02 PM, matthias.klose
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/66e30c4870bb
> changeset:   82872:66e30c4870bb
> branch:      2.7
> parent:      82843:71adf21421d9
> user:        doko at ubuntu.com
> date:        Thu Mar 21 15:02:16 2013 -0700
> summary:
>   - Issue #13150: sysconfig no longer parses the Makefile and config.h files
>   when imported, instead doing it at build time.  This makes importing
>   sysconfig faster and reduces Python startup time by 20%.
>
> files:
>   Lib/distutils/sysconfig.py |  63 +--------------------
>   Lib/pprint.py              |   5 +-
>   Lib/sysconfig.py           |  75 ++++++++++++++++++++++++-
>   Makefile.pre.in            |  12 +++-
>   Misc/NEWS                  |   4 +
>   5 files changed, 93 insertions(+), 66 deletions(-)
>
>
> diff --git a/Lib/distutils/sysconfig.py b/Lib/distutils/sysconfig.py
> --- a/Lib/distutils/sysconfig.py
> +++ b/Lib/distutils/sysconfig.py
> @@ -387,66 +387,11 @@
>
>  def _init_posix():
>      """Initialize the module as appropriate for POSIX systems."""
> -    g = {}
> -    # load the installed Makefile:
> -    try:
> -        filename = get_makefile_filename()
> -        parse_makefile(filename, g)
> -    except IOError, msg:
> -        my_msg = "invalid Python installation: unable to open %s" % filename
> -        if hasattr(msg, "strerror"):
> -            my_msg = my_msg + " (%s)" % msg.strerror
> -
> -        raise DistutilsPlatformError(my_msg)
> -
> -    # load the installed pyconfig.h:
> -    try:
> -        filename = get_config_h_filename()
> -        parse_config_h(file(filename), g)
> -    except IOError, msg:
> -        my_msg = "invalid Python installation: unable to open %s" % filename
> -        if hasattr(msg, "strerror"):
> -            my_msg = my_msg + " (%s)" % msg.strerror
> -
> -        raise DistutilsPlatformError(my_msg)
> -
> -    # On AIX, there are wrong paths to the linker scripts in the Makefile
> -    # -- these paths are relative to the Python source, but when installed
> -    # the scripts are in another directory.
> -    if python_build:
> -        g['LDSHARED'] = g['BLDSHARED']
> -
> -    elif get_python_version() < '2.1':
> -        # The following two branches are for 1.5.2 compatibility.
> -        if sys.platform == 'aix4':          # what about AIX 3.x ?
> -            # Linker script is in the config directory, not in Modules as the
> -            # Makefile says.
> -            python_lib = get_python_lib(standard_lib=1)
> -            ld_so_aix = os.path.join(python_lib, 'config', 'ld_so_aix')
> -            python_exp = os.path.join(python_lib, 'config', 'python.exp')
> -
> -            g['LDSHARED'] = "%s %s -bI:%s" % (ld_so_aix, g['CC'], python_exp)
> -
> -        elif sys.platform == 'beos':
> -            # Linker script is in the config directory.  In the Makefile it is
> -            # relative to the srcdir, which after installation no longer makes
> -            # sense.
> -            python_lib = get_python_lib(standard_lib=1)
> -            linkerscript_path = string.split(g['LDSHARED'])[0]
> -            linkerscript_name = os.path.basename(linkerscript_path)
> -            linkerscript = os.path.join(python_lib, 'config',
> -                                        linkerscript_name)
> -
> -            # XXX this isn't the right place to do this: adding the Python
> -            # library to the link, if needed, should be in the "build_ext"
> -            # command.  (It's also needed for non-MS compilers on Windows, and
> -            # it's taken care of for them by the 'build_ext.get_libraries()'
> -            # method.)
> -            g['LDSHARED'] = ("%s -L%s/lib -lpython%s" %
> -                             (linkerscript, PREFIX, get_python_version()))
> -
> +    # _sysconfigdata is generated at build time, see the sysconfig module
> +    from _sysconfigdata import build_time_vars
>      global _config_vars
> -    _config_vars = g
> +    _config_vars = {}
> +    _config_vars.update(build_time_vars)
>
>
>  def _init_nt():
> diff --git a/Lib/pprint.py b/Lib/pprint.py
> --- a/Lib/pprint.py
> +++ b/Lib/pprint.py
> @@ -37,7 +37,10 @@
>  import sys as _sys
>  import warnings
>
> -from cStringIO import StringIO as _StringIO
> +try:
> +    from cStringIO import StringIO as _StringIO
> +except ImportError:
> +    from StringIO import StringIO as _StringIO
>
>  __all__ = ["pprint","pformat","isreadable","isrecursive","saferepr",
>             "PrettyPrinter"]
> diff --git a/Lib/sysconfig.py b/Lib/sysconfig.py
> --- a/Lib/sysconfig.py
> +++ b/Lib/sysconfig.py
> @@ -278,9 +278,10 @@
>          return os.path.join(_PROJECT_BASE, "Makefile")
>      return os.path.join(get_path('platstdlib'), "config", "Makefile")
>
> -
> -def _init_posix(vars):
> -    """Initialize the module as appropriate for POSIX systems."""
> +def _generate_posix_vars():
> +    """Generate the Python module containing build-time variables."""
> +    import pprint
> +    vars = {}
>      # load the installed Makefile:
>      makefile = _get_makefile_filename()
>      try:
> @@ -308,6 +309,49 @@
>      if _PYTHON_BUILD:
>          vars['LDSHARED'] = vars['BLDSHARED']
>
> +    # There's a chicken-and-egg situation on OS X with regards to the
> +    # _sysconfigdata module after the changes introduced by #15298:
> +    # get_config_vars() is called by get_platform() as part of the
> +    # `make pybuilddir.txt` target -- which is a precursor to the
> +    # _sysconfigdata.py module being constructed.  Unfortunately,
> +    # get_config_vars() eventually calls _init_posix(), which attempts
> +    # to import _sysconfigdata, which we won't have built yet.  In order
> +    # for _init_posix() to work, if we're on Darwin, just mock up the
> +    # _sysconfigdata module manually and populate it with the build vars.
> +    # This is more than sufficient for ensuring the subsequent call to
> +    # get_platform() succeeds.
> +    name = '_sysconfigdata'
> +    if 'darwin' in sys.platform:
> +        import imp
> +        module = imp.new_module(name)
> +        module.build_time_vars = vars
> +        sys.modules[name] = module
> +
> +    pybuilddir = 'build/lib.%s-%s' % (get_platform(), sys.version[:3])
> +    if hasattr(sys, "gettotalrefcount"):
> +        pybuilddir += '-pydebug'
> +    try:
> +        os.makedirs(pybuilddir)
> +    except OSError:
> +        pass
> +    destfile = os.path.join(pybuilddir, name + '.py')
> +
> +    with open(destfile, 'wb') as f:
> +        f.write('# system configuration generated and used by'
> +                ' the sysconfig module\n')
> +        f.write('build_time_vars = ')
> +        pprint.pprint(vars, stream=f)
> +
> +    # Create file used for sys.path fixup -- see Modules/getpath.c
> +    with open('pybuilddir.txt', 'w') as f:
> +        f.write(pybuilddir)
> +
> +def _init_posix(vars):
> +    """Initialize the module as appropriate for POSIX systems."""
> +    # _sysconfigdata is generated at build time, see _generate_posix_vars()
> +    from _sysconfigdata import build_time_vars
> +    vars.update(build_time_vars)
> +
>  def _init_non_posix(vars):
>      """Initialize the module as appropriate for NT"""
>      # set basic install directories
> @@ -565,3 +609,28 @@
>
>  def get_python_version():
>      return _PY_VERSION_SHORT
> +
> +
> +def _print_dict(title, data):
> +    for index, (key, value) in enumerate(sorted(data.items())):
> +        if index == 0:
> +            print '%s: ' % (title)
> +        print '\t%s = "%s"' % (key, value)
> +
> +
> +def _main():
> +    """Display all information sysconfig detains."""
> +    if '--generate-posix-vars' in sys.argv:
> +        _generate_posix_vars()
> +        return
> +    print 'Platform: "%s"' % get_platform()
> +    print 'Python version: "%s"' % get_python_version()
> +    print 'Current installation scheme: "%s"' % _get_default_scheme()
> +    print
> +    _print_dict('Paths', get_paths())
> +    print
> +    _print_dict('Variables', get_config_vars())
> +
> +
> +if __name__ == '__main__':
> +    _main()
> diff --git a/Makefile.pre.in b/Makefile.pre.in
> --- a/Makefile.pre.in
> +++ b/Makefile.pre.in
> @@ -437,15 +437,20 @@
>                         Modules/python.o \
>                         $(BLDLIBRARY) $(LIBS) $(MODLIBS) $(SYSLIBS) $(LDLAST)
>
> -platform: $(BUILDPYTHON)
> +platform: $(BUILDPYTHON) pybuilddir.txt
>         $(RUNSHARED) $(PYTHON_FOR_BUILD) -c 'import sys ; from sysconfig import get_platform ; print get_platform()+"-"+sys.version[0:3]' >platform
>
> +# Create build directory and generate the sysconfig build-time data there.
> +# pybuilddir.txt contains the name of the build dir and is used for
> +# sys.path fixup -- see Modules/getpath.c.
> +pybuilddir.txt: $(BUILDPYTHON)
> +               $(RUNSHARED) $(PYTHON_FOR_BUILD) -S -m sysconfig --generate-posix-vars
>
>  # Build the shared modules
>  # Under GNU make, MAKEFLAGS are sorted and normalized; the 's' for
>  # -s, --silent or --quiet is always the first char.
>  # Under BSD make, MAKEFLAGS might be " -s -v x=y".
> -sharedmods: $(BUILDPYTHON)
> +sharedmods: $(BUILDPYTHON) pybuilddir.txt
>         @case "$$MAKEFLAGS" in \
>             *\ -s*|s*) quiet="-q";; \
>             *) quiet="";; \
> @@ -955,7 +960,7 @@
>                 else    true; \
>                 fi; \
>         done
> -       @for i in $(srcdir)/Lib/*.py $(srcdir)/Lib/*.doc $(srcdir)/Lib/*.egg-info ; \
> +       @for i in $(srcdir)/Lib/*.py `cat pybuilddir.txt`/_sysconfigdata.py $(srcdir)/Lib/*.doc $(srcdir)/Lib/*.egg-info ; \
>         do \
>                 if test -x $$i; then \
>                         $(INSTALL_SCRIPT) $$i $(DESTDIR)$(LIBDEST); \
> @@ -1133,6 +1138,7 @@
>                 --install-scripts=$(BINDIR) \
>                 --install-platlib=$(DESTSHARED) \
>                 --root=$(DESTDIR)/
> +       -rm $(DESTDIR)$(DESTSHARED)/_sysconfigdata.py*
>
>  # Here are a couple of targets for MacOSX again, to install a full
>  # framework-based Python. frameworkinstall installs everything, the
> diff --git a/Misc/NEWS b/Misc/NEWS
> --- a/Misc/NEWS
> +++ b/Misc/NEWS
> @@ -216,6 +216,10 @@
>  Library
>  -------
>
> +- Issue #13150: sysconfig no longer parses the Makefile and config.h files
> +  when imported, instead doing it at build time.  This makes importing
> +  sysconfig faster and reduces Python startup time by 20%.
> +
>  - Issue #10212: cStringIO and struct.unpack support new buffer objects.
>
>  - Issue #12098: multiprocessing on Windows now starts child processes
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>



-- 
Thanks,
Andrew Svetlov

From rakeshgk21 at hotmail.com  Fri Mar 22 10:28:37 2013
From: rakeshgk21 at hotmail.com (rakesh karanth)
Date: Fri, 22 Mar 2013 14:58:37 +0530
Subject: [Python-Dev] Increase the code coverage of "OS" module
Message-ID: <SNT002-W141DFE16EB503592890A2C6C6D40@phx.gbl>

Hi python-dev,
I'm interested in increasing the code coverage of the Python stdlib library "OS"Can some one who is already working on this or on a similar issue enlighten me on this?
Thanks in advance.
RegardsRakesh.G.K 		 	   		  
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130322/6156aae8/attachment.html>

From solipsis at pitrou.net  Fri Mar 22 10:48:46 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Mar 2013 10:48:46 +0100
Subject: [Python-Dev] IDLE in the stdlib
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
	<20130321191300.4ffe831f@pitrou.net> <kifr3u$hqc$1@ger.gmane.org>
Message-ID: <20130322104846.28b4eac9@pitrou.net>

Le Thu, 21 Mar 2013 21:38:41 +0100,
Georg Brandl <g.brandl at gmx.net> a ?crit :

> Am 21.03.2013 19:13, schrieb Antoine Pitrou:
> > On Wed, 20 Mar 2013 19:57:54 -0700
> > Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> >> 
> >> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org>
> >> wrote:
> >> 
> >> > Right.  Ultimately, I think IDLE should be a separate project
> >> > entirely, but I guess there's push back against that too.
> >> 
> >> The most important feature of IDLE is that it ships with the
> >> standard library. Everyone who clicks on the Windows MSI on the
> >> python.org webpage automatically has IDLE.   That is why I
> >> frequently teach Python with IDLE.
> >> 
> >> If this thread results in IDLE being ripped out of the standard
> >> distribution, then I would likely never use it again.
> > 
> > Which says a lot about its usefulness, if the only reason you use
> > it is that it's bundled with the standard distribution.
> 
> Just like a lot of the stdlib, it *gets* a lot of usefulness from
> being a battery.  But just because there are better/more
> comprehensive/prettier replacements out there is not reason enough to
> remove standard libraries.

That's a good point. I guess it's difficult for me to think of IDLE as
an actual library.

Regards

Antoine.



From g.brandl at gmx.net  Fri Mar 22 17:22:08 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Fri, 22 Mar 2013 17:22:08 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130322104846.28b4eac9@pitrou.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
	<20130321191300.4ffe831f@pitrou.net> <kifr3u$hqc$1@ger.gmane.org>
	<20130322104846.28b4eac9@pitrou.net>
Message-ID: <kii0es$o9s$1@ger.gmane.org>

Am 22.03.2013 10:48, schrieb Antoine Pitrou:
> Le Thu, 21 Mar 2013 21:38:41 +0100,
> Georg Brandl <g.brandl at gmx.net> a ?crit :
> 
>> Am 21.03.2013 19:13, schrieb Antoine Pitrou:
>> > On Wed, 20 Mar 2013 19:57:54 -0700
>> > Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
>> >> 
>> >> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org>
>> >> wrote:
>> >> 
>> >> > Right.  Ultimately, I think IDLE should be a separate project
>> >> > entirely, but I guess there's push back against that too.
>> >> 
>> >> The most important feature of IDLE is that it ships with the
>> >> standard library. Everyone who clicks on the Windows MSI on the
>> >> python.org webpage automatically has IDLE.   That is why I
>> >> frequently teach Python with IDLE.
>> >> 
>> >> If this thread results in IDLE being ripped out of the standard
>> >> distribution, then I would likely never use it again.
>> > 
>> > Which says a lot about its usefulness, if the only reason you use
>> > it is that it's bundled with the standard distribution.
>> 
>> Just like a lot of the stdlib, it *gets* a lot of usefulness from
>> being a battery.  But just because there are better/more
>> comprehensive/prettier replacements out there is not reason enough to
>> remove standard libraries.
> 
> That's a good point. I guess it's difficult for me to think of IDLE as
> an actual library.

You're right, "library" is not a good term, but "battery" certainly is.

Georg


From status at bugs.python.org  Fri Mar 22 18:07:30 2013
From: status at bugs.python.org (Python tracker)
Date: Fri, 22 Mar 2013 18:07:30 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20130322170730.9E9F3568E8@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2013-03-15 - 2013-03-22)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3908 (+20)
  closed 25389 (+73)
  total  29297 (+93)

Open issues with patches: 1735 


Issues opened (56)
==================

#12098: Child process running as debug on Windows
http://bugs.python.org/issue12098  reopened by kristjan.jonsson

#17430: missed peephole optimization
http://bugs.python.org/issue17430  opened by Neal.Norwitz

#17432: PyUnicode_ functions not accessible in Limited API on Windows
http://bugs.python.org/issue17432  opened by bdirks

#17433: stdlib generator-like iterators don't forward send/throw
http://bugs.python.org/issue17433  opened by twouters

#17435: threading.Timer.__init__() should use immutable argument defau
http://bugs.python.org/issue17435  opened by denversc

#17436: hashlib: add a method to hash the content of a file
http://bugs.python.org/issue17436  opened by techtonik

#17437: Difference between open and codecs.open
http://bugs.python.org/issue17437  opened by giampaolo.rodola

#17438: json.load docs should mention that it always return unicode
http://bugs.python.org/issue17438  opened by techtonik

#17441: Do not cache re.compile
http://bugs.python.org/issue17441  opened by serhiy.storchaka

#17442: code.InteractiveInterpreter doesn't display the exception caus
http://bugs.python.org/issue17442  opened by pjenvey

#17444: multiprocessing.cpu_count() should use hw.availcpu on Mac OS X
http://bugs.python.org/issue17444  opened by jszakmeister

#17445: Handle bytes comparisons in difflib.Differ
http://bugs.python.org/issue17445  opened by barry

#17446: doctest test finder doesnt find line numbers of properties
http://bugs.python.org/issue17446  opened by Ronny.Pfannschmidt

#17447: str.identifier shouldn't accept Python keywords
http://bugs.python.org/issue17447  opened by rhettinger

#17449: dev guide appears not to cover the benchmarking suite
http://bugs.python.org/issue17449  opened by dmalcolm

#17453: logging.config.fileConfig error
http://bugs.python.org/issue17453  opened by Alzakath

#17454: ld_so_aix not used when linking c++ (scipy)
http://bugs.python.org/issue17454  opened by alef

#17457: Unittest discover fails with namespace packages and builtin mo
http://bugs.python.org/issue17457  opened by Claudiu.Popa

#17462: argparse FAQ: how it is different from optparse
http://bugs.python.org/issue17462  opened by techtonik

#17468: Generator memory leak
http://bugs.python.org/issue17468  opened by Anssi.K????ri??inen

#17469: Fix sys.getallocatedblocks() when running on valgrind
http://bugs.python.org/issue17469  opened by piotr

#17473: -m is not universally applicable
http://bugs.python.org/issue17473  opened by Devin Jeanpierre

#17475: Better doc on using python-gdb.py
http://bugs.python.org/issue17475  opened by cben

#17477: update the bsddb module do build with db 5.x versions
http://bugs.python.org/issue17477  opened by doko

#17478: Tkinter's split() inconsistent for bytes and unicode strings
http://bugs.python.org/issue17478  opened by serhiy.storchaka

#17479: Fix test discovery for test_io.py
http://bugs.python.org/issue17479  opened by zach.ware

#17480: pyvenv should be installed someplace more obvious on Windows
http://bugs.python.org/issue17480  opened by jason.coombs

#17481: inspect.getfullargspec could use __signature__
http://bugs.python.org/issue17481  opened by michael.foord

#17482: functools.update_wrapper mishandles __wrapped__
http://bugs.python.org/issue17482  opened by ncoghlan

#17483: Can not tell urlopen not to check the hostname for https conne
http://bugs.python.org/issue17483  opened by dwoz

#17484: add tests for getpass
http://bugs.python.org/issue17484  opened by Thomas Fenzl

#17486: datetime.timezone returns the wrong tzname()
http://bugs.python.org/issue17486  opened by lregebro

#17487: wave.Wave_read.getparams should be more user friendly
http://bugs.python.org/issue17487  opened by Claudiu.Popa

#17488: subprocess.Popen bufsize=0 parameter behaves differently in Py
http://bugs.python.org/issue17488  opened by gregory.p.smith

#17489: random.Random implements __getstate__() and __reduce__()
http://bugs.python.org/issue17489  opened by vterron

#17490: Improve ast.literal_eval test suite coverage
http://bugs.python.org/issue17490  opened by ncoghlan

#17491: Consolidate traceback.format_tb and traceback.print_tb
http://bugs.python.org/issue17491  opened by raduv

#17492: Increase test coverage for random (up to 99%)
http://bugs.python.org/issue17492  opened by vterron

#17496: OS X test for Tk availability in runtktests.py doesn't work
http://bugs.python.org/issue17496  opened by alex

#17498: error responses from server are masked in smtplib when server 
http://bugs.python.org/issue17498  opened by r.david.murray

#17500: move PC/icons/source.xar to http://www.python.org/community/lo
http://bugs.python.org/issue17500  opened by doko

#17502: unittest.mock: side_effect iterators ignore DEFAULT
http://bugs.python.org/issue17502  opened by michael.foord

#17504: Dropping duplicated docstring explanation of what Mocks' side_
http://bugs.python.org/issue17504  opened by raduv

#17505: email.header.Header.__unicode__ does not decode header
http://bugs.python.org/issue17505  opened by hniksic

#17506: Improve IDLE news handling
http://bugs.python.org/issue17506  opened by terry.reedy

#17507: To add history time format in readline
http://bugs.python.org/issue17507  opened by Zulu

#17510: assertEquals deprecated in test_program.py (unittest)
http://bugs.python.org/issue17510  opened by grooverdan

#17511: Idle find function closes after each find operation
http://bugs.python.org/issue17511  opened by Kuchinsky

#17512: backport of the _sysconfigdata.py module (issue 13150) breaks 
http://bugs.python.org/issue17512  opened by doko

#17514: Add the license to argparse.py
http://bugs.python.org/issue17514  opened by David.James

#17515: Add sys.setasthook() to allow to use a custom AST optimizer
http://bugs.python.org/issue17515  opened by haypo

#17516: Dead code should be removed
http://bugs.python.org/issue17516  opened by haypo

#17518: urllib2 cannnot handle https and BasicAuth via Proxy.
http://bugs.python.org/issue17518  opened by masato kawamura

#17519: unittest should not try to run abstract classes
http://bugs.python.org/issue17519  opened by ??ric.Piel

#17521: fileConfig() disables any previously-used "named" loggers, eve
http://bugs.python.org/issue17521  opened by Bob.Igo

#17522: Add api PyGILState_Check
http://bugs.python.org/issue17522  opened by kristjan.jonsson



Most recent 15 issues with no replies (15)
==========================================

#17522: Add api PyGILState_Check
http://bugs.python.org/issue17522

#17521: fileConfig() disables any previously-used "named" loggers, eve
http://bugs.python.org/issue17521

#17519: unittest should not try to run abstract classes
http://bugs.python.org/issue17519

#17518: urllib2 cannnot handle https and BasicAuth via Proxy.
http://bugs.python.org/issue17518

#17511: Idle find function closes after each find operation
http://bugs.python.org/issue17511

#17510: assertEquals deprecated in test_program.py (unittest)
http://bugs.python.org/issue17510

#17504: Dropping duplicated docstring explanation of what Mocks' side_
http://bugs.python.org/issue17504

#17502: unittest.mock: side_effect iterators ignore DEFAULT
http://bugs.python.org/issue17502

#17492: Increase test coverage for random (up to 99%)
http://bugs.python.org/issue17492

#17491: Consolidate traceback.format_tb and traceback.print_tb
http://bugs.python.org/issue17491

#17488: subprocess.Popen bufsize=0 parameter behaves differently in Py
http://bugs.python.org/issue17488

#17486: datetime.timezone returns the wrong tzname()
http://bugs.python.org/issue17486

#17481: inspect.getfullargspec could use __signature__
http://bugs.python.org/issue17481

#17479: Fix test discovery for test_io.py
http://bugs.python.org/issue17479

#17478: Tkinter's split() inconsistent for bytes and unicode strings
http://bugs.python.org/issue17478



Most recent 15 issues waiting for review (15)
=============================================

#17522: Add api PyGILState_Check
http://bugs.python.org/issue17522

#17516: Dead code should be removed
http://bugs.python.org/issue17516

#17515: Add sys.setasthook() to allow to use a custom AST optimizer
http://bugs.python.org/issue17515

#17512: backport of the _sysconfigdata.py module (issue 13150) breaks 
http://bugs.python.org/issue17512

#17510: assertEquals deprecated in test_program.py (unittest)
http://bugs.python.org/issue17510

#17504: Dropping duplicated docstring explanation of what Mocks' side_
http://bugs.python.org/issue17504

#17498: error responses from server are masked in smtplib when server 
http://bugs.python.org/issue17498

#17496: OS X test for Tk availability in runtktests.py doesn't work
http://bugs.python.org/issue17496

#17492: Increase test coverage for random (up to 99%)
http://bugs.python.org/issue17492

#17491: Consolidate traceback.format_tb and traceback.print_tb
http://bugs.python.org/issue17491

#17490: Improve ast.literal_eval test suite coverage
http://bugs.python.org/issue17490

#17487: wave.Wave_read.getparams should be more user friendly
http://bugs.python.org/issue17487

#17484: add tests for getpass
http://bugs.python.org/issue17484

#17483: Can not tell urlopen not to check the hostname for https conne
http://bugs.python.org/issue17483

#17479: Fix test discovery for test_io.py
http://bugs.python.org/issue17479



Top 10 most discussed issues (10)
=================================

#17409: resource.setrlimit doesn't respect -1
http://bugs.python.org/issue17409  16 msgs

#17445: Handle bytes comparisons in difflib.Differ
http://bugs.python.org/issue17445  16 msgs

#16475: Support object instancing and recursion in marshal
http://bugs.python.org/issue16475  10 msgs

#17206: Py_XDECREF() expands its argument multiple times
http://bugs.python.org/issue17206  10 msgs

#17429: platform.platform() can throw Unicode error
http://bugs.python.org/issue17429   8 msgs

#17444: multiprocessing.cpu_count() should use hw.availcpu on Mac OS X
http://bugs.python.org/issue17444   8 msgs

#17487: wave.Wave_read.getparams should be more user friendly
http://bugs.python.org/issue17487   8 msgs

#5051: test_update2 in test_os.py invalid due to os.environ.clear() f
http://bugs.python.org/issue5051   7 msgs

#10652: test___all_ + test_tcl fails (Windows installed binary)
http://bugs.python.org/issue10652   7 msgs

#17436: hashlib: add a method to hash the content of a file
http://bugs.python.org/issue17436   7 msgs



Issues closed (72)
==================

#3840: if TESTFN == "/tmp/@test", some tests fail
http://bugs.python.org/issue3840  closed by r.david.murray

#5024: sndhdr.whathdr returns -1 for WAV file frame count
http://bugs.python.org/issue5024  closed by r.david.murray

#5045: imaplib should remove length of literal strings
http://bugs.python.org/issue5045  closed by terry.reedy

#5713: smtplib gets out of sync if server returns a 421 status
http://bugs.python.org/issue5713  closed by r.david.murray

#7720: Errors in tests and C implementation of raw FileIO
http://bugs.python.org/issue7720  closed by r.david.murray

#7898: rlcompleter add "real tab" when text is empty feature
http://bugs.python.org/issue7898  closed by r.david.murray

#8273: move generally useful test.support functions into the unittest
http://bugs.python.org/issue8273  closed by michael.foord

#8862: curses.wrapper does not restore terminal if curses.getkey() ge
http://bugs.python.org/issue8862  closed by r.david.murray

#8905: difflib should accept arbitrary line iterators
http://bugs.python.org/issue8905  closed by terry.reedy

#9506: sqlite3 mogrify - return query string
http://bugs.python.org/issue9506  closed by r.david.murray

#10050: urllib.request still has old 2.x urllib primitives
http://bugs.python.org/issue10050  closed by orsenthil

#10296: ctypes catches BreakPoint error on windows 32
http://bugs.python.org/issue10296  closed by kristjan.jonsson

#11420: Make testsuite pass with -B/DONTWRITEBYTECODE set.
http://bugs.python.org/issue11420  closed by ezio.melotti

#15005: corrupted contents of stdout result from subprocess call under
http://bugs.python.org/issue15005  closed by amaury.forgeotdarc

#15038: Optimize python Locks on Windows
http://bugs.python.org/issue15038  closed by kristjan.jonsson

#15235: allow newer berkeley db versions
http://bugs.python.org/issue15235  closed by doko

#15927: csv.reader() does not support escaped newline when quoting=csv
http://bugs.python.org/issue15927  closed by r.david.murray

#16057: Subclasses of JSONEncoder should not be insturcted to call JSO
http://bugs.python.org/issue16057  closed by r.david.murray

#16709: unittest discover order is filesystem specific - hard to repro
http://bugs.python.org/issue16709  closed by python-dev

#16795: Patch: some changes to AST to make it more useful for static l
http://bugs.python.org/issue16795  closed by python-dev

#16997: subtests
http://bugs.python.org/issue16997  closed by pitrou

#17136: ctypes tests fail with clang on non-OS X
http://bugs.python.org/issue17136  closed by benjamin.peterson

#17192: libffi-3.0.13 import
http://bugs.python.org/issue17192  closed by gregory.p.smith

#17209: get_wch() doesn't handle KeyboardInterrupt
http://bugs.python.org/issue17209  closed by haypo

#17245: ctypes libffi needs to align the x86 stack to 16 bytes
http://bugs.python.org/issue17245  closed by gregory.p.smith

#17285: subprocess.check_output incorrectly state that output is alway
http://bugs.python.org/issue17285  closed by gregory.p.smith

#17398: document url argument of RobotFileParser
http://bugs.python.org/issue17398  closed by terry.reedy

#17415: Clarify docs of os.path.normpath()
http://bugs.python.org/issue17415  closed by terry.reedy

#17416: Clarify docs of os.walk()
http://bugs.python.org/issue17416  closed by terry.reedy

#17417: Documentation Modification Suggestion: os.walk, fwalk
http://bugs.python.org/issue17417  closed by terry.reedy

#17423: libffi on 32bit is broken on linux
http://bugs.python.org/issue17423  closed by gregory.p.smith

#17428: replace readdir to readdir_r in function posix_listdir
http://bugs.python.org/issue17428  closed by Rock

#17431: email.parser module has no attribute BytesFeedParser
http://bugs.python.org/issue17431  closed by r.david.murray

#17434: str literals, which are not docstrings, should not be allowed 
http://bugs.python.org/issue17434  closed by benjamin.peterson

#17439: insufficient error message for failed unicode conversion
http://bugs.python.org/issue17439  closed by r.david.murray

#17440: Some IO related problems on x86 windows
http://bugs.python.org/issue17440  closed by amaury.forgeotdarc

#17443: imaplib.IMAP4_stream subprocess is opened unbuffered but ignor
http://bugs.python.org/issue17443  closed by r.david.murray

#17448: test_sax should skip when no xml parsers are found
http://bugs.python.org/issue17448  closed by r.david.murray

#17450: Failed to build _sqlite3
http://bugs.python.org/issue17450  closed by ezio.melotti

#17451: Test to splitdoc in pydoc.py
http://bugs.python.org/issue17451  closed by Matt.Bachmann

#17452: ftplib raises exception if ssl module is not available
http://bugs.python.org/issue17452  closed by giampaolo.rodola

#17455: ImportError (xml.dom.minidom) in /usr/lib/python2.7/dist-packa
http://bugs.python.org/issue17455  closed by christian.heimes

#17456: os.py (unexpected character)
http://bugs.python.org/issue17456  closed by christian.heimes

#17458: Automatic type conversion from set to frozenset
http://bugs.python.org/issue17458  closed by ezio.melotti

#17459: unittest.assertItemsEqual reports wrong order
http://bugs.python.org/issue17459  closed by Zr40

#17460: Remove the strict and related params completely removing the 0
http://bugs.python.org/issue17460  closed by orsenthil

#17461: Carole should be Carol in PEP 396
http://bugs.python.org/issue17461  closed by benjamin.peterson

#17463: Fix test discovery for test_pdb.py
http://bugs.python.org/issue17463  closed by asvetlov

#17464: Improve Test Coverage Of Pydoc
http://bugs.python.org/issue17464  closed by r.david.murray

#17465: Gut devinabox
http://bugs.python.org/issue17465  closed by brett.cannon

#17466: I can't make assignments to a list.
http://bugs.python.org/issue17466  closed by mark.dickinson

#17467: Enhancement: give mock_open readline() and readlines() methods
http://bugs.python.org/issue17467  closed by python-dev

#17470: random.choice should accept a set as input
http://bugs.python.org/issue17470  closed by rhettinger

#17471: Patch for Additional Test Coverage for urllib.error
http://bugs.python.org/issue17471  closed by orsenthil

#17472: Patch for Additional Test Coverage in urllib.parse
http://bugs.python.org/issue17472  closed by r.david.murray

#17474: Remove the deprecated methods of Request class
http://bugs.python.org/issue17474  closed by orsenthil

#17476: Pydoc allmethods does not return all methods
http://bugs.python.org/issue17476  closed by r.david.murray

#17485: Deleting Request data does not update Content-length header.
http://bugs.python.org/issue17485  closed by r.david.murray

#17494: References to stack bottom are confusing
http://bugs.python.org/issue17494  closed by georg.brandl

#17495: email.quoprimime.body_encode can't handle characters that enco
http://bugs.python.org/issue17495  closed by rpatterson

#17497: Unicode support for HTTP headers in http.client
http://bugs.python.org/issue17497  closed by r.david.murray

#17499: inspect.Signature and inspect.Parameter objects are mutable
http://bugs.python.org/issue17499  closed by r.david.murray

#17501: cannot create a raw string ending in backslash
http://bugs.python.org/issue17501  closed by Thomas Fenzl

#17503: replace mode is always on in console
http://bugs.python.org/issue17503  closed by techtonik

#17508: logging.config.ConvertingDict issue with MemoryHandler
http://bugs.python.org/issue17508  closed by python-dev

#17509: Incorrect package version predicate parsing by distutils
http://bugs.python.org/issue17509  closed by ilja_o

#17513: astrike(*) in argv
http://bugs.python.org/issue17513  closed by r.david.murray

#17517: StringIO() does not behave like cStringIO() when given an arra
http://bugs.python.org/issue17517  closed by pitrou

#17520: Except(ValueError) on Integer returns the input value as 1
http://bugs.python.org/issue17520  closed by amaury.forgeotdarc

#1003195: segfault when running smtplib example
http://bugs.python.org/issue1003195  closed by terry.reedy

#17493: Unskip SysModuleTest.test_recursionlimit_fatalerror on Windows
http://bugs.python.org/issue17493  closed by ezio.melotti

#1691387: Call sys.except_hook if exception occurs in __del__
http://bugs.python.org/issue1691387  closed by r.david.murray

From solipsis at pitrou.net  Fri Mar 22 19:16:39 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Fri, 22 Mar 2013 19:16:39 +0100
Subject: [Python-Dev] cpython (2.7): Issue #17508: Handled out-of-order
 handler configuration correctly.
References: <3ZXTKH6dCxzSJs@mail.python.org>
Message-ID: <20130322191639.7e4b2665@pitrou.net>


Hello,

On Fri, 22 Mar 2013 16:28:19 +0100 (CET)
vinay.sajip <python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/8ae1c28445f8
> changeset:   82881:8ae1c28445f8
> branch:      2.7
> user:        Vinay Sajip <vinay_sajip at yahoo.co.uk>
> date:        Fri Mar 22 15:19:24 2013 +0000
> summary:
>   Issue #17508: Handled out-of-order handler configuration correctly.

Could you explain what "out-of-order handler configuration" means?
Also, could you add a Misc/NEWS entry for the change / bugfix?

Thank you

Antoine.



From francismb at email.de  Fri Mar 22 19:51:34 2013
From: francismb at email.de (francis)
Date: Fri, 22 Mar 2013 19:51:34 +0100
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <kiepea$v2$1@ger.gmane.org>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CABicbJ+uRE+u8trzu8NsFroHxSnStMum-P7-yV3_dDjekhTK-g@mail.gmail.com>
	<CACac1F83g0cKT9ON+1rRQ0UXrTQosRu9V-zU+jyFOOtyjGsqcQ@mail.gmail.com>
	<kiepea$v2$1@ger.gmane.org>
Message-ID: <514CA836.8040009@email.de>


> You can use idle from the command line almost as easily as the CP 
> interpreter: 'python -m idlelib' instead of just 'python' (I just 
> tried it to verify). Unlike bare 'python', IDLE includes a grep. Right 
> click on any 'hit' and it opens the file at the specified line. Unlike 
> bare 'python', you can run tests and collect the all the output, from 
> as many tests as you want, in a dynamically right-sized buffer.
I'm just getting:

~$ python2.7 -m idlelib
/usr/bin/python2.7: No module named idlelib.__main__; 'idlelib' is a 
package and cannot be directly executed

Same with python3...

...but thank you for talking about IDLE (the possibility of reediting 
and using history it so easy that I'm going to give it a try at least 
when I'm on windows...)

From dreamingforward at gmail.com  Fri Mar 22 21:22:38 2013
From: dreamingforward at gmail.com (Mark Janssen)
Date: Fri, 22 Mar 2013 13:22:38 -0700
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130322104846.28b4eac9@pitrou.net>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<20130320180942.20A6A2500B3@webabinitio.net>
	<CAF-Rda8DqRrFz1uZ4A49GhU+X+WaRDhwt=8HedwUKpqWaPgxCA@mail.gmail.com>
	<20130320100907.22a24586@anarchist>
	<CAPZV6o-oEPEVQZ-xmFhea7CWmzLsPstKzZthDZqNNbcqMuU0fg@mail.gmail.com>
	<CAP7+vJKYZWbVX829=RanZnGw1tQ=vaWYJOn-UhS4RMvJGNr30g@mail.gmail.com>
	<20130320123833.14886a86@anarchist>
	<8632CEBE-50A9-414C-8E43-B53F427DB8D6@gmail.com>
	<20130321191300.4ffe831f@pitrou.net> <kifr3u$hqc$1@ger.gmane.org>
	<20130322104846.28b4eac9@pitrou.net>
Message-ID: <CAMjeLr88gtzqYLi9gqKmX_KWi++ty9i0rg5fw7oi_wrCkfNk_A@mail.gmail.com>

On Fri, Mar 22, 2013 at 2:48 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> Le Thu, 21 Mar 2013 21:38:41 +0100,
> Georg Brandl <g.brandl at gmx.net> a ?crit :
>
> > Am 21.03.2013 19:13, schrieb Antoine Pitrou:
> > > On Wed, 20 Mar 2013 19:57:54 -0700
> > > Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> > >>
> > >> On Mar 20, 2013, at 12:38 PM, Barry Warsaw <barry at python.org>
> > >> wrote:
> > >>
> > >> > Right.  Ultimately, I think IDLE should be a separate project
> > >> > entirely, but I guess there's push back against that too.
> > >>
> > >> The most important feature of IDLE is that it ships with the
> > >> standard library. Everyone who clicks on the Windows MSI on the
> > >> python.org webpage automatically has IDLE.   That is why I
> > >> frequently teach Python with IDLE.
> > >>
> > >> If this thread results in IDLE being ripped out of the standard
> > >> distribution, then I would likely never use it again.
> > >
> > > Which says a lot about its usefulness, if the only reason you use
> > > it is that it's bundled with the standard distribution.
> >
> > Just like a lot of the stdlib, it *gets* a lot of usefulness from
> > being a battery.  But just because there are better/more
> > comprehensive/prettier replacements out there is not reason enough to
> > remove standard libraries.
>
> That's a good point. I guess it's difficult for me to think of IDLE as
> an actual library.
>
>
It's not a library.  It's an application that is bundled in the standard
distribution.

Mark
Tacoma, Washington.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130322/3ec81bf6/attachment.html>

From rz1991 at foxmail.com  Fri Mar 22 22:34:44 2013
From: rz1991 at foxmail.com (=?gb18030?B?yO7vow==?=)
Date: Sat, 23 Mar 2013 05:34:44 +0800
Subject: [Python-Dev] Interested in GSoC for biopython
Message-ID: <tencent_7A39AD2C5B1B74A85C0E694D@qq.com>

Hi,

I'm Zheng from the University of Georgia. I heard about the GSoC several weeks before and found biopython also involved in the peoject. I plan to apply for the GSoC 2013, hoping to make some contributions this summer. I browsed the proposals in biopython wiki sites and find two of them are all interesting topics, especially the codon alignment functionality. I know this has been implemented by pal2nal, and pal2nal is a good program as it accounts for the mismatches between protein and DNA sequences. However, it may raise an error when the protein sequence contains * indicating a stop codon, which is typical when the sequence is translated from genomic DNA. Maybe I could write a python implementation that relax this requirement. Many interesting statistical tests based on codon alignment can also be implemented. As I am new to this group, can anyone give me some suggestions about what I could do while preparing my proposal? Do I need to read the souce code of some major classes in BioPython to better understand how it works as well as the programming style? Thanks.


Best,
Zheng Ruan
Institute of Bioinformatics
The University of Georgia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130323/7ed01825/attachment.html>

From oscar.j.benjamin at gmail.com  Fri Mar 22 22:43:35 2013
From: oscar.j.benjamin at gmail.com (Oscar Benjamin)
Date: Fri, 22 Mar 2013 21:43:35 +0000
Subject: [Python-Dev] Interested in GSoC for biopython
In-Reply-To: <tencent_7A39AD2C5B1B74A85C0E694D@qq.com>
References: <tencent_7A39AD2C5B1B74A85C0E694D@qq.com>
Message-ID: <CAHVvXxQNdnpiKXPdn54UeR57SMuCSyrPMPiUmM__+Hhq9vWxsw@mail.gmail.com>

On 22 March 2013 21:34, ?? <rz1991 at foxmail.com> wrote:
> Hi,
>
> I'm Zheng from the University of Georgia. I heard about the GSoC several
> weeks before and found biopython also involved in the peoject. I plan to
> apply for the GSoC 2013, hoping to make some contributions this summer.
[SNIP]

This mailing list is for development of Python, not biopython. Try
asking on one of the biopython mailing lists:
http://biopython.org/wiki/Mailing_lists


Oscar

From kbk at shore.net  Sat Mar 23 00:59:18 2013
From: kbk at shore.net (Kurt B. Kaiser)
Date: Fri, 22 Mar 2013 19:59:18 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <20130321103750.311cb6f2@pitrou.net>
References: <E1UIRP3-0004SF-PR@smtp3.shore.net>
	<1363819695.3885.140661207062389.589C7405@webmail.messagingengine.com>
	<20130321103750.311cb6f2@pitrou.net>
Message-ID: <1363996758.25015.140661208001405.3E2993BA@webmail.messagingengine.com>



On Thu, Mar 21, 2013, at 05:37 AM, Antoine Pitrou wrote:
> Le Wed, 20 Mar 2013 18:48:15 -0400, "Kurt B. Kaiser" <kbk at shore.net>
> a ?crit :
> >
> > IDLE has a single keystroke round trip - it's an IDE, not just an
> > editor like Sublime Text or Notepad.  In the 21st century, people
> > expect some sort of IDE.  Or, they should!
>
> I don't think I've used an IDE in years (not seriously anyway).

If you haven't used IDLE lately, you might want to try it.

> I also don't think beginners "expect some sort of IDE", since they
> don't know what it is. They probably don't even expect a text editor
> at first.

Well, they will feel the need in less than a day, IMHO.  These days,
beginning users are accustomed to a GUI that "does something", not a
command line, it seems.

Right, they don't know what they need, at first.  We should provide an
interface that, in our experience, meets a beginner's needs.

>
> > I'd also like to make a plea to keep IDLE's interface clean and
> > basic. There are lots of complex IDEs available for those who want
> > them.  It's natural for developers to add features, that's what they
> > do :-), but you don't hand a novice a Ferrari (or emacs) and expect
> > good results.
>
> What is the point of an IDE without features?

None.

But IDLE has plenty of features - and minimum clutter.  It also works
very well on small screens.


> Also, this is touching another issue: IDLE needs active maintainers,
> who will obviously be experienced Python developers. But if they are
> experienced Python developers, they will certainly want the additional
> features, otherwise's they'll stop using and maintaining IDLE.
>
> In other words, if IDLE were actually usable *and* pleasant for
> experienced developers, I'm sure more developers would be motivated to
> improve and maintain it.

That's not the target audience for IDLE.  There are many great IDEs for
"experienced" developers.

It's not my objective to turn IDLE into PyCharm, just to keep some
developers motivated.  Good design satisfies the target audience - IMHO,
we should be working towards the best possible beginner Python interface
on Windows, Mac, and Raspberry Pi.

To get this done, we need IDLE developers who are interested in
supporting beginners.  Not so much developers who are interested in
adding complex features for their more advanced usage.

The complex IDE space is packed - it doesn't need another entry.  OTOH,
there are few simple IDEs like IDLE.  It's a good niche to be in.

And, yes, getting the IDLE developers to use IDLE is important - I do
so most of the time (emacs for the rest :).  That helps to discover
IDLE and tkinter bugs, and occasionally exposes the need for a
missing feature.

>
> > It's sometimes said that IDLE is "ugly" or "broken".  These terms
> > are subjective!
>
> Subjective statements are not baseless and idiotic.

Please don't put words in my mouth.  I only said those terms are
subjective, which they are.

> They come from the experience of people actually wanting to like a
> piece of software, you shouldn't discard them at face value.

Please don't allege actions I haven't taken!

I agree entirely with you - one has to dig deeper to extract some
constructive criticism, if it is available.

Often, you can't address one person's idea of ugly or broken without
raising those issues with another person.

-- 
KBK

From vinay_sajip at yahoo.co.uk  Sat Mar 23 02:08:26 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 23 Mar 2013 01:08:26 +0000 (UTC)
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on
	Windows
References: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>
Message-ID: <loom.20130323T020643-926@post.gmane.org>

Paul Moore <p.f.moore <at> gmail.com> writes:

> I don't understand what this is saying - can someone clarify the
> reason behind this statement? What is different about a
> "non-system-wide installation" that causes this issue (I assume
> "non-system-wide" means "not All Users")? The reason I ask is that
> virtualenv doesn't do this, and I'm not clear if this is because of a
> potential bug lurking in virtualenv (in which case, I'd like to find
> out how to reproduce it) or because virtualenv takes a different
> approach which avoids this issue somehow.

One example of a non-system-wide installation is a source build of Python.
PEP 405 venvs created from a source build should work in the same way as venvs
created using an installed Python.

Regards,

Vinay Sajip


From fijall at gmail.com  Sat Mar 23 05:35:37 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 22 Mar 2013 21:35:37 -0700
Subject: [Python-Dev] Increase the code coverage of "OS" module
In-Reply-To: <SNT002-W141DFE16EB503592890A2C6C6D40@phx.gbl>
References: <SNT002-W141DFE16EB503592890A2C6C6D40@phx.gbl>
Message-ID: <CAK5idxRz3C8_N+Q8k5_phYQRsNA+5UETp2Y7tbJtmyfRmxcj-Q@mail.gmail.com>

On Fri, Mar 22, 2013 at 2:28 AM, rakesh karanth <rakeshgk21 at hotmail.com> wrote:
> Hi python-dev,
>
> I'm interested in increasing the code coverage of the Python stdlib library
> "OS"
> Can some one who is already working on this or on a similar issue enlighten
> me on this?
>
> Thanks in advance.

Hey

You can check out pypy os tests, we cover a bit more than CPython.
(it's py.test based-though you might need to adapt it)

From vinay_sajip at yahoo.co.uk  Sat Mar 23 10:38:44 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sat, 23 Mar 2013 09:38:44 +0000 (UTC)
Subject: [Python-Dev] cpython (2.7): Issue #17508: Handled out-of-order
	handler configuration correctly.
References: <3ZXTKH6dCxzSJs@mail.python.org>
	<20130322191639.7e4b2665@pitrou.net>
Message-ID: <loom.20130323T103438-451@post.gmane.org>

Antoine Pitrou <solipsis <at> pitrou.net> writes:

>>   Issue #17508: Handled out-of-order handler configuration correctly.
> Could you explain what "out-of-order handler configuration" means?

In logging, a MemoryHandler buffers records and has a reference to another
handler - the "target" - which does the actual output of the buffered records.

It can happen when using dictConfig() that the MemoryHandler is configured before
the target handler, and in this case the reference to the target wasn't set up
correctly. The commit rectifies this.

> Also, could you add a Misc/NEWS entry for the change / bugfix?

Ok, will do.

Regards,

Vinay Sajip


From p.f.moore at gmail.com  Sat Mar 23 11:06:46 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 23 Mar 2013 10:06:46 +0000
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on
	Windows
In-Reply-To: <loom.20130323T020643-926@post.gmane.org>
References: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>
	<loom.20130323T020643-926@post.gmane.org>
Message-ID: <CACac1F-iabCEM-9EcFmaOHuyb=x-uDVfOwjQWDfSdoGZW7yFQw@mail.gmail.com>

On 23 March 2013 01:08, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Paul Moore <p.f.moore <at> gmail.com> writes:
>
>> I don't understand what this is saying - can someone clarify the
>> reason behind this statement? What is different about a
>> "non-system-wide installation" that causes this issue (I assume
>> "non-system-wide" means "not All Users")? The reason I ask is that
>> virtualenv doesn't do this, and I'm not clear if this is because of a
>> potential bug lurking in virtualenv (in which case, I'd like to find
>> out how to reproduce it) or because virtualenv takes a different
>> approach which avoids this issue somehow.
>
> One example of a non-system-wide installation is a source build of Python.
> PEP 405 venvs created from a source build should work in the same way as venvs
> created using an installed Python.

Thanks. I hadn't thought of that case. However, I'm still not entirely
clear *why* the DLLs need to be copied. I'll set up a source build and
test virtualenv against it to see if it fails. Assuming it does, I
should be able to work out what the issue is from that.

Paul

From stefan.bucur at gmail.com  Sat Mar 23 12:05:13 2013
From: stefan.bucur at gmail.com (Stefan Bucur)
Date: Sat, 23 Mar 2013 12:05:13 +0100
Subject: [Python-Dev] Are undocumented exceptions considered bugs?
Message-ID: <CAEg840W=g+rRa6-x6PKky2v_z7-ix7Vu=zWyZgjU0iT4Ex6pSA@mail.gmail.com>

Hi,

I'm not sure this is the right place to ask this question, but I thought
I'd give it a shot since it also concerns the Python standard library.

I'm writing an automated test case generation tool for Python programs that
explores all possible execution paths through a program. When applying this
tool on Python's 2.7.3 urllib package, it discovered input strings for
which the urllib.urlopen(url) call would raise a TypeError. For instance:

urllib.urlopen('\x00\x00\x00')

[...]
  File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 86, in
urlopen
    return opener.open(url)
  File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 207, in
open
    return getattr(self, name)(url)
  File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 462, in
open_file
    return self.open_local_file(url)
  File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 474, in
open_local_file
    stats = os.stat(localname)
TypeError: must be encoded string without NULL bytes, not str

In the urllib documentation it is only mentioned that the IOError is raised
when the connection cannot be established. Since the input passed is a
string (and not some other type), is the TypeError considered a bug (either
in the documentation, or in the implementation)?

Thanks a lot,
Stefan
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130323/fcf304da/attachment.html>

From shibturn at gmail.com  Sat Mar 23 13:57:02 2013
From: shibturn at gmail.com (Richard Oudkerk)
Date: Sat, 23 Mar 2013 12:57:02 +0000
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on
	Windows
In-Reply-To: <CACac1F-iabCEM-9EcFmaOHuyb=x-uDVfOwjQWDfSdoGZW7yFQw@mail.gmail.com>
References: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>
	<loom.20130323T020643-926@post.gmane.org>
	<CACac1F-iabCEM-9EcFmaOHuyb=x-uDVfOwjQWDfSdoGZW7yFQw@mail.gmail.com>
Message-ID: <kik8r0$on$1@ger.gmane.org>

On 23/03/2013 10:06am, Paul Moore wrote:
>> One example of a non-system-wide installation is a source build of Python.
>> PEP 405 venvs created from a source build should work in the same way as venvs
>> created using an installed Python.
>
> Thanks. I hadn't thought of that case. However, I'm still not entirely
> clear *why* the DLLs need to be copied. I'll set up a source build and
> test virtualenv against it to see if it fails. Assuming it does, I
> should be able to work out what the issue is from that.

Also, couldn't hard links be used instead of copying?  (This will fail 
if not on the same NTFS partition, but then one can copy as a fallback.)

-- 
Richard


From solipsis at pitrou.net  Sat Mar 23 13:55:41 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 23 Mar 2013 13:55:41 +0100
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on
	Windows
References: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>
	<loom.20130323T020643-926@post.gmane.org>
	<CACac1F-iabCEM-9EcFmaOHuyb=x-uDVfOwjQWDfSdoGZW7yFQw@mail.gmail.com>
	<kik8r0$on$1@ger.gmane.org>
Message-ID: <20130323135541.75c3cda6@pitrou.net>

On Sat, 23 Mar 2013 12:57:02 +0000
Richard Oudkerk <shibturn at gmail.com> wrote:

> On 23/03/2013 10:06am, Paul Moore wrote:
> >> One example of a non-system-wide installation is a source build of Python.
> >> PEP 405 venvs created from a source build should work in the same way as venvs
> >> created using an installed Python.
> >
> > Thanks. I hadn't thought of that case. However, I'm still not entirely
> > clear *why* the DLLs need to be copied. I'll set up a source build and
> > test virtualenv against it to see if it fails. Assuming it does, I
> > should be able to work out what the issue is from that.
> 
> Also, couldn't hard links be used instead of copying?  (This will fail 
> if not on the same NTFS partition, but then one can copy as a fallback.)

Hard links are generally hard to discover and debug (at least under
Unix, but I suppose the same applies under Windows).

Regards

Antoine.



From p.f.moore at gmail.com  Sat Mar 23 14:51:41 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Sat, 23 Mar 2013 13:51:41 +0000
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on
	Windows
In-Reply-To: <20130323135541.75c3cda6@pitrou.net>
References: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>
	<loom.20130323T020643-926@post.gmane.org>
	<CACac1F-iabCEM-9EcFmaOHuyb=x-uDVfOwjQWDfSdoGZW7yFQw@mail.gmail.com>
	<kik8r0$on$1@ger.gmane.org> <20130323135541.75c3cda6@pitrou.net>
Message-ID: <CACac1F_sDD0MLqF4HyE2FmV7CwUvvu-9BXS-u8d9=D44AmXLnw@mail.gmail.com>

On 23 March 2013 12:55, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 23 Mar 2013 12:57:02 +0000
> Richard Oudkerk <shibturn at gmail.com> wrote:
>
>> On 23/03/2013 10:06am, Paul Moore wrote:
>> >> One example of a non-system-wide installation is a source build of Python.
>> >> PEP 405 venvs created from a source build should work in the same way as venvs
>> >> created using an installed Python.
>> >
>> > Thanks. I hadn't thought of that case. However, I'm still not entirely
>> > clear *why* the DLLs need to be copied. I'll set up a source build and
>> > test virtualenv against it to see if it fails. Assuming it does, I
>> > should be able to work out what the issue is from that.
>>
>> Also, couldn't hard links be used instead of copying?  (This will fail
>> if not on the same NTFS partition, but then one can copy as a fallback.)
>
> Hard links are generally hard to discover and debug (at least under
> Unix, but I suppose the same applies under Windows).

Yes. And links in general are less common, and so more of a surprise,
on Windows as well.
Paul

From ncoghlan at gmail.com  Sat Mar 23 16:21:53 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 23 Mar 2013 08:21:53 -0700
Subject: [Python-Dev] Are undocumented exceptions considered bugs?
In-Reply-To: <CAEg840W=g+rRa6-x6PKky2v_z7-ix7Vu=zWyZgjU0iT4Ex6pSA@mail.gmail.com>
References: <CAEg840W=g+rRa6-x6PKky2v_z7-ix7Vu=zWyZgjU0iT4Ex6pSA@mail.gmail.com>
Message-ID: <CADiSq7d=9kaQ_dsUDzEb2em7HF4g2ypzeCaSs4C2G8wXYWVwKw@mail.gmail.com>

On Sat, Mar 23, 2013 at 4:05 AM, Stefan Bucur <stefan.bucur at gmail.com> wrote:
> Hi,
>
> I'm not sure this is the right place to ask this question, but I thought I'd
> give it a shot since it also concerns the Python standard library.

It's the right place to ask :)

> I'm writing an automated test case generation tool for Python programs that
> explores all possible execution paths through a program. When applying this
> tool on Python's 2.7.3 urllib package, it discovered input strings for which
> the urllib.urlopen(url) call would raise a TypeError.

That sounds like a really interesting tool.

> For instance:
>
> urllib.urlopen('\x00\x00\x00')
>
> [...]
>   File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 86, in
> urlopen
>     return opener.open(url)
>   File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 207, in
> open
>     return getattr(self, name)(url)
>   File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 462, in
> open_file
>     return self.open_local_file(url)
>   File "/home/bucur/onion/python-bin/lib/python2.7/urllib.py", line 474, in
> open_local_file
>     stats = os.stat(localname)
> TypeError: must be encoded string without NULL bytes, not str
>
> In the urllib documentation it is only mentioned that the IOError is raised
> when the connection cannot be established. Since the input passed is a
> string (and not some other type), is the TypeError considered a bug (either
> in the documentation, or in the implementation)?

The general answer is that there are certain exceptions that usually
aren't documented because almost all code can trigger them if you pass
the right kind of invalid argument. For example, almost any API can
emit TypeError or AttributeError if you pass an instance of the wrong
type, and many can emit ValueError, IndexError or KeyError if you pass
an incorrect value. Other errors like SyntaxError, ImportError,
NameError and UnboundLocalError usually indicate bugs or environmental
configuration issues, so are also typically omitted when documenting
the possible exceptions for particular APIs.

In this specific case, the error message is
confusing-but-not-really-wrong, due to the "two-types-in-one" nature
of Python 2.x strings - 8-bit strings are used as both text sequences
(generally not containing NUL characters) and also as arbitrary binary
data, including encoded text (quite likely to contain NUL bytes).

I think a bug report for this would be appropriate, with the aim of
making that error message less confusing (it's a fairly obscure case,
though).

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From luca.sbardella at gmail.com  Sat Mar 23 20:05:32 2013
From: luca.sbardella at gmail.com (Luca Sbardella)
Date: Sat, 23 Mar 2013 19:05:32 +0000
Subject: [Python-Dev] wsgi validator with asynchronous handlers/servers
Message-ID: <CAF94-jZiMuH4_O5Kzo0CsjG-oQmk3+uB0wArXuQWFZ4qxbcdbA@mail.gmail.com>

Hi,

I have an asynchronous wsgi application handler which yields empty bytes
before it is ready to yield the response body and, importantly, to call
start_response.

Something like this:

def wsgi_handler(environ, start_response):
        body = generate_body(environ)
        body = maybe_async(body)
        while is_async(body):
            yield b''
        start_response(...)
        ...

I started using wsgiref.validator recently, nice little gem in the standard
lib, and I discovered that the above handler does not validate! Disaster.
Reading pep 3333

"the application *must* invoke the start_response() callable before the
iterable yields its first body bytestring, so that the server can send the
headers before any body content. However, this invocation *may* be
performed by the iterable's first iteration, so servers *must not* assume
that start_response() has been called before they begin iterating over the
iterable."

The pseudocode above does yields bytes before start_response, but they are
not *body* bytes, they are empty bytes so that the asynchronous wsgi server
releases the eventloop and call back at the next eventloop iteration.

I'm I misinterpreting the pep, or the wsgi validator should be
fixed accordingly?

Regards,
Luca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130323/20d85ed7/attachment.html>

From benjamin at python.org  Sat Mar 23 20:19:40 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Sat, 23 Mar 2013 14:19:40 -0500
Subject: [Python-Dev] wsgi validator with asynchronous handlers/servers
In-Reply-To: <CAF94-jZiMuH4_O5Kzo0CsjG-oQmk3+uB0wArXuQWFZ4qxbcdbA@mail.gmail.com>
References: <CAF94-jZiMuH4_O5Kzo0CsjG-oQmk3+uB0wArXuQWFZ4qxbcdbA@mail.gmail.com>
Message-ID: <CAPZV6o_pZTUNA8LHp4m1iAV0n+nqtXfNVzx5TJ-fd2UZv5ZQ=g@mail.gmail.com>

Hi,
The people who best understand WSGI are to be found on the Web-SIG:
http://mail.python.org/mailman/listinfo/web-sig

2013/3/23 Luca Sbardella <luca.sbardella at gmail.com>:
> Hi,
>
> I have an asynchronous wsgi application handler which yields empty bytes
> before it is ready to yield the response body and, importantly, to call
> start_response.
>
> Something like this:
>
> def wsgi_handler(environ, start_response):
>         body = generate_body(environ)
>         body = maybe_async(body)
>         while is_async(body):
>             yield b''
>         start_response(...)
>         ...
>
> I started using wsgiref.validator recently, nice little gem in the standard
> lib, and I discovered that the above handler does not validate! Disaster.
> Reading pep 3333
>
> "the application must invoke the start_response() callable before the
> iterable yields its first body bytestring, so that the server can send the
> headers before any body content. However, this invocation may be performed
> by the iterable's first iteration, so servers must not assume that
> start_response() has been called before they begin iterating over the
> iterable."
>
> The pseudocode above does yields bytes before start_response, but they are
> not *body* bytes, they are empty bytes so that the asynchronous wsgi server
> releases the eventloop and call back at the next eventloop iteration.
>
> I'm I misinterpreting the pep, or the wsgi validator should be fixed
> accordingly?
>
> Regards,
> Luca
>
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe:
> http://mail.python.org/mailman/options/python-dev/benjamin%40python.org
>



-- 
Regards,
Benjamin

From tim.delaney at aptare.com  Sat Mar 23 20:54:40 2013
From: tim.delaney at aptare.com (Tim Delaney)
Date: Sun, 24 Mar 2013 06:54:40 +1100
Subject: [Python-Dev] PEP 405 (venv) - why does it copy the DLLs on
	Windows
In-Reply-To: <20130323135541.75c3cda6@pitrou.net>
References: <CACac1F_=6SAvd9qaxubH-vpZSvGkqBx3Qa04m2JuQtA7DPRutA@mail.gmail.com>
	<loom.20130323T020643-926@post.gmane.org>
	<CACac1F-iabCEM-9EcFmaOHuyb=x-uDVfOwjQWDfSdoGZW7yFQw@mail.gmail.com>
	<kik8r0$on$1@ger.gmane.org> <20130323135541.75c3cda6@pitrou.net>
Message-ID: <CAN8CLgnyO=PCAzPSR7XxKSrcMe_c8_Q9CaoiWqHs4iu5tqvMdw@mail.gmail.com>

On 23 March 2013 23:55, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Sat, 23 Mar 2013 12:57:02 +0000
> Richard Oudkerk <shibturn at gmail.com> wrote:
>
> > Also, couldn't hard links be used instead of copying?  (This will fail
> > if not on the same NTFS partition, but then one can copy as a fallback.)
>
> Hard links are generally hard to discover and debug (at least under
> Unix, but I suppose the same applies under Windows).
>

(Slightly OT, but I think useful in this case.)

That's what the Link Shell Extension <
http://schinagl.priv.at/nt/hardlinkshellext/hardlinkshellext.html> is for.

Makes it very easy to work with Hardlinks, Symbolic links, Junctions and
Volume Mountpoints. It gives different overlays for each to icons in
Explorer (and Save/Open dialogs) and adds a tab to the properties of any
link which gives details e.g. for hardlinks it displays the reference count
and all the hardlinks to the same file.

There's also a command-line version - ln <
http://schinagl.priv.at/nt/ln/ln.html>.

Highly recommended.

Tim Delaney
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130324/caa80ba2/attachment.html>

From pje at telecommunity.com  Sat Mar 23 21:27:31 2013
From: pje at telecommunity.com (PJ Eby)
Date: Sat, 23 Mar 2013 16:27:31 -0400
Subject: [Python-Dev] wsgi validator with asynchronous handlers/servers
In-Reply-To: <CAF94-jZiMuH4_O5Kzo0CsjG-oQmk3+uB0wArXuQWFZ4qxbcdbA@mail.gmail.com>
References: <CAF94-jZiMuH4_O5Kzo0CsjG-oQmk3+uB0wArXuQWFZ4qxbcdbA@mail.gmail.com>
Message-ID: <CALeMXf6HzuehHNufewiBaE4CgAB1dNbOQkAB6txZ=FJAi=yfLA@mail.gmail.com>

On Sat, Mar 23, 2013 at 3:05 PM, Luca Sbardella
<luca.sbardella at gmail.com> wrote:
> The pseudocode above does yields bytes before start_response, but they are
> not *body* bytes, they are empty bytes so that the asynchronous wsgi server
> releases the eventloop and call back at the next eventloop iteration.
>
> I'm I misinterpreting the pep, or the wsgi validator should be fixed
> accordingly?

The validator is correct for the spec.  You *must* call
start_response() before yielding any strings at all.

From jeanpierreda at gmail.com  Sun Mar 24 02:09:50 2013
From: jeanpierreda at gmail.com (Devin Jeanpierre)
Date: Sat, 23 Mar 2013 21:09:50 -0400
Subject: [Python-Dev] Are undocumented exceptions considered bugs?
In-Reply-To: <CADiSq7d=9kaQ_dsUDzEb2em7HF4g2ypzeCaSs4C2G8wXYWVwKw@mail.gmail.com>
References: <CAEg840W=g+rRa6-x6PKky2v_z7-ix7Vu=zWyZgjU0iT4Ex6pSA@mail.gmail.com>
	<CADiSq7d=9kaQ_dsUDzEb2em7HF4g2ypzeCaSs4C2G8wXYWVwKw@mail.gmail.com>
Message-ID: <CABicbJK0pyJ4NhjBujsiuevQS8NLp51zEJ_0g6JbVbZCaujftA@mail.gmail.com>

On Sat, Mar 23, 2013 at 11:21 AM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> In this specific case, the error message is
> confusing-but-not-really-wrong, due to the "two-types-in-one" nature
> of Python 2.x strings - 8-bit strings are used as both text sequences
> (generally not containing NUL characters) and also as arbitrary binary
> data, including encoded text (quite likely to contain NUL bytes).

With your terminology, three types: char, non-NUL-text, encoded-text
(e.g. what happens with ord('ab'))

That's pretty silly, considering that these are all one Python type,
and TypeError is raised into Python code. Obviously it can't change,
because of historical reasons, but documenting it would be
straightforward and helpful. These are not errors you can just infer
will happen, you need to see it via documentation, reading the source,
or experimentation (and re experimentation, then you have to establish
whether or not this was an accident or deliberate).

-- Devin

From benjamin at python.org  Mon Mar 25 01:30:06 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Sun, 24 Mar 2013 20:30:06 -0400
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
Message-ID: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>

I'm happy to announce the first release candidate of 2.7.4.

2.7.4 will be the latest maintenance release in the Python 2.7 series.
It includes hundreds of bugfixes to the core language and standard
library.

There has recently been a lot of discussion about XML-based denial of service
attacks. Specifically, certain XML files can cause XML parsers, including ones
in the Python stdlib, to consume gigabytes of RAM and swamp the CPU. 2.7.4 does
not include any changes in Python XML code to address these issues. Interested
parties should examine the defusedxml package on PyPI:
https://pypi.python.org/pypi/defusedxml

2.7.4 release candidate 1 is a testing release. Deploying it in production is
not recommended. However, please download it and test with your libraries and
applications, reporting any bugs you may find.

Assuming no horrible bugs rear their heads, a final release of 2.7.4 will occur
in 2 weeks.

Downloads are at

    http://python.org/download/releases/2.7.4/

As always, please report bugs to

    http://bugs.python.org/

Enjoy,
Benjamin Peterson
2.7 Release Manager

From rovitotv at gmail.com  Mon Mar 25 20:30:59 2013
From: rovitotv at gmail.com (Todd Rovito)
Date: Mon, 25 Mar 2013 15:30:59 -0400
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
Message-ID: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>

Python 2.7.4 release candidate was just announced and is ready for testing here:
http://python.org/download/releases/2.7.4/

Now the clock is ticking and we have two weeks to get IDLE issues
pushed into CPython before the final release of 2.7.4.  Below is my
list of low risk issues that would be great to get pushed into
CPython.  I hope this email will encourage CPython Core Developers to
make the commits or tell us what we can do, like more testing or
better documentation, to get these issues cleaned up and committed
before 2.7.4 ships.  PEP-434 is still being discussed, I asked PEP
authors to post latest version on 3/24, but I think most folks agree
with the principle.  IDLE might have more issues that could be fixed
easily so feel free to add to the list.  Thanks for everybody?s hard
work to make IDLE better!!!!!
http://bugs.python.org/issue7136 Idle File Menu Option Improvement
http://bugs.python.org/issue17390 display python version on idle title bar
http://bugs.python.org/issue17511 Idle find function closes after each
find operation
http://bugs.python.org/issue6699 IDLE: Warn user about overwriting a
file that has a newer version on filesystem
http://bugs.python.org/issue10747 Include version info in Windows
shortcuts the issue is for 3.x but perhaps simple to implement for
2.7?

The issue below has been talked about in the recent past on idle-dev
(http://mail.python.org/pipermail/idle-dev/2013-March/003228.html) it
will take some work to get committed because of the deletion of the
Option menu on the Mac.  But even if it gets committed then is
disabled for the Mac with a simple sys.platform it would be a huge
victory for IDLE in my mind.
http://bugs.python.org/issue2704 IDLE: Patch to make PyShell behave
more like a Terminal interface

From benjamin at python.org  Mon Mar 25 21:40:33 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 25 Mar 2013 16:40:33 -0400
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
Message-ID: <CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>

2013/3/25 Todd Rovito <rovitotv at gmail.com>:
> Python 2.7.4 release candidate was just announced and is ready for testing here:
> http://python.org/download/releases/2.7.4/
>
> Now the clock is ticking and we have two weeks to get IDLE issues
> pushed into CPython before the final release of 2.7.4.

I'm afraid the ship has sailed on that. After a bugfix release
candidate, I only take patches that fix regressions from earlier
releases of the same series. (The goal is to not make any changes
between the release candidate and final release.) Assuming PEP 343
becomes policy, IDLE changes can land for 2.7.5, which will be in
approximately 6 months.


-- 
Regards,
Benjamin

From victor.stinner at gmail.com  Mon Mar 25 22:16:19 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Mon, 25 Mar 2013 22:16:19 +0100
Subject: [Python-Dev] Can we triple quoted string as a comment?
Message-ID: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>

Hi,

I just realized that the Python peephole optimizer removes useless
instructions like numbers and strings between other instructions,
without raising an error nor emiting an error. Example:

$ python -Wd -c 'print "Hello"; "World"'
Hello

As part of my astoptimizer project, I wrote a function to detect such
useless instructions which emit a warning. I opened the following
issue to report what I found:
http://bugs.python.org/issue17516

Different modules use long strings as comments. What is the "official"
policy about such strings? Should we use strings or comments?

(IMO a comment should be used instead.)

Victor

From greg at krypto.org  Mon Mar 25 22:22:39 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Mon, 25 Mar 2013 14:22:39 -0700
Subject: [Python-Dev] Can we triple quoted string as a comment?
In-Reply-To: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
Message-ID: <CAGE7PNLcajDkO8=sjs1T1QuauR=ztiQM_zVn6ddN0bh3R5AnAw@mail.gmail.com>

On Mon, Mar 25, 2013 at 2:16 PM, Victor Stinner <victor.stinner at gmail.com>wrote:

> Hi,
>
> I just realized that the Python peephole optimizer removes useless
> instructions like numbers and strings between other instructions,
> without raising an error nor emiting an error. Example:
>
> $ python -Wd -c 'print "Hello"; "World"'
> Hello
>
> As part of my astoptimizer project, I wrote a function to detect such
> useless instructions which emit a warning. I opened the following
> issue to report what I found:
> http://bugs.python.org/issue17516
>
> Different modules use long strings as comments. What is the "official"
> policy about such strings? Should we use strings or comments?
>

Comments.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130325/abf55dc9/attachment.html>

From nad at acm.org  Mon Mar 25 22:31:16 2013
From: nad at acm.org (Ned Deily)
Date: Mon, 25 Mar 2013 14:31:16 -0700
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
	release in two weeks on 4/6/2013
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
Message-ID: <nad-FEC915.14311625032013@news.gmane.org>

In article 
<CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg at mail.gmail.com>,
 Todd Rovito <rovitotv at gmail.com> wrote:
> Now the clock is ticking and we have two weeks to get IDLE issues
> pushed into CPython before the final release of 2.7.4.  Below is my
> list of low risk issues that would be great to get pushed into
> CPython.  I hope this email will encourage CPython Core Developers to
> make the commits or tell us what we can do, like more testing or
> better documentation, to get these issues cleaned up and committed
> before 2.7.4 ships.

Todd,

Thanks for your suggestions and work to improve IDLE and Python, in 
general.  There has been a lot of discussions recently, including your 
PEP proposal.  I have not commented on the discussions yet, though I 
plan to in the next few days.  Unfortunately, it comes at one of the 
busiest times for those of us working on releases.  Not only is the 
2.7.4 release in preparation but 3.2.4rc1 and 3.3.0.rc1 are about to be 
announced, all on similar schedules.  So just a few comments from my 
release team perspective.  No doubt, others may have other opinions.

Bugfix releases, like these three, follow an abbreviated version of the 
full feature release process, outlined in the developer's guide:

   http://docs.python.org/devguide/devcycle.html#stages

For bugfix releases, we typically skip alphas and betas and go right to 
the release candidate stage, under the assumption that the criteria used 
for commits added to bugfix releases are designed to avoid incompatible 
changes and new features, unless explicitly approved for good reasons.  
That means that a release candidate is meant to be just that: the final 
set of bits that will be released.   All of us involved in software 
development have our own war stories of how some little last-minute 
change caused some unexpected breakage.  So the normal expectation is 
that, if any change is accepted and cherry-picked  after a release 
candidate has been published, a new RC cycle will need to happen unless 
the release manager decides the change is trivial enough that the risk 
is truly minimal, e.g. something like an obvious typo or a doc change.   
Certainly the changes proposed here would not normally fit those 
criteria.

Also, before the changes could be considered to be cherry-picked for a 
release, they need to be applied to the branches first and given some 
amount of testing, preferably on all of the major platforms where we 
support IDLE: X11, Windows, OS X.  So that's what needs to happen next.  
There are various developers who have been applying IDLE fixes and now 
Roger is able to do so as well (yay!).  Once they are in, then the 
question of release becomes relevant.  There are a couple of possible 
scenarios I can see.   1. It's possible that problems will show up in 
one or more of the current RCs, necessitating another RC cycle, at which 
time the release manager(s) *might* be amenable to cherrypicking one or 
more fixes from the current branches.  2.  It's also possible (probable, 
I hope) that 2.7.5 and/or 3.3.2 will follow relatively quickly after 
2.7.4 and 3.3.1.  (3.2.4 is expected to be the final 3.2.x bugfix 
release before it enters security-only fix mode.)  The period since the 
last maint releases for 2.7.x and 3.2.x was unusually long, for various 
good reasons, so there are about a year's worth of fixes going out for 
them this time, thereby raising the likelihood that new problems will be 
found requiring a fix in a new bugfix release.  Plus there are some 
security issues that need a final resolution in a release.  So, I'm 
hopeful that we won't have to wait nearly as long to see 2.7.5 after 
2.7.4.  There's not as long a gap since 3.3.0 but still somewhat long 
for a first bugfix release.

BTW, there is a fair amount of activity that goes on somewhat behind the 
scenes with getting releases out-the-door.  There a number of release 
artifacts that need to be produced and tested, webpages that need to be 
updated, announcements sent, etc.  For example, for OS X, we normally 
release two variants of installers for each beta, rc, and final release.  
Between the two variants, we support 13 different 
architecture/os-release combinations (only 11 for 3.3.x).  That means, 
at the moment, we have 37 different combinations we could test 
(including those for 2.7.4rc1, 3.2.4rc1, and 3.3.1rc1).  I don't 
personally test every one of them but I do run the Python tests on a 
representative sample (various OS levels vs. ppc/Intel-32/Intel-64) of 
configurations, including at least very minimal manual tests of IDLE to 
cover things like different versions of Tcl/Tk we support on OS X and 
current fixes like the recent infamous preferences panel crash.  Then 
there's the time required to investigate and writeup test failures, and 
decide on a fix strategy (e.g. is it a release blocker?).  Similar 
things happen for the WIndows installers and for the main source 
packages.  All of these things are part of what goes in to having a good 
batteries-included experience for our users, including IDLE users.  We 
all work on Python releases because we want to do it and we like to do 
it.  That doesn't mean the process is cast-in-concrete or can't be 
improved.  But there is a cost involved in producing releases that isn't 
always readily apparent.

Specially for IDLE, I think there are a number of things we can do to 
improve and speed its development process.  As noted, I'll have some 
concrete suggestions soon but, right now, I should get back to the 
release candidates at hand.  In the meantime, let's see what we can do 
to get those patches checked in, documented, and tested.

-- 
 Ned Deily,
 nad at acm.org


From glyph at twistedmatrix.com  Mon Mar 25 23:57:53 2013
From: glyph at twistedmatrix.com (Glyph)
Date: Mon, 25 Mar 2013 15:57:53 -0700
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
	release in two weeks on 4/6/2013
In-Reply-To: <CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
	<CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>
Message-ID: <4D28596D-C465-47E7-B568-CD90B1ADFA5D@twistedmatrix.com>


On Mar 25, 2013, at 1:40 PM, Benjamin Peterson <benjamin at python.org> wrote:

> ... Assuming PEP 343 becomes policy ...

Are you sure you got this PEP number right?  The 'with' statement?

<http://www.python.org/dev/peps/pep-0343/>

-glyph
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130325/451ad8a0/attachment.html>

From benjamin at python.org  Tue Mar 26 00:04:51 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Mon, 25 Mar 2013 19:04:51 -0400
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <4D28596D-C465-47E7-B568-CD90B1ADFA5D@twistedmatrix.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
	<CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>
	<4D28596D-C465-47E7-B568-CD90B1ADFA5D@twistedmatrix.com>
Message-ID: <CAPZV6o9Gk4CBn_OV2c_H8EFQFasiFCoBjTJtdE6sYPU80QXpPg@mail.gmail.com>

2013/3/25 Glyph <glyph at twistedmatrix.com>:
>
> On Mar 25, 2013, at 1:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
>
> ... Assuming PEP 343 becomes policy ...
>
>
> Are you sure you got this PEP number right?  The 'with' statement?

Sorry, I meant PEP 434.



-- 
Regards,
Benjamin

From fijall at gmail.com  Tue Mar 26 01:17:17 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Mon, 25 Mar 2013 17:17:17 -0700
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <4D28596D-C465-47E7-B568-CD90B1ADFA5D@twistedmatrix.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
	<CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>
	<4D28596D-C465-47E7-B568-CD90B1ADFA5D@twistedmatrix.com>
Message-ID: <CAK5idxRKC8UaPZRPkm-tdfEaekPW8Mjo8fX9=GGMon+s7N2bzA@mail.gmail.com>

On Mon, Mar 25, 2013 at 3:57 PM, Glyph <glyph at twistedmatrix.com> wrote:
>
> On Mar 25, 2013, at 1:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
>
> ... Assuming PEP 343 becomes policy ...
>
>
> Are you sure you got this PEP number right?  The 'with' statement?
>
> <http://www.python.org/dev/peps/pep-0343/>
>

Don't worry even google confuses the two :)

From rovitotv at gmail.com  Tue Mar 26 01:54:41 2013
From: rovitotv at gmail.com (Todd Rovito)
Date: Mon, 25 Mar 2013 20:54:41 -0400
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
	<CAPZV6o_7sFPPQPyPc9Xdy3HKKgo81+fFB059_6N7MgifdWrp3g@mail.gmail.com>
Message-ID: <CALZ1apwbijE4pyfxue85JXbv89sRcHgwwYuPEkg1_HkvTcegAg@mail.gmail.com>

On Mon, Mar 25, 2013 at 4:40 PM, Benjamin Peterson <benjamin at python.org> wrote:
> I'm afraid the ship has sailed on that. After a bugfix release
> candidate, I only take patches that fix regressions from earlier
> releases of the same series. (The goal is to not make any changes
> between the release candidate and final release.) Assuming PEP 343
> becomes policy, IDLE changes can land for 2.7.5, which will be in
> approximately 6 months.
Benjamin,
   No problem we can wait 6 months.  I should of read the dev guide
section about Python releases before I sent the email.

From raymond.hettinger at gmail.com  Tue Mar 26 02:16:47 2013
From: raymond.hettinger at gmail.com (Raymond Hettinger)
Date: Mon, 25 Mar 2013 18:16:47 -0700
Subject: [Python-Dev] Can we triple quoted string as a comment?
In-Reply-To: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
Message-ID: <7B40444C-FA79-4776-A356-7FC76A5CADC7@gmail.com>


On Mar 25, 2013, at 2:16 PM, Victor Stinner <victor.stinner at gmail.com> wrote:

> Hi,
> 
> I just realized that the Python peephole optimizer removes useless
> instructions like numbers and strings between other instructions,
> without raising an error nor emiting an error. Example:
> 
> $ python -Wd -c 'print "Hello"; "World"'
> Hello

IIRC, this happens upstream from the peephole optimizer
and has been a part of Python for a long time.

You can also "comment-out" code with "if 0:"

>>> def f(x):
	if 0:
		print x
	return x+1

>>> from dis import dis
>>> dis(f)
  4           0 LOAD_FAST                0 (x)
              3 LOAD_CONST               1 (1)
              6 BINARY_ADD          
              7 RETURN_VALUE 

> 
> As part of my astoptimizer project, I wrote a function to detect such
> useless instructions which emit a warning. I opened the following
> issue to report what I found:
> http://bugs.python.org/issue17516

Make sure it is a warning you can turn-off.

I've seen code in many organizations that use multi-line
strings to "turn-off" a section of code but not actually remove the code
from the source.


> 
> Different modules use long strings as comments. What is the "official"
> policy about such strings? Should we use strings or comments?
> 
> (IMO a comment should be used instead.)

The module authors typically make their own decisions
with respect to readability and ease of commenting.

If you're editing with Emacs, it is really easy to reflow paragraphs
and to insert or remove multiline comments each prefixed with #.
But with other editors, it can be a PITA and a multiline string is
the easiest to maintain and works well when cutting-and-pasting
the comments from somewhere else.

I worry that because you just discovered this feature, the initial
reaction is that is a horribly wrong thing to do and should be
"fixed" everywhere.

Instead, it would be better to live-and-let live.
No need for wholesale code changes or imposition
"you must do it the way I do it" policies.


my-two-cents,


Raymond
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130325/7baca9e5/attachment-0001.html>

From rovitotv at gmail.com  Tue Mar 26 03:18:41 2013
From: rovitotv at gmail.com (Todd Rovito)
Date: Mon, 25 Mar 2013 22:18:41 -0400
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <nad-FEC915.14311625032013@news.gmane.org>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
	<nad-FEC915.14311625032013@news.gmane.org>
Message-ID: <CALZ1apzXrR36xO+QNEf6d8kvh-fTLdUzXrkey4=-L6fFb7dr4g@mail.gmail.com>

On Mon, Mar 25, 2013 at 5:31 PM, Ned Deily <nad at acm.org> wrote:
> Todd,
>
> Thanks for your suggestions and work to improve IDLE and Python, in
> general.  There has been a lot of discussions recently, including your
> PEP proposal.  I have not commented on the discussions yet, though I
> plan to in the next few days.
No problem I enjoy working on Python!!!!  I look forward to reading
your comments.

> Unfortunately, it comes at one of the
> busiest times for those of us working on releases.  Not only is the
> 2.7.4 release in preparation but 3.2.4rc1 and 3.3.0.rc1 are about to be
> announced, all on similar schedules.  So just a few comments from my
> release team perspective.  No doubt, others may have other opinions.
>
> Bugfix releases, like these three, follow an abbreviated version of the
> full feature release process, outlined in the developer's guide:
>
>    http://docs.python.org/devguide/devcycle.html#stages
Thanks for the reference I will read it again this time more
carefully.  Before I sent the email I should have reviewed it so
perhaps I jumped the gun a little.

> For bugfix releases, we typically skip alphas and betas and go right to
> the release candidate stage, under the assumption that the criteria used
> for commits added to bugfix releases are designed to avoid incompatible
> changes and new features, unless explicitly approved for good reasons.
> That means that a release candidate is meant to be just that: the final
> set of bits that will be released.   All of us involved in software
> development have our own war stories of how some little last-minute
> change caused some unexpected breakage.  So the normal expectation is
> that, if any change is accepted and cherry-picked  after a release
> candidate has been published, a new RC cycle will need to happen unless
> the release manager decides the change is trivial enough that the risk
> is truly minimal, e.g. something like an obvious typo or a doc change.
> Certainly the changes proposed here would not normally fit those
> criteria.
Thanks for taking the time to explain that to me....for sure I don't
mean to rush the process.  I do know that the Python team releases a
high quality product and why change a good thing.

> Also, before the changes could be considered to be cherry-picked for a
> release, they need to be applied to the branches first and given some
> amount of testing, preferably on all of the major platforms where we
> support IDLE: X11, Windows, OS X.  So that's what needs to happen next.
I agree 100% so we just need to get these simple issues committed and
done.  It seems like now is a bad time so I can wait patiently, I
don't want to be a pest but it drives me nuts that some of these
issues have been fixed and they go uncommitted.  More than anything I
want the Python Core Developers to know that work has happened on IDLE
and many times the patches simply don't get committed.  I admit there
are various reasons some of these IDLE issues don't get resolved but
these first four are a slam dunk.  Please Python Core Developers don't
take offense I know everybody is busy moving the language forward
which is super important, I want to gently tickle your belly buttons
as a reminder only.

> There are various developers who have been applying IDLE fixes and now
> Roger is able to do so as well (yay!).  Once they are in, then the
> question of release becomes relevant.
Giving Roger commit rights will speed things up no doubt about that
but he is learning the ropes and I was hoping to get these four simple
issues committed and done with before 2.7.4 comes out.  Knowing now
that we have 6 months (instead of two weeks) relaxes me a little.

> There are a couple of possible
> scenarios I can see.   1. It's possible that problems will show up in
> one or more of the current RCs, necessitating another RC cycle, at which
> time the release manager(s) *might* be amenable to cherrypicking one or
> more fixes from the current branches.  2.  It's also possible (probable,
> I hope) that 2.7.5 and/or 3.3.2 will follow relatively quickly after
> 2.7.4 and 3.3.1.  (3.2.4 is expected to be the final 3.2.x bugfix
> release before it enters security-only fix mode.)  The period since the
> last maint releases for 2.7.x and 3.2.x was unusually long, for various
> good reasons, so there are about a year's worth of fixes going out for
> them this time, thereby raising the likelihood that new problems will be
> found requiring a fix in a new bugfix release.  Plus there are some
> security issues that need a final resolution in a release.  So, I'm
> hopeful that we won't have to wait nearly as long to see 2.7.5 after
> 2.7.4.  There's not as long a gap since 3.3.0 but still somewhat long
> for a first bugfix release.
All this is good information and makes me feel even better about
waiting a few months to get these four issues and possibly more IDLE
issues fixed before the next release.


> BTW, there is a fair amount of activity that goes on somewhat behind the
> scenes with getting releases out-the-door.  There a number of release
> artifacts that need to be produced and tested, webpages that need to be
> updated, announcements sent, etc.  For example, for OS X, we normally
> release two variants of installers for each beta, rc, and final release.
> Between the two variants, we support 13 different
> architecture/os-release combinations (only 11 for 3.3.x).  That means,
> at the moment, we have 37 different combinations we could test
> (including those for 2.7.4rc1, 3.2.4rc1, and 3.3.1rc1).  I don't
> personally test every one of them but I do run the Python tests on a
> representative sample (various OS levels vs. ppc/Intel-32/Intel-64) of
> configurations, including at least very minimal manual tests of IDLE to
> cover things like different versions of Tcl/Tk we support on OS X and
> current fixes like the recent infamous preferences panel crash.  Then
> there's the time required to investigate and writeup test failures, and
> decide on a fix strategy (e.g. is it a release blocker?).  Similar
> things happen for the WIndows installers and for the main source
> packages.  All of these things are part of what goes in to having a good
> batteries-included experience for our users, including IDLE users.  We
> all work on Python releases because we want to do it and we like to do
> it.  That doesn't mean the process is cast-in-concrete or can't be
> improved.  But there is a cost involved in producing releases that isn't
> always readily apparent.
Again I really appreciate the information now I have a much better
idea what goes into a release.

> Specially for IDLE, I think there are a number of things we can do to
> improve and speed its development process.  As noted, I'll have some
> concrete suggestions soon but, right now, I should get back to the
> release candidates at hand.  In the meantime, let's see what we can do
> to get those patches checked in, documented, and tested.
It would be great to hear those ideas when you get time.

Several people and I have been saying IDLE development is not dead
these four patches represent some fairly trivial yet important fixes.
I am willing to do almost anything to improve the state of IDLE
(bribery is not out of the question).  Just know that I am more
persistent than most people and I will not go away or give up, if the
issues I outlined have problems please indicate that with the issue
and we will get them fixed.  As people have stated on this list over
the last few days far better than I, Python users do care about IDLE.
 It would be great if we could all work together and make sure these
issues listed are all committed way before the next major release
cycle.  Thanks for your time and effort!

From stephen at xemacs.org  Tue Mar 26 04:00:25 2013
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 26 Mar 2013 12:00:25 +0900
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <CALZ1apzXrR36xO+QNEf6d8kvh-fTLdUzXrkey4=-L6fFb7dr4g@mail.gmail.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
	<nad-FEC915.14311625032013@news.gmane.org>
	<CALZ1apzXrR36xO+QNEf6d8kvh-fTLdUzXrkey4=-L6fFb7dr4g@mail.gmail.com>
Message-ID: <87d2umdfw6.fsf@uwakimon.sk.tsukuba.ac.jp>

Todd Rovito writes:

 > All this is good information and makes me feel even better about
 > waiting a few months to get these four issues and possibly more IDLE
 > issues fixed before the next release.

+1 for more issues fixed in the next release! :-)


From tjreedy at udel.edu  Tue Mar 26 07:01:58 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 26 Mar 2013 02:01:58 -0400
Subject: [Python-Dev] IDLE in the stdlib
In-Reply-To: <514CA836.8040009@email.de>
References: <CAF-Rda_U3amLSHeWCKLquhNrgjhVbfrg8ngec0M-0UkXzbWxhg@mail.gmail.com>
	<kidv0r$tcv$1@ger.gmane.org>
	<CAF-Rda8n-JyFMwja2AAu9rZO-ypmhYgauvzcD=jboWnkD=7ZeA@mail.gmail.com>
	<kiea4v$iar$1@ger.gmane.org>
	<CABicbJ+uRE+u8trzu8NsFroHxSnStMum-P7-yV3_dDjekhTK-g@mail.gmail.com>
	<CACac1F83g0cKT9ON+1rRQ0UXrTQosRu9V-zU+jyFOOtyjGsqcQ@mail.gmail.com>
	<kiepea$v2$1@ger.gmane.org> <514CA836.8040009@email.de>
Message-ID: <kirdkk$f0f$1@ger.gmane.org>

On 3/22/2013 2:51 PM, francis wrote:

> ~$ python2.7 -m idlelib
> /usr/bin/python2.7: No module named idlelib.__main__; 'idlelib' is a
> package and cannot be directly executed
>
> Same with python3...

C:\Programs>python33\python.exe -m idlelib

brings up IDLE on Windows. 2.7 and 3.2 do not work as above but require 
'idlelib.idle' instead of just 'idlelib'.

C:\Programs>python32\python.exe -m idlelib.idle

C:\Programs>python27\python.exe -m idlelib.idle

I have no idea if the change is to '-m' processing or to idlelib. If the 
latter, it is an example of a patch that might have been harmlessly 
backported with PEP434 accepted.

-- 
Terry Jan Reedy


From tjreedy at udel.edu  Tue Mar 26 07:40:22 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue, 26 Mar 2013 02:40:22 -0400
Subject: [Python-Dev] Simple IDLE issues to commit before Python 2.7.4
 release in two weeks on 4/6/2013
In-Reply-To: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
References: <CALZ1apxb0nbcHvGf4_5Hip1bze-xuSPFp_X1GFhJQ8-frqv+jg@mail.gmail.com>
Message-ID: <kirfsj$qd$1@ger.gmane.org>

On 3/25/2013 3:30 PM, Todd Rovito wrote:

> http://bugs.python.org/issue7136 Idle File Menu Option Improvement
> http://bugs.python.org/issue17390 display python version on idle title bar
> http://bugs.python.org/issue17511 Idle find function closes after each
> find operation
> http://bugs.python.org/issue6699 IDLE: Warn user about overwriting a
> file that has a newer version on filesystem
> http://bugs.python.org/issue10747 Include version info in Windows
> shortcuts the issue is for 3.x but perhaps simple to implement for
> 2.7?

> http://bugs.python.org/issue2704 IDLE: Patch to make PyShell behave
> more like a Terminal interface

I am waiting for 434 or an alternative to be accepted before pushing any 
IDLE patch. I have no interest in bikeshedding a somewhat meaningless, 
for IDLE, distinction between behavior and enhancement, nor in being 
asked to revert a patch that I tested on multiple versions.

However, I might use your list to select patches for testing on Windows.

-- 
Terry Jan Reedy


From georg at python.org  Tue Mar 26 07:47:38 2013
From: georg at python.org (Georg Brandl)
Date: Tue, 26 Mar 2013 07:47:38 +0100
Subject: [Python-Dev] [RELEASED] Python 3.2.4 rc 1 and Python 3.3.1 rc 1
Message-ID: <5151448A.2010103@python.org>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On behalf of the Python development team, I am pleased to announce the
first release candidates of Python 3.2.4 and 3.3.1.

Python 3.2.4 will be the last regular maintenance release for the Python 3.2
series, while Python 3.3.1 is the first maintenance release for the 3.3
series.  Both releases include hundreds of bugfixes.

There has recently been a lot of discussion about XML-based denial of service
attacks. Specifically, certain XML files can cause XML parsers, including ones
in the Python stdlib, to consume gigabytes of RAM and swamp the CPU. These
releases do not include any changes in Python XML code to address these issues.
Interested parties should examine the defusedxml package on PyPI:
https://pypi.python.org/pypi/defusedxml

These are testing releases: Please consider trying them with your code
and reporting any bugs you may notice to:

    http://bugs.python.org/

To download Python 3.2.4 or Python 3.3.1, visit:

    http://www.python.org/download/releases/3.2.4/  or
    http://www.python.org/download/releases/3.3.1/

respectively.


Enjoy!

- -- 
Georg Brandl, Release Manager
georg at python.org
(on behalf of the entire python-dev team and all contributors)
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.19 (GNU/Linux)

iEYEARECAAYFAlFRRIoACgkQN9GcIYhpnLD6jACgnzYdYRKZ4kwkKeN3zSLSZ3Zr
M/IAn17vlpxI3a3xk+i/ODOrCkMnRZro
=B5sA
-----END PGP SIGNATURE-----

From doko at ubuntu.com  Tue Mar 26 11:54:26 2013
From: doko at ubuntu.com (Matthias Klose)
Date: Tue, 26 Mar 2013 11:54:26 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
Message-ID: <51517E62.2080301@ubuntu.com>

Am 25.03.2013 01:30, schrieb Benjamin Peterson:
> 2.7.4 will be the latest maintenance release in the Python 2.7 series.

I hope it's not (and in the IDLE thread you say so otherwise too).

  Matthias



From regebro at gmail.com  Tue Mar 26 12:05:05 2013
From: regebro at gmail.com (Lennart Regebro)
Date: Tue, 26 Mar 2013 12:05:05 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <51517E62.2080301@ubuntu.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
Message-ID: <CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>

On Tue, Mar 26, 2013 at 11:54 AM, Matthias Klose <doko at ubuntu.com> wrote:
> Am 25.03.2013 01:30, schrieb Benjamin Peterson:
>> 2.7.4 will be the latest maintenance release in the Python 2.7 series.
>
> I hope it's not (and in the IDLE thread you say so otherwise too).

It most certainly will be the latest once it's released. But hopefully
not the last. :-)

//Lennart

From victor.stinner at gmail.com  Tue Mar 26 12:34:34 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 26 Mar 2013 12:34:34 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
Message-ID: <CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>

2013/3/26 Lennart Regebro <regebro at gmail.com>:
> On Tue, Mar 26, 2013 at 11:54 AM, Matthias Klose <doko at ubuntu.com> wrote:
>> Am 25.03.2013 01:30, schrieb Benjamin Peterson:
>>> 2.7.4 will be the latest maintenance release in the Python 2.7 series.
>>
>> I hope it's not (and in the IDLE thread you say so otherwise too).
>
> It most certainly will be the latest once it's released. But hopefully
> not the last. :-)

I also read "the last" by mistake!

Anyway, you should trust Brett Canon: "Python 3.3: Trust Me, It's
Better Than Python 2.7".

https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-than-python-2-dot-7-by-dr-brett-cannon

Victor

From a.cavallo at cavallinux.eu  Tue Mar 26 12:54:11 2013
From: a.cavallo at cavallinux.eu (a.cavallo at cavallinux.eu)
Date: Tue, 26 Mar 2013 12:54:11 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
Message-ID: <365d993ec4df283ee38b27cd82ac6872@cavallinux.eu>

It's already hard to sell 2.7 in most companies.

Regards,
Antonio

> Anyway, you should trust Brett Canon: "Python 3.3: Trust Me, It's
> Better Than Python 2.7".
>
> 
> https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-than-python-2-dot-7-by-dr-brett-cannon
>
> Victor


From benjamin at python.org  Tue Mar 26 13:13:31 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 26 Mar 2013 08:13:31 -0400
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <51517E62.2080301@ubuntu.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
Message-ID: <CAPZV6o-YsP9nejDh2qx8hyJOvS2wDwJojGZ5cKSX3nYVgN4K_Q@mail.gmail.com>

2013/3/26 Matthias Klose <doko at ubuntu.com>:
> Am 25.03.2013 01:30, schrieb Benjamin Peterson:
>> 2.7.4 will be the latest maintenance release in the Python 2.7 series.
>
> I hope it's not (and in the IDLE thread you say so otherwise too).

"latest" is different from "last" :)


-- 
Regards,
Benjamin

From rdmurray at bitdance.com  Tue Mar 26 14:28:51 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 26 Mar 2013 09:28:51 -0400
Subject: [Python-Dev] Can we triple quoted string as a comment?
In-Reply-To: <7B40444C-FA79-4776-A356-7FC76A5CADC7@gmail.com>
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
	<7B40444C-FA79-4776-A356-7FC76A5CADC7@gmail.com>
Message-ID: <20130326132851.8D4AB250069@webabinitio.net>

On Mon, 25 Mar 2013 18:16:47 -0700, Raymond Hettinger <raymond.hettinger at gmail.com> wrote:
> If you're editing with Emacs, it is really easy to reflow paragraphs
> and to insert or remove multiline comments each prefixed with #.
> But with other editors, it can be a PITA and a multiline string is
> the easiest to maintain and works well when cutting-and-pasting
> the comments from somewhere else.

Just FYI it is also very easy in vim: gq plus whatever movement prefix
suits the situation.

--David

From solipsis at pitrou.net  Tue Mar 26 14:54:24 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 26 Mar 2013 14:54:24 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
Message-ID: <20130326145424.33c9326b@pitrou.net>

Le Tue, 26 Mar 2013 12:34:34 +0100,
Victor Stinner <victor.stinner at gmail.com> a ?crit :

> 2013/3/26 Lennart Regebro <regebro at gmail.com>:
> > On Tue, Mar 26, 2013 at 11:54 AM, Matthias Klose <doko at ubuntu.com>
> > wrote:
> >> Am 25.03.2013 01:30, schrieb Benjamin Peterson:
> >>> 2.7.4 will be the latest maintenance release in the Python 2.7
> >>> series.
> >>
> >> I hope it's not (and in the IDLE thread you say so otherwise too).
> >
> > It most certainly will be the latest once it's released. But
> > hopefully not the last. :-)
> 
> I also read "the last" by mistake!
> 
> Anyway, you should trust Brett Canon: "Python 3.3: Trust Me, It's
> Better Than Python 2.7".
> 
> https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-than-python-2-dot-7-by-dr-brett-cannon

You can always trust Brett Cannon!

cheers

Antoine.



From solipsis at pitrou.net  Tue Mar 26 14:55:05 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 26 Mar 2013 14:55:05 +0100
Subject: [Python-Dev] Can we triple quoted string as a comment?
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
	<7B40444C-FA79-4776-A356-7FC76A5CADC7@gmail.com>
	<20130326132851.8D4AB250069@webabinitio.net>
Message-ID: <20130326145505.393ae8e5@pitrou.net>

Le Tue, 26 Mar 2013 09:28:51 -0400,
"R. David Murray" <rdmurray at bitdance.com> a ?crit :
> On Mon, 25 Mar 2013 18:16:47 -0700, Raymond Hettinger
> <raymond.hettinger at gmail.com> wrote:
> > If you're editing with Emacs, it is really easy to reflow paragraphs
> > and to insert or remove multiline comments each prefixed with #.
> > But with other editors, it can be a PITA and a multiline string is
> > the easiest to maintain and works well when cutting-and-pasting
> > the comments from somewhere else.
> 
> Just FYI it is also very easy in vim: gq plus whatever movement prefix
> suits the situation.

And on a user-friendly editor such as Kate, you can press Ctrl+D.

Regards

Antoine.



From ethan at stoneleaf.us  Tue Mar 26 16:46:23 2013
From: ethan at stoneleaf.us (Ethan Furman)
Date: Tue, 26 Mar 2013 08:46:23 -0700
Subject: [Python-Dev] Can we triple quoted string as a comment?
In-Reply-To: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
Message-ID: <5151C2CF.8070809@stoneleaf.us>

On 03/25/2013 02:16 PM, Victor Stinner wrote:
> Hi,
>
> I just realized that the Python peephole optimizer removes useless
> instructions like numbers and strings between other instructions,
> without raising an error nor emiting an error. Example:
>
> $ python -Wd -c 'print "Hello"; "World"'
> Hello
>
> As part of my astoptimizer project, I wrote a function to detect such
> useless instructions which emit a warning. I opened the following
> issue to report what I found:
> http://bugs.python.org/issue17516
>
> Different modules use long strings as comments. What is the "official"
> policy about such strings? Should we use strings or comments?
>
> (IMO a comment should be used instead.)

Someone will correct me if I'm wrong, I'm sure, but I believe Guido himself has said that a neat feature of 
triple-quoted strings is their ability to be used as comments.

--
~Ethan~

From guido at python.org  Tue Mar 26 17:11:12 2013
From: guido at python.org (Guido van Rossum)
Date: Tue, 26 Mar 2013 09:11:12 -0700
Subject: [Python-Dev] Can we triple quoted string as a comment?
In-Reply-To: <5151C2CF.8070809@stoneleaf.us>
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
	<5151C2CF.8070809@stoneleaf.us>
Message-ID: <CAP7+vJ+sHbJM==J9E+iY7TsQN3Shy6ztjTjn1eTN1Rg0s8TqOA@mail.gmail.com>

And I still think it's neat. :-)


On Tue, Mar 26, 2013 at 8:46 AM, Ethan Furman <ethan at stoneleaf.us> wrote:

> On 03/25/2013 02:16 PM, Victor Stinner wrote:
>
>> Hi,
>>
>> I just realized that the Python peephole optimizer removes useless
>> instructions like numbers and strings between other instructions,
>> without raising an error nor emiting an error. Example:
>>
>> $ python -Wd -c 'print "Hello"; "World"'
>> Hello
>>
>> As part of my astoptimizer project, I wrote a function to detect such
>> useless instructions which emit a warning. I opened the following
>> issue to report what I found:
>> http://bugs.python.org/**issue17516 <http://bugs.python.org/issue17516>
>>
>> Different modules use long strings as comments. What is the "official"
>> policy about such strings? Should we use strings or comments?
>>
>> (IMO a comment should be used instead.)
>>
>
> Someone will correct me if I'm wrong, I'm sure, but I believe Guido
> himself has said that a neat feature of triple-quoted strings is their
> ability to be used as comments.
>
> --
> ~Ethan~
>
> ______________________________**_________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/**mailman/listinfo/python-dev<http://mail.python.org/mailman/listinfo/python-dev>
> Unsubscribe: http://mail.python.org/**mailman/options/python-dev/**
> guido%40python.org<http://mail.python.org/mailman/options/python-dev/guido%40python.org>
>



-- 
--Guido van Rossum (python.org/~guido)
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130326/824dac1f/attachment.html>

From solipsis at pitrou.net  Tue Mar 26 19:41:46 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 26 Mar 2013 19:41:46 +0100
Subject: [Python-Dev] cpython (2.7): Issue 17538: Document XML
	vulnerabilties
References: <3ZZz1v2N95zQBT@mail.python.org>
Message-ID: <20130326194146.68662198@pitrou.net>

On Tue, 26 Mar 2013 17:53:39 +0100 (CET)
christian.heimes <python-checkins at python.org> wrote:
> +
> +The XML processing modules are not secure against maliciously constructed data.
> +An attacker can abuse vulnerabilities for e.g. denial of service attacks, to
> +access local files, to generate network connections to other machines, or
> +to or circumvent firewalls.

Really? Where does the "access local files, generate network
connections, circumvent firewalls" allegation come from?

Regards

Antoine.



From donald at stufft.io  Tue Mar 26 19:57:37 2013
From: donald at stufft.io (Donald Stufft)
Date: Tue, 26 Mar 2013 14:57:37 -0400
Subject: [Python-Dev] cpython (2.7): Issue 17538: Document XML
	vulnerabilties
In-Reply-To: <20130326194146.68662198@pitrou.net>
References: <3ZZz1v2N95zQBT@mail.python.org>
	<20130326194146.68662198@pitrou.net>
Message-ID: <B0FCE840-0273-434C-91AE-42823F06599B@stufft.io>


On Mar 26, 2013, at 2:41 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:

> On Tue, 26 Mar 2013 17:53:39 +0100 (CET)
> christian.heimes <python-checkins at python.org> wrote:
>> +
>> +The XML processing modules are not secure against maliciously constructed data.
>> +An attacker can abuse vulnerabilities for e.g. denial of service attacks, to
>> +access local files, to generate network connections to other machines, or
>> +to or circumvent firewalls.
> 
> Really? Where does the "access local files, generate network
> connections, circumvent firewalls" allegation come from?

https://pypi.python.org/pypi/defusedxml#external-entity-expansion-remote

https://pypi.python.org/pypi/defusedxml#external-entity-expansion-local-file

https://pypi.python.org/pypi/defusedxml#dtd-retrieval

> 
> Regards
> 
> Antoine.
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/donald%40stufft.io


-----------------
Donald Stufft
PGP: 0x6E3CBCE93372DCFA // 7C6B 7C5D 5E2B 6356 A926 F04F 6E3C BCE9 3372 DCFA

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 841 bytes
Desc: Message signed with OpenPGP using GPGMail
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130326/2823df91/attachment.pgp>

From christian at python.org  Tue Mar 26 20:04:11 2013
From: christian at python.org (Christian Heimes)
Date: Tue, 26 Mar 2013 20:04:11 +0100
Subject: [Python-Dev] cpython (2.7): Issue 17538: Document XML
	vulnerabilties
In-Reply-To: <20130326194146.68662198@pitrou.net>
References: <3ZZz1v2N95zQBT@mail.python.org>
	<20130326194146.68662198@pitrou.net>
Message-ID: <5151F12B.2070509@python.org>

Am 26.03.2013 19:41, schrieb Antoine Pitrou:
> On Tue, 26 Mar 2013 17:53:39 +0100 (CET)
> christian.heimes <python-checkins at python.org> wrote:
>> +
>> +The XML processing modules are not secure against maliciously constructed data.
>> +An attacker can abuse vulnerabilities for e.g. denial of service attacks, to
>> +access local files, to generate network connections to other machines, or
>> +to or circumvent firewalls.
> 
> Really? Where does the "access local files, generate network
> connections, circumvent firewalls" allegation come from?

Really!

https://bitbucket.org/PSF/defusedxml/src/e76248a8c2b3102c565bd3451656130cb29f04f8/other/python_external.py?at=default


REQUEST:
--------
<weather>Aachen</weather>

RESPONSE:
---------
<weather>The weather in Aachen is terrible.</weather


REQUEST:
--------
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE weather [
<!ENTITY passwd SYSTEM "file:///etc/passwd">
]>
<weather>&passwd;</weather>


RESPONSE:
---------
<error>Unknown city root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh
sync:x:4:65534:sync:/bin:/bin/sync
games:x:5:60:games:/usr/games:/bin/sh
man:x:6:12:man:/var/cache/man:/bin/sh
lp:x:7:7:lp:/var/spool/lpd:/bin/sh
mail:x:8:8:mail:/var/mail:/bin/sh
news:x:9:9:news:/var/spool/news:/bin/sh
uucp:x:10:10:uucp:/var/spool/uucp:/bin/sh
proxy:x:13:13:proxy:/bin:/bin/sh
www-data:x:33:33:www-data:/var/www:/bin/sh
backup:x:34:34:backup:/var/backups:/bi</error>


REQUEST:
--------
<?xml version="1.0" encoding="utf-8"?>
<!DOCTYPE weather [
<!ENTITY url SYSTEM
"http://hg.python.org/cpython/raw-file/a11ddd687a0b/Lib/test/dh512.pem">
]>
<weather>&url;</weather>


RESPONSE:
---------
<error>Unknown city -----BEGIN DH PARAMETERS-----
MEYCQQD1Kv884bEpQBgRjXyEpwpy1obEAxnIByl6ypUM2Zafq9AKUJsCRtMIPWak
XUGfnHy9iUsiGSa6q6Jew1XpKgVfAgEC
-----END DH PARAMETERS-----

These are the 512 bit DH parameters from "Assigned Number for SKIP
Protocols"
(http://www.skip-vpn.org/spec/numbers.html).
See there for how they were generated.
Note that g is not a generator, but this is not a problem since p is a
safe prime.
</error>


Q.E.D.
Christian

From solipsis at pitrou.net  Tue Mar 26 20:04:58 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Tue, 26 Mar 2013 20:04:58 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
	<365d993ec4df283ee38b27cd82ac6872@cavallinux.eu>
Message-ID: <20130326200458.67ce6780@pitrou.net>

On Tue, 26 Mar 2013 12:54:11 +0100
a.cavallo at cavallinux.eu wrote:
> It's already hard to sell 2.7 in most companies.

Sure, it's often hard to sell free software!

Regards

Antoine.



From brian at python.org  Tue Mar 26 20:51:29 2013
From: brian at python.org (Brian Curtin)
Date: Tue, 26 Mar 2013 14:51:29 -0500
Subject: [Python-Dev] Google Summer of Code - Organization Deadline
	Approaching - March 29
Message-ID: <CAD+XWwobKwo-A3=1gSvyuZCN-dig_cBrn7XM=QyAFG=WRZY2CQ@mail.gmail.com>

Just an FYI that there are under 3 days to apply to Google Summer of
Code for mentoring organizations:
http://www.google-melange.com/gsoc/homepage/google/gsoc2013. The
student application deadline is later on in May.

If you run a project that is interested in applying under the Python
umbrella organization, contact Terri Oda at terri at zone12.com

Is anyone here interested in leading CPython through GSOC? Anyone have
potential students to get involved, or interested in being a mentor?

From g.brandl at gmx.net  Tue Mar 26 21:19:42 2013
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 26 Mar 2013 21:19:42 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAPZV6o-YsP9nejDh2qx8hyJOvS2wDwJojGZ5cKSX3nYVgN4K_Q@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAPZV6o-YsP9nejDh2qx8hyJOvS2wDwJojGZ5cKSX3nYVgN4K_Q@mail.gmail.com>
Message-ID: <kisvs8$trg$1@ger.gmane.org>

Am 26.03.2013 13:13, schrieb Benjamin Peterson:
> 2013/3/26 Matthias Klose <doko at ubuntu.com>:
>> Am 25.03.2013 01:30, schrieb Benjamin Peterson:
>>> 2.7.4 will be the latest maintenance release in the Python 2.7 series.
>>
>> I hope it's not (and in the IDLE thread you say so otherwise too).
> 
> "latest" is different from "last" :)

As opposed to 3.2.4, which will really be the last (the last regular one,
that is -- if/when the XML stuff gets released there will be a source-only
release).

Georg


From victor.stinner at gmail.com  Tue Mar 26 22:56:53 2013
From: victor.stinner at gmail.com (Victor Stinner)
Date: Tue, 26 Mar 2013 22:56:53 +0100
Subject: [Python-Dev] astoptimizer: static optimizer working on the AST
Message-ID: <CAMpsgwaU8RHQDFibH5p5TxRd=eR7OT0cxy2y9xhEpHS_RDX_hw@mail.gmail.com>

Hi,

I made progress since last August on my astoptimizer project (read the
Changelog). Previous email thread:
http://mail.python.org/pipermail/python-dev/2012-August/121286.html

The astoptimizer project is an optimizer rewriting Python AST. It
executes as much code as possible during the compilation. The
optimizer itself is not designed to be fast, but to emit faster code.

https://bitbucket.org/haypo/astoptimizer/

Some optimizations are not "pythonic": don't respect the Python
language by making some assumptions (ex: on the namespace), which may
be wrong in some cases. astoptimizer is written for adults which know
which optimizations can be enabled, and which ones must be disabled in
their application (application, not module). I'm trying to write a
safe and "pythonic" default configuration. For example, len("abc") is
not replaced with 3 by default. You have to enable explicitly the
"builtin_funcs" configuration feature.

astoptimizer can be used as a pythonic preprocessor: it replaces
os.name, sys.platform and also your own constants by their value. It
removes dead code and so may be used to remove completly the overhead
of checks on the Python version or an the platform (if python3: ...
else: ...).

I would like to improve the integration of astoptimizer in Python 3.4.
Brett Canon asked me to use an hook in importlib, I proposed to add a
generic AST hook which would also be called by eval(), compile(), etc.
I would like to allow anyone to use its own AST modifier, and simplify
the usage of astoptimizer.
=> http://bugs.python.org/issue17515

Open issues:

 * what should be the name of "pyc" files?
 * how to handle different configuration of astoptimizer: generate
different "pyc" files?
 * no sys.getasthook(): it would permit to chain different AST hooks

--

There are many open issues proposing to write a better Python
optimizer. Some of them are stuck because implementing them using the
bytecode optimizer is hard.

 * http://bugs.python.org/issue1346238:
   A constant folding optimization pass for the AST
 * http://bugs.python.org/issue2181:
   optimize out local variables at end of function
 * http://bugs.python.org/issue2499:
   Fold unary + and not on constants
 * http://bugs.python.org/issue4264:
   Patch: optimize code to use LIST_APPEND instead of calling list.append
 * http://bugs.python.org/issue7682:
   Optimisation of if with constant expression
 * http://bugs.python.org/issue10399:
   AST Optimization: inlining of function calls
 * http://bugs.python.org/issue11549:
   Build-out an AST optimizer, moving some functionality out of the
peephole optimizer
 * http://bugs.python.org/issue17068:
   peephole optimization for constant strings
 * http://bugs.python.org/issue17430:
   missed peephole optimization

astoptimizer implements many optimizations listed in these issues.
Read the README file to the list of optimizations already implemented,
and the TODO file for ideas of new optimizations.

--

The project is still experimental. I ran Python 2.7 and 3.4 test
suites. Some tests are failing because the AST or the bytecode is
different (which is expected). More tests are failing if you enable
more agressive optimizations (especially if you enable the agressive
mode to remove dead code). At least, Python does not crash :-) (It
occurs sometimes when astoptimizer generates invalid AST!)

Victor

From ether.joe at gmail.com  Tue Mar 26 23:05:04 2013
From: ether.joe at gmail.com (Sean Felipe Wolfe)
Date: Tue, 26 Mar 2013 15:05:04 -0700
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
Message-ID: <CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>

On Tue, Mar 26, 2013 at 4:34 AM, Victor Stinner
<victor.stinner at gmail.com> wrote:
> Anyway, you should trust Brett Canon: "Python 3.3: Trust Me, It's
> Better Than Python 2.7".
>
> https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-than-python-2-dot-7-by-dr-brett-cannon

Was there supposed to be audio with this, or is it slides only? I got
no audio :P

From hs at ox.cx  Tue Mar 26 23:26:17 2013
From: hs at ox.cx (Hynek Schlawack)
Date: Tue, 26 Mar 2013 23:26:17 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>
Message-ID: <8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>

Am 26.03.2013 um 23:05 schrieb Sean Felipe Wolfe <ether.joe at gmail.com>:

>> Anyway, you should trust Brett Canon: "Python 3.3: Trust Me, It's
>> Better Than Python 2.7".
>> 
>> https://speakerdeck.com/pyconslides/python-3-dot-3-trust-me-its-better-than-python-2-dot-7-by-dr-brett-cannon
> Was there supposed to be audio with this, or is it slides only? I got
> no audio :P

Speakerdeck is slides only. The video is here: http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27


From ether.joe at gmail.com  Wed Mar 27 00:18:34 2013
From: ether.joe at gmail.com (Sean Felipe Wolfe)
Date: Tue, 26 Mar 2013 16:18:34 -0700
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>
Message-ID: <CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>

On Tue, Mar 26, 2013 at 3:26 PM, Hynek Schlawack <hs at ox.cx> wrote:
> Speakerdeck is slides only. The video is here: http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27

Sweet thanks!

From ether.joe at gmail.com  Wed Mar 27 00:49:32 2013
From: ether.joe at gmail.com (Sean Felipe Wolfe)
Date: Tue, 26 Mar 2013 16:49:32 -0700
Subject: [Python-Dev] noob contributions to unit tests
Message-ID: <CAOFf2a1Uf_hP-xHuHbkrXdEk6sYSiqTRiT3nbvo8LZk3AvJGJQ@mail.gmail.com>

Hey everybody how are you all :)

I am an intermediate-level python coder looking to get help out. I've
been reading over the dev guide about helping increase test coverage
-->
http://docs.python.org/devguide/coverage.html

And also the third-party code coverage referenced in the devguide page:
http://coverage.livinglogic.de/

I'm seeing that according to the coverage tool, two of my favorite
libraries, urllib/urllib2, have no unit tests? Is that correct or am I
reading it wrong?

If that's correct it seems like a great place perhaps for me to cut my
teeth and I would be excited to learn and help out here.

And of course any thoughts or advice for an aspiring Python
contributor would be appreciated. Of course the dev guide gives me
plenty of good info.

Thanks!



-- 
A musician must make music, an artist must paint, a poet must write,
if he is to be ultimately at peace with himself.
- Abraham Maslow

From fijall at gmail.com  Wed Mar 27 00:59:06 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Tue, 26 Mar 2013 16:59:06 -0700
Subject: [Python-Dev] noob contributions to unit tests
In-Reply-To: <CAOFf2a1Uf_hP-xHuHbkrXdEk6sYSiqTRiT3nbvo8LZk3AvJGJQ@mail.gmail.com>
References: <CAOFf2a1Uf_hP-xHuHbkrXdEk6sYSiqTRiT3nbvo8LZk3AvJGJQ@mail.gmail.com>
Message-ID: <CAK5idxS_Ze9bTGsdznTjMUr3rQzbLafeZ_ge9R+KjyeC+TMV0A@mail.gmail.com>

On Tue, Mar 26, 2013 at 4:49 PM, Sean Felipe Wolfe <ether.joe at gmail.com> wrote:
> Hey everybody how are you all :)
>
> I am an intermediate-level python coder looking to get help out. I've
> been reading over the dev guide about helping increase test coverage
> -->
> http://docs.python.org/devguide/coverage.html
>
> And also the third-party code coverage referenced in the devguide page:
> http://coverage.livinglogic.de/
>
> I'm seeing that according to the coverage tool, two of my favorite
> libraries, urllib/urllib2, have no unit tests? Is that correct or am I
> reading it wrong?
>
> If that's correct it seems like a great place perhaps for me to cut my
> teeth and I would be excited to learn and help out here.
>
> And of course any thoughts or advice for an aspiring Python
> contributor would be appreciated. Of course the dev guide gives me
> plenty of good info.
>
> Thanks!

That looks like an error in the coverage report, there are certainly
urllib and urllib2 tests in test/test_urllib*

From rocky at gnu.org  Wed Mar 27 03:11:14 2013
From: rocky at gnu.org (Rocky Bernstein)
Date: Tue, 26 Mar 2013 22:11:14 -0400
Subject: [Python-Dev] Can I introspect/reflect to get arguments exec()?
Message-ID: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>

[asked on comp.lang.python but no takers. So I'm bumping it up a notch.]

I have ported my Python debugger pydbgr to Python3. See [1] or [2].

Inside the debugger, when there is an exec() somewhere in the call stack,
I'd like to be able to retrieve the string parameter. With this, the
debugger can show part of the string in a call stack. Or it can show the
text when the frame is set to that exec() frame.

Going further, the debugger could write the exec string out to a temporary
file. And when reporting locations, it could report not just something like
"<string> line 4", but also give that temporary file name which a front-end
could use as well.

So consider this code using inspect.getargvalues() and
inspect.currentframe():

    import inspect
    def my_exec(string):
        show_args(inspect.currentframe()) # simulate exec(string)

    def show_args(frame):
        print(inspect.getargvalues(frame))

    my_exec("show_args(inspect.currentframe())")
    exec("show_args(inspect.currentframe())")


When run this is the output:

    python3 exec-args.py
    ArgInfo(args=['string'], varargs=None, keywords=None, locals={'string':
'show_args(inspect.currentframe())'})
    ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec':
<function my_exec at 0xb6f828ec>,, ...


In a different setting, CPython byte-code assembly that gets generated for
running exec() is:

     25       88 LOAD_GLOBAL            10 (exec)
              91 LOAD_CONST              4
('show_args(inspect.currentframe())')
       -->    94 CALL_FUNCTION           1
              97 POP_TOP

What's going on?

Also, I have the same question for CPython 2.6 or Python 2.7.

Thanks!

[1] http://code.google.com/p/pydbgr/
[2] http://code.google.com/p/python3-trepan/
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130326/c0f12357/attachment.html>

From benjamin at python.org  Wed Mar 27 03:18:11 2013
From: benjamin at python.org (Benjamin Peterson)
Date: Tue, 26 Mar 2013 22:18:11 -0400
Subject: [Python-Dev] Can I introspect/reflect to get arguments exec()?
In-Reply-To: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>
References: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>
Message-ID: <CAPZV6o9Vczz3k5AHwZKs_+46kR23FKrsGqhuK93N6JGeyjeY0Q@mail.gmail.com>

2013/3/26 Rocky Bernstein <rocky at gnu.org>:
> [asked on comp.lang.python but no takers. So I'm bumping it up a notch.]
>
> I have ported my Python debugger pydbgr to Python3. See [1] or [2].
>
> Inside the debugger, when there is an exec() somewhere in the call stack,
> I'd like to be able to retrieve the string parameter. With this, the
> debugger can show part of the string in a call stack. Or it can show the
> text when the frame is set to that exec() frame.
>
> Going further, the debugger could write the exec string out to a temporary
> file. And when reporting locations, it could report not just something like
> "<string> line 4", but also give that temporary file name which a front-end
> could use as well.
>
> So consider this code using inspect.getargvalues() and
> inspect.currentframe():
>
>     import inspect
>     def my_exec(string):
>         show_args(inspect.currentframe()) # simulate exec(string)
>
>     def show_args(frame):
>         print(inspect.getargvalues(frame))
>
>     my_exec("show_args(inspect.currentframe())")
>     exec("show_args(inspect.currentframe())")
>
>
> When run this is the output:
>
>     python3 exec-args.py
>     ArgInfo(args=['string'], varargs=None, keywords=None, locals={'string':
> 'show_args(inspect.currentframe())'})
>     ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec':
> <function my_exec at 0xb6f828ec>,, ...
>
>
> In a different setting, CPython byte-code assembly that gets generated for
> running exec() is:
>
>      25       88 LOAD_GLOBAL            10 (exec)
>               91 LOAD_CONST              4
> ('show_args(inspect.currentframe())')
>        -->    94 CALL_FUNCTION           1
>               97 POP_TOP
>
> What's going on?

execing something is not the same as calling it, so there are no arguments.



-- 
Regards,
Benjamin

From rdmurray at bitdance.com  Wed Mar 27 03:24:22 2013
From: rdmurray at bitdance.com (R. David Murray)
Date: Tue, 26 Mar 2013 22:24:22 -0400
Subject: [Python-Dev] noob contributions to unit tests
In-Reply-To: <CAK5idxS_Ze9bTGsdznTjMUr3rQzbLafeZ_ge9R+KjyeC+TMV0A@mail.gmail.com>
References: <CAOFf2a1Uf_hP-xHuHbkrXdEk6sYSiqTRiT3nbvo8LZk3AvJGJQ@mail.gmail.com>
	<CAK5idxS_Ze9bTGsdznTjMUr3rQzbLafeZ_ge9R+KjyeC+TMV0A@mail.gmail.com>
Message-ID: <20130327022422.D7ACA250BCA@webabinitio.net>

On Tue, 26 Mar 2013 16:59:06 -0700, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Tue, Mar 26, 2013 at 4:49 PM, Sean Felipe Wolfe <ether.joe at gmail.com> wrote:
> > Hey everybody how are you all :)
> >
> > I am an intermediate-level python coder looking to get help out. I've
> > been reading over the dev guide about helping increase test coverage
> > -->
> > http://docs.python.org/devguide/coverage.html
> >
> > And also the third-party code coverage referenced in the devguide page:
> > http://coverage.livinglogic.de/
> >
> > I'm seeing that according to the coverage tool, two of my favorite
> > libraries, urllib/urllib2, have no unit tests? Is that correct or am I
> > reading it wrong?
> >
> > If that's correct it seems like a great place perhaps for me to cut my
> > teeth and I would be excited to learn and help out here.
> >
> > And of course any thoughts or advice for an aspiring Python
> > contributor would be appreciated. Of course the dev guide gives me
> > plenty of good info.
> >
> > Thanks!
> 
> That looks like an error in the coverage report, there are certainly
> urllib and urllib2 tests in test/test_urllib*

The devguide contains instructions for running coverage yourself,
and if I recall correctly the 'fullcoverage' recipe does a better
job than what runs at coverage.livinglogic.de.

On the other hand, I'm fairly certain that even if the coverage were at
100% code-and-branch coverage, there'd still be tests worth adding,
if you are as familiar with the modules as your intro suggests :)

However, if you are writing new tests, please write them against
the default branch, which means urllib in Python3 (the test files
are still named like they are in Python2, though).

--David

PS: If you aren't aware of the core-mentorship mailing list, you
might want to check that out as well.

From rocky at gnu.org  Wed Mar 27 04:00:55 2013
From: rocky at gnu.org (Rocky Bernstein)
Date: Tue, 26 Mar 2013 23:00:55 -0400
Subject: [Python-Dev] Can I introspect/reflect to get arguments exec()?
In-Reply-To: <CAPZV6o9Vczz3k5AHwZKs_+46kR23FKrsGqhuK93N6JGeyjeY0Q@mail.gmail.com>
References: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>
	<CAPZV6o9Vczz3k5AHwZKs_+46kR23FKrsGqhuK93N6JGeyjeY0Q@mail.gmail.com>
Message-ID: <CANCp2gYB8w0VyzZ4RVTQ6VQjTx0-xDPGxojpHNoyRqV9n_4y6A@mail.gmail.com>

On Tue, Mar 26, 2013 at 10:18 PM, Benjamin Peterson <benjamin at python.org>wrote:

> 2013/3/26 Rocky Bernstein <rocky at gnu.org>:
> > [asked on comp.lang.python but no takers. So I'm bumping it up a notch.]
> >
> > I have ported my Python debugger pydbgr to Python3. See [1] or [2].
> >
> > Inside the debugger, when there is an exec() somewhere in the call stack,
> > I'd like to be able to retrieve the string parameter. With this, the
> > debugger can show part of the string in a call stack. Or it can show the
> > text when the frame is set to that exec() frame.
> >
> > Going further, the debugger could write the exec string out to a
> temporary
> > file. And when reporting locations, it could report not just something
> like
> > "<string> line 4", but also give that temporary file name which a
> front-end
> > could use as well.
> >
> > So consider this code using inspect.getargvalues() and
> > inspect.currentframe():
> >
> >     import inspect
> >     def my_exec(string):
> >         show_args(inspect.currentframe()) # simulate exec(string)
> >
> >     def show_args(frame):
> >         print(inspect.getargvalues(frame))
> >
> >     my_exec("show_args(inspect.currentframe())")
> >     exec("show_args(inspect.currentframe())")
> >
> >
> > When run this is the output:
> >
> >     python3 exec-args.py
> >     ArgInfo(args=['string'], varargs=None, keywords=None,
> locals={'string':
> > 'show_args(inspect.currentframe())'})
> >     ArgInfo(args=[], varargs=None, keywords=None, locals={'my_exec':
> > <function my_exec at 0xb6f828ec>,, ...
> >
> >
> > In a different setting, CPython byte-code assembly that gets generated
> for
> > running exec() is:
> >
> >      25       88 LOAD_GLOBAL            10 (exec)
> >               91 LOAD_CONST              4
> > ('show_args(inspect.currentframe())')
> >        -->    94 CALL_FUNCTION           1
> >               97 POP_TOP
> >
> > What's going on?
>
> execing something is not the same as calling it, so there are no arguments.
>

Okay. But is the string is still somewhere in the CPython VM stack? (The
result of LOAD_CONST 4 above). Is there a way to pick it up from there? At
the point that we are stopped the exec action hasn't taken place yet.




>
>
> --
> Regards,
> Benjamin
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130326/a92cbe26/attachment-0001.html>

From andriy.kornatskyy at live.com  Wed Mar 27 08:40:29 2013
From: andriy.kornatskyy at live.com (Andriy Kornatskyy)
Date: Wed, 27 Mar 2013 10:40:29 +0300
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>,
	<51517E62.2080301@ubuntu.com>,
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>,
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>,
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>,
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>,
	<CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>
Message-ID: <DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>

Any plans backport decimal C implementation from 3.3?

Thanks.

Andriy Kornatskyy


----------------------------------------
> Date: Tue, 26 Mar 2013 16:18:34 -0700
> From: ether.joe at gmail.com
> To: hs at ox.cx
> CC: python-dev at python.org
> Subject: Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
>
> On Tue, Mar 26, 2013 at 3:26 PM, Hynek Schlawack <hs at ox.cx> wrote:
> > Speakerdeck is slides only. The video is here: http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27
>
> Sweet thanks!
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andriy.kornatskyy%40live.com 		 	   		  

From andrew.svetlov at gmail.com  Wed Mar 27 09:24:47 2013
From: andrew.svetlov at gmail.com (Andrew Svetlov)
Date: Wed, 27 Mar 2013 10:24:47 +0200
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>
	<CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>
	<DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>
Message-ID: <CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>

No. _decimal is new functionality that will never be backported.

On Wed, Mar 27, 2013 at 9:40 AM, Andriy Kornatskyy
<andriy.kornatskyy at live.com> wrote:
> Any plans backport decimal C implementation from 3.3?
>
> Thanks.
>
> Andriy Kornatskyy
>
>
> ----------------------------------------
>> Date: Tue, 26 Mar 2013 16:18:34 -0700
>> From: ether.joe at gmail.com
>> To: hs at ox.cx
>> CC: python-dev at python.org
>> Subject: Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
>>
>> On Tue, Mar 26, 2013 at 3:26 PM, Hynek Schlawack <hs at ox.cx> wrote:
>> > Speakerdeck is slides only. The video is here: http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27
>>
>> Sweet thanks!
>> _______________________________________________
>> Python-Dev mailing list
>> Python-Dev at python.org
>> http://mail.python.org/mailman/listinfo/python-dev
>> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andriy.kornatskyy%40live.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com



-- 
Thanks,
Andrew Svetlov

From andriy.kornatskyy at live.com  Wed Mar 27 10:52:54 2013
From: andriy.kornatskyy at live.com (Andriy Kornatskyy)
Date: Wed, 27 Mar 2013 12:52:54 +0300
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>,
	<51517E62.2080301@ubuntu.com>,
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>,
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>,
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>,
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>,
	<CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>,
	<DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>,
	<CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>
Message-ID: <DUB117-W57CEF1B2B9DDED4FE7348D91D10@phx.gbl>

Andrew,

Thank you for the prompt response back.

> will never be backported.

Who knows?

Is python 3.3 _decimal interface 100% compatible with one pure python implementation in 2.7?

I suppose code written using decimals in python 2.7 should work in python 3 with no changes, or there should be taken caution for something?

According to presentation slides, 30x performance boost is something definitely valuable for 2.7 crowd.

Thanks.

Andriy


----------------------------------------
> Date: Wed, 27 Mar 2013 10:24:47 +0200
> Subject: Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
> From: andrew.svetlov at gmail.com
> To: andriy.kornatskyy at live.com
> CC: python-dev at python.org
>
> No. _decimal is new functionality that will never be backported.
>
> On Wed, Mar 27, 2013 at 9:40 AM, Andriy Kornatskyy
> <andriy.kornatskyy at live.com> wrote:
> > Any plans backport decimal C implementation from 3.3?
> >
> > Thanks.
> >
> > Andriy Kornatskyy
> >
> >
> > ----------------------------------------
> >> Date: Tue, 26 Mar 2013 16:18:34 -0700
> >> From: ether.joe at gmail.com
> >> To: hs at ox.cx
> >> CC: python-dev at python.org
> >> Subject: Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
> >>
> >> On Tue, Mar 26, 2013 at 3:26 PM, Hynek Schlawack <hs at ox.cx> wrote:
> >> > Speakerdeck is slides only. The video is here: http://pyvideo.org/video/1730/python-33-trust-me-its-better-than-27
> >>
> >> Sweet thanks!
> >> _______________________________________________
> >> Python-Dev mailing list
> >> Python-Dev at python.org
> >> http://mail.python.org/mailman/listinfo/python-dev
> >> Unsubscribe: http://mail.python.org/mailman/options/python-dev/andriy.kornatskyy%40live.com
> > _______________________________________________
> > Python-Dev mailing list
> > Python-Dev at python.org
> > http://mail.python.org/mailman/listinfo/python-dev
> > Unsubscribe: http://mail.python.org/mailman/options/python-dev/andrew.svetlov%40gmail.com
>
>
>
> --
> Thanks,
> Andrew Svetlov 		 	   		  

From amauryfa at gmail.com  Wed Mar 27 10:54:37 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 27 Mar 2013 10:54:37 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>
	<51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>
	<CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>
	<DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>
	<CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>
Message-ID: <CAGmFidYXMACXakRAP0tSydnfi888YR-3VCuPNw8yRLaxVipQnQ@mail.gmail.com>

2013/3/27 Andrew Svetlov <andrew.svetlov at gmail.com>

> No. _decimal is new functionality that will never be backported.
>
> On Wed, Mar 27, 2013 at 9:40 AM, Andriy Kornatskyy
> <andriy.kornatskyy at live.com> wrote:
> > Any plans backport decimal C implementation from 3.3?
>

No. 2.7 does not accept new features anymore, but you can install
this backport from PyPI: https://pypi.python.org/pypi/cdecimal/2.3

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130327/0ab73f9e/attachment.html>

From andriy.kornatskyy at live.com  Wed Mar 27 11:08:28 2013
From: andriy.kornatskyy at live.com (Andriy Kornatskyy)
Date: Wed, 27 Mar 2013 13:08:28 +0300
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <CAGmFidYXMACXakRAP0tSydnfi888YR-3VCuPNw8yRLaxVipQnQ@mail.gmail.com>
References: <CAPZV6o8PNPmEit4Y-sYFma3TtwKWV1VZL23g1JPGEmr8Gxr80w@mail.gmail.com>,
	<51517E62.2080301@ubuntu.com>,
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>,
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>,
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>,
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>,
	<CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>,
	<DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>,
	<CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>,
	<CAGmFidYXMACXakRAP0tSydnfi888YR-3VCuPNw8yRLaxVipQnQ@mail.gmail.com>
Message-ID: <DUB117-W139D178A30FBC0B3FEF46B91D10@phx.gbl>

> No. 2.7 does not accept new features anymore,?

It is clear now.

> but you can install this backport from PyPI: https://pypi.python.org/pypi/cdecimal/2.3

This is what I was looking for.

Thanks.

Andriy


________________________________
> Date: Wed, 27 Mar 2013 10:54:37 +0100 
> Subject: Re: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1 
> From: amauryfa at gmail.com 
> To: andrew.svetlov at gmail.com 
> CC: andriy.kornatskyy at live.com; python-dev at python.org 
> 
> 2013/3/27 Andrew Svetlov 
> <andrew.svetlov at gmail.com<mailto:andrew.svetlov at gmail.com>> 
> No. _decimal is new functionality that will never be backported. 
> 
> On Wed, Mar 27, 2013 at 9:40 AM, Andriy Kornatskyy 
> <andriy.kornatskyy at live.com<mailto:andriy.kornatskyy at live.com>> wrote: 
> > Any plans backport decimal C implementation from 3.3? 
> 
> No. 2.7 does not accept new features anymore, but you can install 
> this backport from PyPI: https://pypi.python.org/pypi/cdecimal/2.3 
> 
> -- 
> Amaury Forgeot d'Arc 		 	   		  

From stefan at bytereef.org  Wed Mar 27 12:34:35 2013
From: stefan at bytereef.org (Stefan Krah)
Date: Wed, 27 Mar 2013 12:34:35 +0100
Subject: [Python-Dev] [RELEASE] Python 2.7.4 release candidate 1
In-Reply-To: <DUB117-W139D178A30FBC0B3FEF46B91D10@phx.gbl>
References: <51517E62.2080301@ubuntu.com>
	<CAL0kPAWDT3gXAnuCjQtyup_9ESLAgkfW5TOP7+U4fSH3mCG-sA@mail.gmail.com>
	<CAMpsgwYHVTXsFXZenAk8RXY5ijpiv8wfSugWhMuKFOwyPjD7VQ@mail.gmail.com>
	<CAOFf2a22Ef_wLcSVDB7HWDsT9gqD0FGCywuG9Jm8DmvZW7wzNA@mail.gmail.com>
	<8ADF93F8-51CA-4943-8C9A-BD21D2D474D7@ox.cx>
	<CAOFf2a3otcRaup9T-TeDiZTN6N_NUD0z4Ws0P8H8K+afjZYCdA@mail.gmail.com>
	<DUB117-W3477A8C5E49B7C37D903E791D10@phx.gbl>
	<CAL3CFcWAkuY=hV7FfcZNH8bcK+RZz7sqj90vfX1NsJDAAcb0+w@mail.gmail.com>
	<CAGmFidYXMACXakRAP0tSydnfi888YR-3VCuPNw8yRLaxVipQnQ@mail.gmail.com>
	<DUB117-W139D178A30FBC0B3FEF46B91D10@phx.gbl>
Message-ID: <20130327113435.GA32709@sleipnir.bytereef.org>

Andriy Kornatskyy <andriy.kornatskyy at live.com> wrote:
> > but you can install this backport from PyPI: https://pypi.python.org/pypi/cdecimal/2.3
> 
> This is what I was looking for.

Note that for numerical work _decimal from Python3.3 is vastly faster than
cdecimal. I've added two major speedups (credit for one of them goes to
Antoine), so these are the numbers for the pi benchmark:

Python3.3 (_decimal):    time:  0.15s
Python3.3 (decimal.py):  time: 17.22s
=====================================

Python2.7 (cdecimal):    time:  0.29s
Python2.7 (decimal.py):  time: 17.74s
=====================================


For database work and such the numbers should be about the same.


Stefan Krah



From vinay_sajip at yahoo.co.uk  Wed Mar 27 20:38:03 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 27 Mar 2013 19:38:03 +0000 (UTC)
Subject: [Python-Dev] Safely importing zip files with C extensions
Message-ID: <loom.20130327T203142-944@post.gmane.org>

> This quote is here to stop GMane complaining that I'm top-posting. Ignore.

I've already posted this to distutils-sig, but thought that it might be of
interest to readers here as it relates to importing C extensions ...

zipimport is great, but there can be issues importing software that contains
C extensions. But the new wheel format (PEP 427) may give us a better way of
importing zip files containing C extensions. Since wheels are .zip files, they
can sometimes be used to provide functionality without needing to be installed.
But whereas .zip files contain no convention for indicating compatibility with
a particular Python, wheels do contain this compatibility information. Thus, it
is possible to check if a wheel can be directly imported from, and the wheel
support in distlib allows you to take advantage of this using the mount() and
unmount() methods. When you mount a wheel, its absolute path name is added to
sys.path, allowing the Python code in it to be imported. (A DistlibException is
raised if the wheel isn't compatible with the Python which calls the mount()
method.)

You don't need mount() just to add the wheel's name to sys.path, or to import
pure-Python wheels, of course. But the mount() method goes further than just
enabling Python imports - any C extensions in the wheel are also made available
for import. For this to be possible, the wheel has to be built with additional
metadata about extensions - a JSON file called EXTENSIONS which serialises an
extension mapping dictionary. This maps extension module names to the names in
the wheel of the shared libraries which implement those modules.

Running unmount() on the wheel removes its absolute pathname from sys.path and
makes its C extensions, if any, also unavailable for import.

Wheels built with the new "distil" tool contain the EXTENSIONS metadata, so can
be mounted complete with C extensions:

$ distil download -d /tmp simplejson
Downloading simplejson-3.1.2.tar.gz to /tmp/simplejson-3.1.2
    63KB @  73 KB/s 100 % Done: 00:00:00
Unpacking ... done.
$ distil package --fo=wh -d /tmp /tmp/simplejson-3.1.2/
The following packages were built:
  /tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl
$ python
Python 2.7.2+ (default, Jul 20 2012, 22:15:08) 
[GCC 4.6.1] on linux2
Type "help", "copyright", "credits" or "license" for more information.
>>> from distlib.wheel import Wheel
>>> w = Wheel('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
>>> w.mount()
>>> import simplejson._speedups
>>> dir(simplejson._speedups)
['__doc__', '__file__', '__loader__', '__name__', '__package__',
 'encode_basestring_ascii', 'make_encoder', 'make_scanner', 'scanstring']
>>> simplejson._speedups.__file__
'/home/vinay/.distlib/dylib-cache/simplejson/_speedups.so'
>>> 

Does anyone see any problems with this approach to importing C extensions from
zip files?

Regards,

Vinay Sajip


From amauryfa at gmail.com  Wed Mar 27 21:13:27 2013
From: amauryfa at gmail.com (Amaury Forgeot d'Arc)
Date: Wed, 27 Mar 2013 21:13:27 +0100
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <loom.20130327T203142-944@post.gmane.org>
References: <loom.20130327T203142-944@post.gmane.org>
Message-ID: <CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>

2013/3/27 Vinay Sajip <vinay_sajip at yahoo.co.uk>

> When you mount a wheel, its absolute path name is added to
> sys.path, allowing the Python code in it to be imported.
>

Better: just put the wheel path to sys.path
    sys.path.append('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
and let a sys.path_hook entry do the job.

Such a WheelImporter could even inherit from zipimporter, plus the magic
required for C extensions.

It avoids the mount/nomount methods, only the usual sys.path operations are
necessary from the user.

-- 
Amaury Forgeot d'Arc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130327/61141a0e/attachment.html>

From stefan_ml at behnel.de  Wed Mar 27 21:23:36 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 27 Mar 2013 21:23:36 +0100
Subject: [Python-Dev] astoptimizer: static optimizer working on the AST
In-Reply-To: <CAMpsgwaU8RHQDFibH5p5TxRd=eR7OT0cxy2y9xhEpHS_RDX_hw@mail.gmail.com>
References: <CAMpsgwaU8RHQDFibH5p5TxRd=eR7OT0cxy2y9xhEpHS_RDX_hw@mail.gmail.com>
Message-ID: <kivkg4$jls$1@ger.gmane.org>

Victor Stinner, 26.03.2013 22:56:
>  * what should be the name of "pyc" files?
>  * how to handle different configuration of astoptimizer: generate
> different "pyc" files?

You could use (a part of) a crypto hash of the serialised options as part
of the filename. One drawback of a .pyc duplication is that when you ship
code to other environments that run with different optimiser flags, those
would not pick up the same .pyc files. Although, would it really be a
problem if different modules that were compiled with different optimisation
settings get mixed on the same machine? Are there any non semantics
preserving AST modifications for a module that have an impact on other
modules? I.e., is this just a development time concern when trying out
different optimisations, or is this a problem for deployed code as well?

Stefan



From stefan_ml at behnel.de  Wed Mar 27 21:34:19 2013
From: stefan_ml at behnel.de (Stefan Behnel)
Date: Wed, 27 Mar 2013 21:34:19 +0100
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <loom.20130327T203142-944@post.gmane.org>
References: <loom.20130327T203142-944@post.gmane.org>
Message-ID: <kivl48$rt1$1@ger.gmane.org>

Vinay Sajip, 27.03.2013 20:38:
> >>> w = Wheel('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
> >>> w.mount()
> >>> import simplejson._speedups
> >>> dir(simplejson._speedups)
> ['__doc__', '__file__', '__loader__', '__name__', '__package__',
>  'encode_basestring_ascii', 'make_encoder', 'make_scanner', 'scanstring']
> >>> simplejson._speedups.__file__
> '/home/vinay/.distlib/dylib-cache/simplejson/_speedups.so'

I've always hated this setuptools misfeature of copying C extensions from
an installed archive into a user directory, one for each user. At least
during normal installation, they should be properly unpacked into normal
shared library files in the file system.

Whether it then makes sense to special case one-shot trial imports like the
above without installation is a bit of a different question, but I don't
see a compelling reason for adding complexity here. It's not really an
important use case.

Stefan



From vinay_sajip at yahoo.co.uk  Wed Mar 27 21:41:05 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 27 Mar 2013 20:41:05 +0000 (UTC)
Subject: [Python-Dev] Safely importing zip files with C extensions
References: <loom.20130327T203142-944@post.gmane.org>
	<CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>
Message-ID: <loom.20130327T213424-139@post.gmane.org>

Amaury Forgeot d'Arc <amauryfa <at> gmail.com> writes:


> Better: just put the wheel path to sys.path? ?
sys.path.append('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
> and let a sys.path_hook entry do the job.

That's what the mount() actually does - adds the wheel to a registry that an
import hook uses. You also need a place to check that the wheel being mounted
is compatible with the Python doing the mounting - I'm not sure whether what
the import hook should do if e.g. there is a compatibility problem with the
wheel (e.g. is it clear that it should always raise an ImportError? Or ignore
the wheel - seems wrong? Or do something else?)

Regards,

Vinay Sajip


From dholth at gmail.com  Wed Mar 27 21:49:59 2013
From: dholth at gmail.com (Daniel Holth)
Date: Wed, 27 Mar 2013 16:49:59 -0400
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <loom.20130327T213424-139@post.gmane.org>
References: <loom.20130327T203142-944@post.gmane.org>
	<CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>
	<loom.20130327T213424-139@post.gmane.org>
Message-ID: <CAG8k2+67V8+sh2Cn2GM5DO5rjCfNsmCSKqPMmh7x22ErYsSu7w@mail.gmail.com>

Jim Fulton is right that weird failures are a characteristic of zipped
eggs, so one of the #1 requests for setuptools is how to prohibit
zipping from ever happening. This is an important reason why wheel is
billed as an installation format -- fewer users with pitchforks. It's
very cool that it works though. Debugging is slightly easier than it
was in the old days because pdb can now read the source code from the
zip.

An unzipped wheel as a directory with the same name as the wheel would
be a more reliable solution that might be interesting to work with. It
would work the same as egg unless you had important files in the
.data/ (currently mostly used for console scripts and include files)
directory. However it was always confusing to have more than one kind
(zipped, unzipped) of egg.

On Wed, Mar 27, 2013 at 4:41 PM, Vinay Sajip <vinay_sajip at yahoo.co.uk> wrote:
> Amaury Forgeot d'Arc <amauryfa <at> gmail.com> writes:
>
>
>> Better: just put the wheel path to sys.path
> sys.path.append('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
>> and let a sys.path_hook entry do the job.
>
> That's what the mount() actually does - adds the wheel to a registry that an
> import hook uses. You also need a place to check that the wheel being mounted
> is compatible with the Python doing the mounting - I'm not sure whether what
> the import hook should do if e.g. there is a compatibility problem with the
> wheel (e.g. is it clear that it should always raise an ImportError? Or ignore
> the wheel - seems wrong? Or do something else?)
>
> Regards,
>
> Vinay Sajip
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From vinay_sajip at yahoo.co.uk  Wed Mar 27 21:59:11 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 27 Mar 2013 20:59:11 +0000 (UTC)
Subject: [Python-Dev] Safely importing zip files with C extensions
References: <loom.20130327T203142-944@post.gmane.org>
	<kivl48$rt1$1@ger.gmane.org>
Message-ID: <loom.20130327T215131-474@post.gmane.org>

Stefan Behnel <stefan_ml <at> behnel.de> writes:


> I've always hated this setuptools misfeature of copying C extensions from
> an installed archive into a user directory, one for each user. At least
> during normal installation, they should be properly unpacked into normal
> shared library files in the file system.

The user directory location is not a key part of the functionality, it could
just as well be a shared location across all users. And this is an option for
specific scenarios, not a general substitute for installing the wheel (which
unpacks everything into FHS-style locations). A lot of people use virtual envs,
which are per-user anyway. I'm not suggesting this is a good idea for system-wide
deployments of software.

> Whether it then makes sense to special case one-shot trial imports like the
> above without installation is a bit of a different question, but I don't
> see a compelling reason for adding complexity here. It's not really an
> important use case.

Well, my post was to elicit some comment about the usefulness of the feature,
so fair enough. It doesn't seem especially complex though, unless I've missed
something.

Regards,

Vinay Sajip


From vinay_sajip at yahoo.co.uk  Wed Mar 27 22:04:40 2013
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Wed, 27 Mar 2013 21:04:40 +0000 (UTC)
Subject: [Python-Dev] Safely importing zip files with C extensions
References: <loom.20130327T203142-944@post.gmane.org>
	<CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>
	<loom.20130327T213424-139@post.gmane.org>
	<CAG8k2+67V8+sh2Cn2GM5DO5rjCfNsmCSKqPMmh7x22ErYsSu7w@mail.gmail.com>
Message-ID: <loom.20130327T220207-616@post.gmane.org>

Daniel Holth <dholth <at> gmail.com> writes:

> zipping from ever happening. This is an important reason why wheel is
> billed as an installation format -- fewer users with pitchforks. It's
> very cool that it works though. Debugging is slightly easier than it
> was in the old days because pdb can now read the source code from the
> zip.

Well, it's just an experiment, and I was soliciting comments because I'm not as
familiar with the issues as some others are. Distlib is still only at version
0.1.1, and the mount()/unmount() functionality is not set in stone :-)

Regards,

Vinay Sajip


From brad.froehle at gmail.com  Wed Mar 27 22:19:39 2013
From: brad.froehle at gmail.com (Bradley M. Froehle)
Date: Wed, 27 Mar 2013 14:19:39 -0700
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>
References: <loom.20130327T203142-944@post.gmane.org>
	<CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>
Message-ID: <CAHXv-Mi7vGUR+UVmnVBjAgY8KW-PJ6fMJfd_zXxo_mxzM3+iUQ@mail.gmail.com>

On Wed, Mar 27, 2013 at 1:13 PM, Amaury Forgeot d'Arc <amauryfa at gmail.com>wrote:

> 2013/3/27 Vinay Sajip <vinay_sajip at yahoo.co.uk>
>
>> When you mount a wheel, its absolute path name is added to
>> sys.path, allowing the Python code in it to be imported.
>>
>
> Better: just put the wheel path to sys.path
>     sys.path.append('/tmp/simplejson-3.1.2-cp27-none-linux_x86_64.whl')
> and let a sys.path_hook entry do the job.
>
> Such a WheelImporter could even inherit from zipimporter, plus the magic
> required for C extensions.
>

I implemented just such a path hook ---- zipimporter plus the magic
required for C extensions --- as a challenge to myself to learn more about
the Python import mechanisms.

See https://github.com/bfroehle/pydzipimport.

Cheers,
Brad
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130327/3c4baf7b/attachment.html>

From p.f.moore at gmail.com  Wed Mar 27 23:59:55 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Wed, 27 Mar 2013 22:59:55 +0000
Subject: [Python-Dev] Writing importers and path hooks
Message-ID: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>

On 27 March 2013 21:19, Bradley M. Froehle <brad.froehle at gmail.com> wrote:
> I implemented just such a path hook ---- zipimporter plus the magic required
> for C extensions --- as a challenge to myself to learn more about the Python
> import mechanisms.
>
> See https://github.com/bfroehle/pydzipimport.

Apologies for hijacking the thread, but it's interesting that you
implemented your hook like this. I notice that you didn't use any of
the importlib functionality in doing so. Was there a particular
reason? I ask because a few days ago, I was writing a very similar
importer, as I wanted to try a proof of concept importer based on the
new importlib stuff (which is intended to make writing custom
importers easier), and I really struggled to get something working.

It seems to me that the importlib documentation doesn't help much for
people trying to import path hooks. But it might be just me. Does
anyone have an example of a simple importlib-based finder/loader? That
would be a huge help for me. In return for any pointers, I'll look at
putting together a doc patch to clarify how to use importlib to build
your own path hooks :-)

Thanks,
Paul

From terri at zone12.com  Thu Mar 28 00:25:29 2013
From: terri at zone12.com (Terri Oda)
Date: Wed, 27 Mar 2013 17:25:29 -0600
Subject: [Python-Dev] Google Summer of Code - Organization Deadline
 Approaching - March 29
In-Reply-To: <CAD+XWwobKwo-A3=1gSvyuZCN-dig_cBrn7XM=QyAFG=WRZY2CQ@mail.gmail.com>
References: <CAD+XWwobKwo-A3=1gSvyuZCN-dig_cBrn7XM=QyAFG=WRZY2CQ@mail.gmail.com>
Message-ID: <51537FE9.4000402@zone12.com>

On 03/26/2013 01:51 PM, Brian Curtin wrote:
> Just an FYI that there are under 3 days to apply to Google Summer of
> Code for mentoring organizations:
> http://www.google-melange.com/gsoc/homepage/google/gsoc2013. The
> student application deadline is later on in May.
>
> If you run a project that is interested in applying under the Python
> umbrella organization, contact Terri Oda at terri at zone12.com

I would obviously love it if you got your ideas up before the March 29th 
deadline so our application will totally shine and we'll be accepted 
immediately in to the program.

But I should note that while the Google deadline is March 29th, I'm 
happy to accept applications for sub-organizations and new project ideas 
after that deadline.  The students will be descending on April 9th, so 
if you want to attract the very best, try to get your project ideas up 
near then if at all possible!  Presuming we get accepted, though, I can 
take extra ideas almost up until the student applications start to come 
in, but I'd prefer them before April 15th (again, so you can attract the 
best students).

> Is anyone here interested in leading CPython through GSOC? Anyone have
> potential students to get involved, or interested in being a mentor?

For those on the fence about mentoring or worried about doing a good 
job, I have a few experienced GSoC mentors who are currently without 
projects who have volunteered to help us out this year.  I'm happy to 
pair anyone who's interested with someone who loves working with GSoC 
students but maybe won't be familiar with the project, so you can learn 
the mentoring ropes and do code reviews but have a backup mentor there 
to help both you and the student through the GSoC process.  Please let 
me know ASAP if you'd like to do this so I can introduce you to the 
person and give them time to learn as much as they can about the 
projects you're hoping to run before students start arriving.

  Terri


From pje at telecommunity.com  Thu Mar 28 05:10:32 2013
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 28 Mar 2013 00:10:32 -0400
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <CAHXv-Mi7vGUR+UVmnVBjAgY8KW-PJ6fMJfd_zXxo_mxzM3+iUQ@mail.gmail.com>
References: <loom.20130327T203142-944@post.gmane.org>
	<CAGmFidZhioRCCX9J1cx-AJX=fy+eVsNWeoU5eZ9y4qvy0fWnrQ@mail.gmail.com>
	<CAHXv-Mi7vGUR+UVmnVBjAgY8KW-PJ6fMJfd_zXxo_mxzM3+iUQ@mail.gmail.com>
Message-ID: <CALeMXf5LR96DYzvr1Gvrk26638Y7709Do1kn44+6nocOTBB_Cw@mail.gmail.com>

On Wed, Mar 27, 2013 at 5:19 PM, Bradley M. Froehle
<brad.froehle at gmail.com> wrote:
> I implemented just such a path hook ---- zipimporter plus the magic required
> for C extensions --- as a challenge to myself to learn more about the Python
> import mechanisms.
>
> See https://github.com/bfroehle/pydzipimport.

FYI, there appears to be a bug for Windows with packages: you're using
'/__init__' in a couple places that should actually be
os.sep+'__init__'.

This does seem like a good way to address the issue, for those rare
situations where this would be a good idea.

The zipped .egg approach was originally intended for user-managed
plugin directories for certain types of extensible platforms, where
"download a file and stick it in the plugins directory" is a
low-effort way to install plugins, without having to build a lot of
specialized install capability.

As Jim has pointed out, though, this doesn't generalize well to a
full-blown packaging system.

Technically, you can blame Bob Ippolito for this, since he's the one
who talked me into using eggs to install Python libraries in general,
not just as a plugin packaging mechanism.  ;-)

That being said, *unpacked* egg, er, wheels, are still a great way to
meet all of the "different apps needing different versions" use cases
(without needing one venv per app), and nowadays the existence of
automated installer tools means that using one to install a plugin for
a low-tech plugin system is not a big deal, as long as that tool
supports the simple unpacked wheel scenario.  So I wholeheartedly
support some kind of mount/unmount or "require"-type mechanism for
finding plugins.  pkg_resources even has an API for handling simple
dynamic plugin dependency resolution scenarios:

  http://peak.telecommunity.com/DevCenter/PkgResources#locating-plugins

It'd be a good idea if distlib provides a similar feature, or at least
the APIs upon which apps or frameworks can implement such features.

(Historical note for those who weren't around back then: easy_install
wasn't even an *idea* until well after eggs were created; the original
idea was just that people would build plugins and libraries as eggs
and manually drop them in directories, where a plugin support library
would discover them and add them to sys.path as needed.  And Bob and I
also considered a sort of "update site" mechanism ala Eclipse, with a
library to let apps fetch plugins.  But as soon as eggs existed and
PyPI allowed uploads, it was kind of an obvious follow-up to make an
installation tool as a kind of "technology demonstration"....  which
promptly became a monster.   The full story with all its twists and
turns can also be found here:
http://mail.python.org/pipermail/python-dev/2006-April/064145.html )

From pje at telecommunity.com  Thu Mar 28 06:45:18 2013
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 28 Mar 2013 01:45:18 -0400
Subject: [Python-Dev] Can I introspect/reflect to get arguments exec()?
In-Reply-To: <CANCp2gYB8w0VyzZ4RVTQ6VQjTx0-xDPGxojpHNoyRqV9n_4y6A@mail.gmail.com>
References: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>
	<CAPZV6o9Vczz3k5AHwZKs_+46kR23FKrsGqhuK93N6JGeyjeY0Q@mail.gmail.com>
	<CANCp2gYB8w0VyzZ4RVTQ6VQjTx0-xDPGxojpHNoyRqV9n_4y6A@mail.gmail.com>
Message-ID: <CALeMXf7YoC6JYPc_PkvWrg38uMdJvhJSKJGcsxzzpCcYPd1uSA@mail.gmail.com>

On Tue, Mar 26, 2013 at 11:00 PM, Rocky Bernstein <rocky at gnu.org> wrote:
> Okay. But is the string is still somewhere in the CPython VM stack? (The
> result of LOAD_CONST 4 above). Is there a way to pick it up from there?

Maybe using C you could peek into the frame's value stack, but that's
not exposed to any Python API I know of.  But that still doesn't help
you, because the value will be removed from the stack before exec() is
actually called, which means if you go looking for it in code called
from the exec (e.g. the call event itself), you aren't going to see
the data.

> At the point that we are stopped the exec action hasn't taken place yet.

That doesn't help if you're using line-level tracing events.  At the
beginning of the line, the data's not on the call stack yet, and by
the time you enter the frame of the code being exec'd, it'll be off
the stack again.

Basically, there is no way to do what you're asking for, short of
replacing the built-in exec function with your own version.  And it
still won't help you with stepping through the source of functions
that are *compiled* using an exec() or compile(), or any other way of
ending up with dynamically-generated code you want to debug.

(Unless you use something like DecoratorTools to generate it, that is
-- DecoratorTools has some facilities for caching
dynamically-generated code so that it works properly with debuggers.
But that has to be done by the code doing the generation, not the
debugger.  If the code generator uses DecoratorTools' caching support,
then any debugger that uses the linecache module will Just Work.  It
might be nice for  the stdlib should have something like this, but you
could also potentially fake it by replacing the builtin eval, exec,
compile, etc. functions w/versions that cache the source.)

From trent at snakebite.org  Thu Mar 28 07:26:51 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 28 Mar 2013 02:26:51 -0400
Subject: [Python-Dev] Post-PyCon updates to PyParallel
Message-ID: <20130328062651.GA64080@snakebite.org>

    [ python-dev: I've set up a new list for pyparallel discussions:
      https://lists.snakebite.net/mailman/listinfo/pyparallel.  This
      e-mail will be the last I'll send to python-dev@ regarding the
      on-going pyparallel work; please drop python-dev@ from the CC
      and just send to pyparallel at lists.snakebite.net -- I'll stay on
      top of the posts-from-unsubscribed-users moderation for those that
      want to reply to this e-mail but not subscribe. ]

Hi folks,

    Wanted to give a quick update on the parallel work both during and
    after PyCon.  During the language summit when I presented the slides
    I uploaded to speakerdeck.com, the majority of questions from other
    developers revolved around the big issues like data integrity and
    what happens when parallel objects interact with main-thread objects
    and vice-versa.

    So, during the sprints, I explored putting guards in place to throw
    an exception if we detect that a user has assigned a parallel object
    to a non-protected main-thread object.

        (I describe the concept of 'protection' in my follow up posts to
         python-dev last week: http://mail.python.org/pipermail/python-dev/2013-March/124690.html.
         
         Basically, protecting a main-thread object allows code like
         this to work without crashing:
            d = async.dict()
            def foo():
                # async.rdtsc() is a helper method
                # that basically wraps the result of
                # the assembly RDTSC (read time-
                # stamp counter) instruction into a
                # PyLong object.  So, it's handy when
                # I need to test the very functionality
                # being demonstrated here (creating
                # an object within a parallel context
                # and persisting it elsewhere).
                d['foo'] = async.rdtsc()

            def bar():
                d['bar'] = async.rdtsc()

            async.submit_work(foo)
            async.submit_work(bar)
         )

    It was actually pretty easy, far easier than I expected.  It was
    achieved via Px_CHECK_PROTECTION():

        https://bitbucket.org/tpn/pyparallel/commits/f3fe082668c6f3f699db990f046291ff66b1b467#LInclude/object.hT1072

    Various new tests related to the protection functionality:

        https://bitbucket.org/tpn/pyparallel/commits/f3fe082668c6f3f699db990f046291ff66b1b467#LLib/async/test/test_primitives.pyT58

    The type of changes I had to make to other parts of CPython to
    perform the protection checks:

        https://bitbucket.org/tpn/pyparallel/commits/f3fe082668c6f3f699db990f046291ff66b1b467#LObjects/abstract.cT170

    That was all working fine... until I started looking at adding
    support for lists (i.e. appending a parallel thread object to a
    protected, main-thread list).

    The problem is that appending to a list will often involve a list
    resize, which is done via PyMem_REALLOC() and some custom fiddling.
    That would mean if a parallel thread attempts to append to a list
    and it needs resizing, all the newly realloc'd memory would be
    allocated from the parallel context's heap.  Now, this heap would
    stick around as long as the parallel objects have a refcount > 0.

    However, as soon as the last parallel object's refcount hits 0, the
    entire context will be scheduled for the cleanup/release/free dance,
    which will eventually blow away the entire heap and all the memory
    allocated against that heap... which means all the **ob_item stuff
    that was reallocated as part of the list resize.

    Not particularly desirable :-)  As I was playing around with ways to
    potentially pre-allocate lists, it occurred to me that dicts would
    be affected in the exact same way; I just hadn't run into it yet
    because my unit tests only ever assigned a few (<5) objects to the
    protected dicts.

    Once the threshold gets reached (10?), a "dict resize" would take
    place, which would involve lots of PyMem_REALLOCs, and we get into
    the exact same situation mentioned above.

    So, at that point, I concluded that whole async protection stuff was
    not a viable long term solution.  (In fact, the reason I first added
    it was simply to have an easy way to test things in unit tests.)

    The new solution I came up with: new thread-safe, interlocked data
    types that are *specifically* designed for this exact use case;
    transferring results from computation in a parallel thread back to
    a main thread 'container' object.

    First up is a new list type: xlist() (PyXListObject/PyXList_Type).
    I've just committed the work-in-progress stuff I've been able to
    hack out whilst traveling the past few days:

        https://bitbucket.org/tpn/pyparallel/commits/5b662eba4efe83e94d31bd9db4520a779aea612a

    It's not finished, and I'm pretty sure it doesn't even compile yet,
    but the idea is something like this:

        results = xlist()

        def worker1(input):
            # do work
            result = useful_work1()
            results.push(result)

        def worker2(input):
            # do work
            result = useful_work2()
            results.push(result)

        data = data_to_process()
        async.submit_work(worker1, data[:len(data)])
        async.submit_work(worker2, data[len(data):])
        async.run()

        for result in results:
            print(result)

    The big change is what happens during xlist.push():

        https://bitbucket.org/tpn/pyparallel/commits/5b662eba4efe83e94d31bd9db4520a779aea612a#LPython/pyparallel.cT3844

+PyObject *
+xlist_push(PyObject *obj, PyObject *src)
+{
+    PyXListObject *xlist = (PyXListObject *)obj;
+    assert(src);
+
+    if (!Py_PXCTX)
+        PxList_PushObject(xlist->head, src);
+    else {
+        PyObject *dst;
+        _PyParallel_SetHeapOverride(xlist->heap_handle);
+        dst = PyObject_Clone(src, "objects of type %s cannot "
+                                  "be pushed to xlists");
+        _PyParallel_RemoveHeapOverride();
+        if (!dst)
+            return NULL;
+        PxList_PushObject(xlist->head, dst);
+    }
+
+    /*
+    if (Px_CV_WAITERS(xlist))
+        ConditionVariableWakeOne(&(xlist->cv));
+    */
+
+    Py_RETURN_NONE;
 }

    Note the heap override and PyObject_Clone(), which currently looks
    like this:

+PyObject *
+PyObject_Clone(PyObject *src, const char *errmsg)
+{
+    int valid_type;
+    PyObject *dst;
+    PyTypeObject *tp;
+
+    tp = Py_TYPE(src);
+
+    valid_type = (
+        PyBytes_CheckExact(src)         ||
+        PyByteArray_CheckExact(src)     ||
+        PyUnicode_CheckExact(src)       ||
+        PyLong_CheckExact(src)          ||
+        PyFloat_CheckExact(src)
+    );
+
+    if (!valid_type) {
+        PyErr_Format(PyExc_ValueError, errmsg, tp->tp_name);
+        return NULL;
+    }
+
+    if (PyLong_CheckExact(src)) {
+
+    } else if (PyFloat_CheckExact(src)) {
+
+    } else if (PyUnicode_CheckExact(src)) {
+
+    } else {
+        assert(0);
+    }
+
+
+}

    Initially, I just want to get support working for simple types that
    are easy to clone.  Any sort of GC/container types will obviously
    take a lot more work as they need to be deep-copied.

    You might also note the Px_CV_WAITERS() bit; these interlocked lists
    could quite easily function as producer/consumer queues, so, maybe
    you could do something like this:

        queue = xlist()

        def consumer(input):
            # do work
            ...

        def producer():
            for i in xrange(100):
                queue.push(i)

        async.submit_queue(queue, consumer)
        async.submit_work(producer)

    Oh, forgot to mention the heap-override specifics: each xlist() gets
    its own heap handle -- when the "pushing" is done and the parallel
    object needs to be copied, the new memory is allocated against the
    xlist's heap.  That heap will stick around until the xlist's refcnt
    hits 0, then everything will be blown away in one fell swoop.

    (Which means I'll need to tweak the memory/refcnt intercepts to
     handle this new concept -- like I had to do to support the notion
     of persisted contexts.  Not a big deal.)

    I really like this approach; much more so than the persisted context
    stuff and the even-more-convoluted promotion stuff (yet to be
    written).  Encapsulating all the memory associated with parallel to
    main-thread object transitions in the very object that is used to
    effect the transition just feels right.

    So, that means there are three main "memory alloc override"-type
    modes currently:

        - Normal.  (Main-thread stuff, ref counting, PyMalloc stuff.)
        - Purely parallel.  (Context-specific heap stuff, very fast.)
        - Parallel->main-thread transitions.  (The stuff above.)

    (Or rather, there will be, once I finish this xlist stuff.  That'll
     allow me to deprecate the TLS override stuff and the Context
     persistence stuff, both of which were nice experiments but fizzled
     out in practice.)

    ....the Px_CHECK_PROTECTION() work was definitely useful though and
    will need to be expanded to all objects.  This will allow us to
    raise an exception if someone attempts to assign a parallel object to
    a normal main thread object (instead of one of the approved
    interlocked/parallel objects (like xlist)).


    Regards,

        Trent.

From trent at snakebite.org  Thu Mar 28 07:32:56 2013
From: trent at snakebite.org (Trent Nelson)
Date: Thu, 28 Mar 2013 02:32:56 -0400
Subject: [Python-Dev] Post-PyCon updates to PyParallel
In-Reply-To: <20130328062651.GA64080@snakebite.org>
References: <20130328062651.GA64080@snakebite.org>
Message-ID: <20130328063256.GD64080@snakebite.org>

On Wed, Mar 27, 2013 at 11:26:51PM -0700, Trent Nelson wrote:
>     [ python-dev: I've set up a new list for pyparallel discussions:
>       https://lists.snakebite.net/mailman/listinfo/pyparallel.  This
>       e-mail will be the last I'll send to python-dev@ regarding the
>       on-going pyparallel work; please drop python-dev@ from the CC
>       and just send to pyparallel at lists.snakebite.net -- I'll stay on
>       top of the posts-from-unsubscribed-users moderation for those that
>       want to reply to this e-mail but not subscribe. ]

    Gah, wrong e-mail address, it's pyparallel at snakebite.net.

        Trent.

From rocky at gnu.org  Thu Mar 28 11:43:10 2013
From: rocky at gnu.org (Rocky Bernstein)
Date: Thu, 28 Mar 2013 06:43:10 -0400
Subject: [Python-Dev] Can I introspect/reflect to get arguments exec()?
In-Reply-To: <CALeMXf7YoC6JYPc_PkvWrg38uMdJvhJSKJGcsxzzpCcYPd1uSA@mail.gmail.com>
References: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>
	<CAPZV6o9Vczz3k5AHwZKs_+46kR23FKrsGqhuK93N6JGeyjeY0Q@mail.gmail.com>
	<CANCp2gYB8w0VyzZ4RVTQ6VQjTx0-xDPGxojpHNoyRqV9n_4y6A@mail.gmail.com>
	<CALeMXf7YoC6JYPc_PkvWrg38uMdJvhJSKJGcsxzzpCcYPd1uSA@mail.gmail.com>
Message-ID: <CANCp2gZVkRxiFsYVpRxcpf3H0szZTFPX3Vv-t-jCfabGjndvNg@mail.gmail.com>

Thank you for your very thoughtful and detailed explanation of what is
going on and for your considerations as to how to get this done (as opposed
to why it *can't *be done).

It looks like DecoratorTools or more likely a customized version of it is
the way to go!

(The important info is above. The rest are just some geeky details)

You see, there are some places where additional care may be needed in my
setting.

The debuggers I write use sometimes don't use just the getline module but
also use my own pyficache module. I want to also cache things like file
stat() info, provide a SHA1 for the text of the file, and provide colorized
syntax-highlighted versions of the text when desired. But since I control
pyficache, I can mirror the changes made to getline.

Of course the debugger uses sys.settrace too, so the evil-ness of that is
definitely not a concern. But possibly I need to make sure that since the
DecoratorTools and the debugger both hook into trace hooks they play nice
together and fire in the right order. And for that I created another module
called tracer() which takes into account that one might want to specify
priorities in the chain hook order, and that one might want a to filter out
(i.e. ignore) certain calls to the hook function for specific hooks.

It may be a while before I seriously get to this, but again, it is good to
have in mind an approach to take. So thanks again.

On Thu, Mar 28, 2013 at 1:45 AM, PJ Eby <pje at telecommunity.com> wrote:

> On Tue, Mar 26, 2013 at 11:00 PM, Rocky Bernstein <rocky at gnu.org> wrote:
> > Okay. But is the string is still somewhere in the CPython VM stack? (The
> > result of LOAD_CONST 4 above). Is there a way to pick it up from there?
>
> Maybe using C you could peek into the frame's value stack, but that's
> not exposed to any Python API I know of.  But that still doesn't help
> you, because the value will be removed from the stack before exec() is
> actually called, which means if you go looking for it in code called
> from the exec (e.g. the call event itself), you aren't going to see
> the data.
>
> > At the point that we are stopped the exec action hasn't taken place yet.
>
> That doesn't help if you're using line-level tracing events.  At the
> beginning of the line, the data's not on the call stack yet, and by
> the time you enter the frame of the code being exec'd, it'll be off
> the stack again.
>
> Basically, there is no way to do what you're asking for, short of
> replacing the built-in exec function with your own version.  And it
> still won't help you with stepping through the source of functions
> that are *compiled* using an exec() or compile(), or any other way of
> ending up with dynamically-generated code you want to debug.
>
> (Unless you use something like DecoratorTools to generate it, that is
> -- DecoratorTools has some facilities for caching
> dynamically-generated code so that it works properly with debuggers.
> But that has to be done by the code doing the generation, not the
> debugger.  If the code generator uses DecoratorTools' caching support,
> then any debugger that uses the linecache module will Just Work.  It
> might be nice for  the stdlib should have something like this, but you
> could also potentially fake it by replacing the builtin eval, exec,
> compile, etc. functions w/versions that cache the source.)
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130328/28f96e24/attachment.html>

From xdegaye at gmail.com  Thu Mar 28 12:58:17 2013
From: xdegaye at gmail.com (Xavier de Gaye)
Date: Thu, 28 Mar 2013 12:58:17 +0100
Subject: [Python-Dev] Can we triple quoted string as a comment?
In-Reply-To: <20130326132851.8D4AB250069@webabinitio.net>
References: <CAMpsgwbQbBb0d7OtYVdm9SajGCzK3ZrE=P+0i7gTXtj0Jk_0cQ@mail.gmail.com>
	<7B40444C-FA79-4776-A356-7FC76A5CADC7@gmail.com>
	<20130326132851.8D4AB250069@webabinitio.net>
Message-ID: <CAN4cRFwUv+uvngzA4cjsTQo9_NhmZfYHxTtAWJyULnsfKx5nLw@mail.gmail.com>

On Tue, Mar 26, 2013 at 2:28 PM, R. David Murray wrote:
> On Mon, 25 Mar 2013 18:16:47 -0700, Raymond Hettinger wrote:
>> If you're editing with Emacs, it is really easy to reflow paragraphs
>> and to insert or remove multiline comments each prefixed with #.
>> But with other editors, it can be a PITA and a multiline string is
>> the easiest to maintain and works well when cutting-and-pasting
>> the comments from somewhere else.
>
> Just FYI it is also very easy in vim: gq plus whatever movement prefix
> suits the situation.

And to comment out multiple lines in vim, each prefixed with #, see
":help v_b_I" and ":help v_b_I_example".

Xavier

From brett at python.org  Thu Mar 28 14:42:08 2013
From: brett at python.org (Brett Cannon)
Date: Thu, 28 Mar 2013 09:42:08 -0400
Subject: [Python-Dev] Writing importers and path hooks
In-Reply-To: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
References: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
Message-ID: <CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>

On Wed, Mar 27, 2013 at 6:59 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 27 March 2013 21:19, Bradley M. Froehle <brad.froehle at gmail.com> wrote:
> > I implemented just such a path hook ---- zipimporter plus the magic
> required
> > for C extensions --- as a challenge to myself to learn more about the
> Python
> > import mechanisms.
> >
> > See https://github.com/bfroehle/pydzipimport.
>
> Apologies for hijacking the thread, but it's interesting that you
> implemented your hook like this. I notice that you didn't use any of
> the importlib functionality in doing so. Was there a particular
> reason? I ask because a few days ago, I was writing a very similar
> importer, as I wanted to try a proof of concept importer based on the
> new importlib stuff (which is intended to make writing custom
> importers easier), and I really struggled to get something working.
>

Struggling how? With the finder? The loader? What exactly were you trying
to accomplish and how were you deviating from the standard import system?


>
> It seems to me that the importlib documentation doesn't help much for
> people trying to import path hooks.


There is a bug to clarify the docs to have more geared towards writing new
importers instead of just documenting what's available:
http://bugs.python.org/issue15867


> But it might be just me. Does
> anyone have an example of a simple importlib-based finder/loader?


Define simple. =) I would argue importlib itself is easy enough to read.


> That
> would be a huge help for me. In return for any pointers, I'll look at
> putting together a doc patch to clarify how to use importlib to build
> your own path hooks :-)
>

Do you specifically mean the path hook aspect or the whole package of hook,
finder, and loader?
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130328/61590229/attachment.html>

From theller at ctypes.org  Thu Mar 28 15:44:08 2013
From: theller at ctypes.org (Thomas Heller)
Date: Thu, 28 Mar 2013 15:44:08 +0100
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <loom.20130327T203142-944@post.gmane.org>
References: <loom.20130327T203142-944@post.gmane.org>
Message-ID: <kj1ktg$362$1@ger.gmane.org>

Am 27.03.2013 20:38, schrieb Vinay Sajip:
>> This quote is here to stop GMane complaining that I'm top-posting. Ignore.
>
> I've already posted this to distutils-sig, but thought that it might be of
> interest to readers here as it relates to importing C extensions ...
>
> zipimport is great, but there can be issues importing software that contains
> C extensions. But the new wheel format (PEP 427) may give us a better way of
> importing zip files containing C extensions. Since wheels are .zip files, they
> can sometimes be used to provide functionality without needing to be installed.
> But whereas .zip files contain no convention for indicating compatibility with
> a particular Python, wheels do contain this compatibility information. Thus, it
> is possible to check if a wheel can be directly imported from, and the wheel
> support in distlib allows you to take advantage of this using the mount() and
> unmount() methods. When you mount a wheel, its absolute path name is added to
> sys.path, allowing the Python code in it to be imported. (A DistlibException is
> raised if the wheel isn't compatible with the Python which calls the mount()
> method.)

The zip-file itself could support importing compiled extensions when it 
contains a python-wrapper module that unpacks the .so/.dll file 
somewhere, and finally calls imp.load_dynamic() to import it and replace 
itself.

Thomas



From p.f.moore at gmail.com  Thu Mar 28 16:38:19 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 28 Mar 2013 15:38:19 +0000
Subject: [Python-Dev] Writing importers and path hooks
In-Reply-To: <CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>
References: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
	<CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>
Message-ID: <CACac1F9uV3quw3YZKOQy6jBaoKnbZ5esGDaUe3o7-jh-crfxEA@mail.gmail.com>

On 28 March 2013 13:42, Brett Cannon <brett at python.org> wrote:
>> importer, as I wanted to try a proof of concept importer based on the
>> new importlib stuff (which is intended to make writing custom
>> importers easier), and I really struggled to get something working.
>
> Struggling how? With the finder? The loader? What exactly were you trying to
> accomplish and how were you deviating from the standard import system?

What I was trying to do was to write a path hook that would allow me
to add a string to sys.path which contained a base-64 encoded zipfile
(plus some sort of marker so it could be distinguished from a normal
path entry) and as a result the contents of that embedded zip file
would be available as if I'd added an actual zip file with that
content to sys.path.

I got in a complete mess as I tried to strip out the (essentially
non-interesting) zipfile handling to get to a dummy "do nothing,
everything is valid" type of example. But I don't think I would have
fared much better if I'd stuck to the original full requirement.

>> It seems to me that the importlib documentation doesn't help much for
>> people trying to import path hooks.
>
> There is a bug to clarify the docs to have more geared towards writing new
> importers instead of just documenting what's available:
> http://bugs.python.org/issue15867

Thanks. I'll keep an eye on that.

>> But it might be just me. Does
>> anyone have an example of a simple importlib-based finder/loader?
>
> Define simple. =) I would argue importlib itself is easy enough to read.

:-) Fair point. I guess importlib is not *that* hard to read, but the
only case that implements packages in the filesystem one, and that
also deals with C extensions and other complexities that I don't have
a need for. I'll try again to have a deeper look at it, but I didn't
find it easy to extract the essentials when I looked before.

>> That
>> would be a huge help for me. In return for any pointers, I'll look at
>> putting together a doc patch to clarify how to use importlib to build
>> your own path hooks :-)
>
> Do you specifically mean the path hook aspect or the whole package of hook,
> finder, and loader?

OK, after some more digging, it looks like I misunderstood the process
somewhat. Writing a loader that inherits from *both* FileLoader and
SourceLoader, and then implementing get_data (and module_repr - why do
I need that, couldn't the ABC offer a default implementation?) does
the job for that.

But the finder confuses me. I assume I want a PathEntryFinder and
hence I should implement find_loader(). The documentation on what I
need to return from there is very sparse... In the end I worked out
that for a package, I need to return (MyLoader(modulename,
'foo/__init__.py'), ['foo']) (here, "foo" is my dummy marker for my
example). In essence, PathEntryFinder really has to implement some
form of virtual filesystem mount point, and preserve the standard
filesystem semantics of modules having a filename of .../__init__.py.

So I managed to work out what was needed in the end, but it was a lot
harder than I'd expected. On reflection, getting the finder semantics
right (and in particular the path entry finder semantics) was the hard
bit.

I'm now 100% sure that some cookbook examples would help a lot. I'll
see what I can do.

Thanks,
Paul

From brett at python.org  Thu Mar 28 17:08:29 2013
From: brett at python.org (Brett Cannon)
Date: Thu, 28 Mar 2013 12:08:29 -0400
Subject: [Python-Dev] Writing importers and path hooks
In-Reply-To: <CACac1F9uV3quw3YZKOQy6jBaoKnbZ5esGDaUe3o7-jh-crfxEA@mail.gmail.com>
References: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
	<CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>
	<CACac1F9uV3quw3YZKOQy6jBaoKnbZ5esGDaUe3o7-jh-crfxEA@mail.gmail.com>
Message-ID: <CAP1=2W5yu3azH7bH4ZRH4esdRfXizKEkvUKaqEuy5oaBumOJWw@mail.gmail.com>

On Thu, Mar 28, 2013 at 11:38 AM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 28 March 2013 13:42, Brett Cannon <brett at python.org> wrote:
> >> importer, as I wanted to try a proof of concept importer based on the
> >> new importlib stuff (which is intended to make writing custom
> >> importers easier), and I really struggled to get something working.
> >
> > Struggling how? With the finder? The loader? What exactly were you
> trying to
> > accomplish and how were you deviating from the standard import system?
>
> What I was trying to do was to write a path hook that would allow me
> to add a string to sys.path which contained a base-64 encoded zipfile
> (plus some sort of marker so it could be distinguished from a normal
> path entry) and as a result the contents of that embedded zip file
> would be available as if I'd added an actual zip file with that
> content to sys.path.
>
> I got in a complete mess as I tried to strip out the (essentially
> non-interesting) zipfile handling to get to a dummy "do nothing,
> everything is valid" type of example. But I don't think I would have
> fared much better if I'd stuck to the original full requirement.
>
> >> It seems to me that the importlib documentation doesn't help much for
> >> people trying to import path hooks.
> >
> > There is a bug to clarify the docs to have more geared towards writing
> new
> > importers instead of just documenting what's available:
> > http://bugs.python.org/issue15867
>
> Thanks. I'll keep an eye on that.
>
> >> But it might be just me. Does
> >> anyone have an example of a simple importlib-based finder/loader?
> >
> > Define simple. =) I would argue importlib itself is easy enough to read.
>
> :-) Fair point. I guess importlib is not *that* hard to read, but the
> only case that implements packages in the filesystem one, and that
> also deals with C extensions and other complexities that I don't have
> a need for. I'll try again to have a deeper look at it, but I didn't
> find it easy to extract the essentials when I looked before.
>
> >> That
> >> would be a huge help for me. In return for any pointers, I'll look at
> >> putting together a doc patch to clarify how to use importlib to build
> >> your own path hooks :-)
> >
> > Do you specifically mean the path hook aspect or the whole package of
> hook,
> > finder, and loader?
>
> OK, after some more digging, it looks like I misunderstood the process
> somewhat. Writing a loader that inherits from *both* FileLoader and
> SourceLoader,


You only need SourceLoader since you are dealing with Python source. You
don't need FileLoader since you are not reading from disk but an in-memory
zipfile.


> and then implementing get_data (and module_repr - why do
> I need that, couldn't the ABC offer a default implementation?)


http://bugs.python.org/issue17093 and http://bugs.python.org/issue17566


> does
> the job for that.
>

You should be implementing get_data, get_filename, and path_stats for
SourceLoader.


>
> But the finder confuses me. I assume I want a PathEntryFinder and
> hence I should implement find_loader().


Yes since you are pulling from sys.path.


> The documentation on what I
> need to return from there is very sparse... In the end I worked out
> that for a package, I need to return (MyLoader(modulename,
> 'foo/__init__.py'), ['foo']) (here, "foo" is my dummy marker for my
> example).


The second argument should just be None: "An empty list can be used for
portion to signify the loader is not part of a [namespace] package".
Unfortunately a key word is missing in that sentence.
http://bugs.python.org/issue17567


> In essence, PathEntryFinder really has to implement some
> form of virtual filesystem mount point, and preserve the standard
> filesystem semantics of modules having a filename of .../__init__.py.
>

Well, if your zip file decided to create itself with a different file
extension then it wouldn't be required, but then other people's code might
break if they don't respect module abstractions (i.e. looking at
__package__/__name__ or __path__ to see if something is a package).


>
> So I managed to work out what was needed in the end, but it was a lot
> harder than I'd expected. On reflection, getting the finder semantics
> right (and in particular the path entry finder semantics) was the hard
> bit.
>

Yep, that bit has had the least API tweaks as most people don't muck with
finders but with loaders.


>
> I'm now 100% sure that some cookbook examples would help a lot. I'll
> see what I can do.
>

I plan on writing a pure Python zip importer for Python 3.4 which should be
fairly minimal and work out as a good example chunk of code.  And no one
need bother writing it as I'm going to do it myself regardless to make sure
I plug any missing holes in the API. If you really want something to try
for fun go for a sqlite3-backed setup (don't see it going in the stdlib but
it would be a project to have).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130328/7463eeea/attachment.html>

From brett at python.org  Thu Mar 28 17:09:26 2013
From: brett at python.org (Brett Cannon)
Date: Thu, 28 Mar 2013 12:09:26 -0400
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <kj1ktg$362$1@ger.gmane.org>
References: <loom.20130327T203142-944@post.gmane.org>
	<kj1ktg$362$1@ger.gmane.org>
Message-ID: <CAP1=2W5xxmDNi=5y=f68DKZnX60MZYxSi5+hC+yRKT5fvFauEA@mail.gmail.com>

On Thu, Mar 28, 2013 at 10:44 AM, Thomas Heller <theller at ctypes.org> wrote:

> Am 27.03.2013 20:38, schrieb Vinay Sajip:
>
>  This quote is here to stop GMane complaining that I'm top-posting. Ignore.
>>>
>>
>> I've already posted this to distutils-sig, but thought that it might be of
>> interest to readers here as it relates to importing C extensions ...
>>
>> zipimport is great, but there can be issues importing software that
>> contains
>> C extensions. But the new wheel format (PEP 427) may give us a better way
>> of
>> importing zip files containing C extensions. Since wheels are .zip files,
>> they
>> can sometimes be used to provide functionality without needing to be
>> installed.
>> But whereas .zip files contain no convention for indicating compatibility
>> with
>> a particular Python, wheels do contain this compatibility information.
>> Thus, it
>> is possible to check if a wheel can be directly imported from, and the
>> wheel
>> support in distlib allows you to take advantage of this using the mount()
>> and
>> unmount() methods. When you mount a wheel, its absolute path name is
>> added to
>> sys.path, allowing the Python code in it to be imported. (A
>> DistlibException is
>> raised if the wheel isn't compatible with the Python which calls the
>> mount()
>> method.)
>>
>
> The zip-file itself could support importing compiled extensions when it
> contains a python-wrapper module that unpacks the .so/.dll file somewhere,
> and finally calls imp.load_dynamic() to import it and replace itself.


Which must be done carefully to prevent a security issue. It shouldn't be
unzipped anywhere but into a directory only writable by the process.
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130328/2d110e62/attachment.html>

From p.f.moore at gmail.com  Thu Mar 28 17:33:23 2013
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 28 Mar 2013 16:33:23 +0000
Subject: [Python-Dev] Writing importers and path hooks
In-Reply-To: <CAP1=2W5yu3azH7bH4ZRH4esdRfXizKEkvUKaqEuy5oaBumOJWw@mail.gmail.com>
References: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
	<CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>
	<CACac1F9uV3quw3YZKOQy6jBaoKnbZ5esGDaUe3o7-jh-crfxEA@mail.gmail.com>
	<CAP1=2W5yu3azH7bH4ZRH4esdRfXizKEkvUKaqEuy5oaBumOJWw@mail.gmail.com>
Message-ID: <CACac1F_kjuYjwogS5akbW4RW1HD3CCsWCfaVDBVJsca5Kbb1vA@mail.gmail.com>

On 28 March 2013 16:08, Brett Cannon <brett at python.org> wrote:
> You only need SourceLoader since you are dealing with Python source. You
> don't need FileLoader since you are not reading from disk but an in-memory
> zipfile.
>
> You should be implementing get_data, get_filename, and path_stats for
> SourceLoader.

OK, cool. That helps a lot.

The biggest gap here is that I don't think that anywhere has a good
explanation of the required semantics of get_filename - particularly
where we're not actually dealing with real filenames. My initial stab
at this would be:

A module name is a dot-separated list of parts.
A filename is an arbitrary token that can be used with get_data to get
the module content. However, the following rules should be followed:
- Filenames should be made up of parts separated by the OS path separator.
- For packages, the final section of the filename *must* be
__init__.py if the standard package detection is being used.
- The initial part of the filename needs to match your path entry if
submodule lookups are going to work sanely

In practice, you need to implement filenames as if your finder is
managing a virtual filesystem mounted at your sys.path entry, with
module->filename semantics being the usual subdirectory layout. And
packages have a basename of __init__.py.

I'd like to know how to implement packages without the artificial
__init__.py (something like a sqlite database can attach content and
an "is_package" flag to the same entry). But that's advanced usage,
and I can probably hack around until I work out how to do that now.

>> The documentation on what I
>> need to return from there is very sparse... In the end I worked out
>> that for a package, I need to return (MyLoader(modulename,
>> 'foo/__init__.py'), ['foo']) (here, "foo" is my dummy marker for my
>> example).
>
> The second argument should just be None: "An empty list can be used for
> portion to signify the loader is not part of a [namespace] package".
> Unfortunately a key word is missing in that sentence.
> http://bugs.python.org/issue17567

Ha. Yes, that makes a lot of difference :-) Did you mean None or [], by the way?

>> In essence, PathEntryFinder really has to implement some
>> form of virtual filesystem mount point, and preserve the standard
>> filesystem semantics of modules having a filename of .../__init__.py.
>
> Well, if your zip file decided to create itself with a different file
> extension then it wouldn't be required, but then other people's code might
> break if they don't respect module abstractions (i.e. looking at
> __package__/__name__ or __path__ to see if something is a package).

I'm not quite sure what you mean by this, but I take your point about
making sure to break people's expectations as little as possible...

>> So I managed to work out what was needed in the end, but it was a lot
>> harder than I'd expected. On reflection, getting the finder semantics
>> right (and in particular the path entry finder semantics) was the hard
>> bit.
>
> Yep, that bit has had the least API tweaks as most people don't muck with
> finders but with loaders.

Hmm. I'm not sure how you can ever write a loader without needing to
write an associated finder. The existing finders wouldn't return your
loader, surely?

>> I'm now 100% sure that some cookbook examples would help a lot. I'll
>> see what I can do.
>
> I plan on writing a pure Python zip importer for Python 3.4 which should be
> fairly minimal and work out as a good example chunk of code.  And no one
> need bother writing it as I'm going to do it myself regardless to make sure
> I plug any missing holes in the API. If you really want something to try for
> fun go for a sqlite3-backed setup (don't see it going in the stdlib but it
> would be a project to have).

I'm pretty sure I'll write a zip importer first - it feels like one of
those essential but largely useless exercises that people have to
start with - a bit like scales on the piano :-) But I'd be interested
in trying a sqlite importer as well. I might well see how I go with
that.

Thanks for the help with this.
Paul

From christian at python.org  Thu Mar 28 18:12:05 2013
From: christian at python.org (Christian Heimes)
Date: Thu, 28 Mar 2013 18:12:05 +0100
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <CAP1=2W5xxmDNi=5y=f68DKZnX60MZYxSi5+hC+yRKT5fvFauEA@mail.gmail.com>
References: <loom.20130327T203142-944@post.gmane.org>
	<kj1ktg$362$1@ger.gmane.org>
	<CAP1=2W5xxmDNi=5y=f68DKZnX60MZYxSi5+hC+yRKT5fvFauEA@mail.gmail.com>
Message-ID: <515479E5.1090805@python.org>

Am 28.03.2013 17:09, schrieb Brett Cannon:
> Which must be done carefully to prevent a security issue. It shouldn't
> be unzipped anywhere but into a directory only writable by the process. 

Cleanup is going to be tricky or even impossible. Windows locks loaded
DLLs and therefore prevents their removal. It's possible to unload DLLs
but I don't know the implications.

From dan at woz.io  Thu Mar 28 18:16:47 2013
From: dan at woz.io (Daniel Wozniak)
Date: Thu, 28 Mar 2013 10:16:47 -0700
Subject: [Python-Dev] noob contributions to unit tests
In-Reply-To: <20130327022422.D7ACA250BCA@webabinitio.net>
References: <CAOFf2a1Uf_hP-xHuHbkrXdEk6sYSiqTRiT3nbvo8LZk3AvJGJQ@mail.gmail.com>
	<CAK5idxS_Ze9bTGsdznTjMUr3rQzbLafeZ_ge9R+KjyeC+TMV0A@mail.gmail.com>
	<20130327022422.D7ACA250BCA@webabinitio.net>
Message-ID: <51547AFF.3070906@woz.io>

Sean,

During the PyCon sprints I was helping work on unittests in urllib. I 
think as it stands right now urllib/error.py and urllib/parse.py are at 
100% line coverage. I have some additions to urllib/request.py which I 
have yet to submit a patch for (anything above line 700 is covered thus 
far) and I noticed there is a large chunk of code there which has line 
coverage when running tests on OSX. There is a note to refactor that to 
run on all platforms since there is nothing OSX specific in those tests. 
I believe if those tests were running on all platforms it would 
drastically increase line coverage for request.py, assuming your not on 
OSX of coarse. I have not looked at response.py or robotparser.py yet.

Just wanted to give you a little brain dump in case it can save you some 
time.

~Daniel


From brett at python.org  Thu Mar 28 18:39:03 2013
From: brett at python.org (Brett Cannon)
Date: Thu, 28 Mar 2013 13:39:03 -0400
Subject: [Python-Dev] Writing importers and path hooks
In-Reply-To: <CACac1F_kjuYjwogS5akbW4RW1HD3CCsWCfaVDBVJsca5Kbb1vA@mail.gmail.com>
References: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
	<CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>
	<CACac1F9uV3quw3YZKOQy6jBaoKnbZ5esGDaUe3o7-jh-crfxEA@mail.gmail.com>
	<CAP1=2W5yu3azH7bH4ZRH4esdRfXizKEkvUKaqEuy5oaBumOJWw@mail.gmail.com>
	<CACac1F_kjuYjwogS5akbW4RW1HD3CCsWCfaVDBVJsca5Kbb1vA@mail.gmail.com>
Message-ID: <CAP1=2W6M97AsA6w1G7ibLhLrbGC4csAmpfsQzSeO10m-wQn4Sw@mail.gmail.com>

On Thu, Mar 28, 2013 at 12:33 PM, Paul Moore <p.f.moore at gmail.com> wrote:

> On 28 March 2013 16:08, Brett Cannon <brett at python.org> wrote:
> > You only need SourceLoader since you are dealing with Python source. You
> > don't need FileLoader since you are not reading from disk but an
> in-memory
> > zipfile.
> >
> > You should be implementing get_data, get_filename, and path_stats for
> > SourceLoader.
>
> OK, cool. That helps a lot.
>
> The biggest gap here is that I don't think that anywhere has a good
> explanation of the required semantics of get_filename - particularly
> where we're not actually dealing with real filenames.


It's because there aren't any. =) This is the first time alternative
storage mechanisms are really easily viable without massive amounts of
work, so no one has figured this out. The real question is how code out in
the wild would react if you did something like /path/to/sqlite3:pkg.mod
which is very much not a file path.


> My initial stab
> at this would be:
>
> A module name is a dot-separated list of parts.
> A filename is an arbitrary token that can be used with get_data to get
> the module content. However, the following rules should be followed:
> - Filenames should be made up of parts separated by the OS path separator.
>

And why is that? A database doesn't need those separators as the module
name would just be the primary key.


> - For packages, the final section of the filename *must* be
> __init__.py if the standard package detection is being used.
>

Once again, why? A column in a database that is nothing more than a package
flag would solve this as well, negating the need for this. The whole point
of is_package() on loaders is to get away from this reliance on __file__
having any meaning beyond "this is the string that represents where this
module's code was loaded from".


> - The initial part of the filename needs to match your path entry if
> submodule lookups are going to work sanely
>

When applicable that's fine.


>
> In practice, you need to implement filenames as if your finder is
> managing a virtual filesystem mounted at your sys.path entry, with
> module->filename semantics being the usual subdirectory layout. And
> packages have a basename of __init__.py.
>

That's one way of doing it, but it does very much tie imports to files and
it doesn't generalize the concept to places where file paths simply do not
need to apply.


>
> I'd like to know how to implement packages without the artificial
> __init__.py (something like a sqlite database can attach content and
> an "is_package" flag to the same entry). But that's advanced usage,
> and I can probably hack around until I work out how to do that now.
>

Define is_package(). I personally want to change the API somehow so you ask
for what __path__ should be set to. Unfortunately without going down the
"False means not a package, everything else means it is and what is
returned should be set on __path__" is a bit hairy and not
backwards-compatible unless you require a list that always evaluates to
True for packages.


>
> >> The documentation on what I
> >> need to return from there is very sparse... In the end I worked out
> >> that for a package, I need to return (MyLoader(modulename,
> >> 'foo/__init__.py'), ['foo']) (here, "foo" is my dummy marker for my
> >> example).
> >
> > The second argument should just be None: "An empty list can be used for
> > portion to signify the loader is not part of a [namespace] package".
> > Unfortunately a key word is missing in that sentence.
> > http://bugs.python.org/issue17567
>
> Ha. Yes, that makes a lot of difference :-) Did you mean None or [], by
> the way?
>

Empty list. You can check the code to see if it would work with None, but a
list is expected to be used so an empty list is more consistent and still
false.


>
> >> In essence, PathEntryFinder really has to implement some
> >> form of virtual filesystem mount point, and preserve the standard
> >> filesystem semantics of modules having a filename of .../__init__.py.
> >
> > Well, if your zip file decided to create itself with a different file
> > extension then it wouldn't be required, but then other people's code
> might
> > break if they don't respect module abstractions (i.e. looking at
> > __package__/__name__ or __path__ to see if something is a package).
>
> I'm not quite sure what you mean by this, but I take your point about
> making sure to break people's expectations as little as possible...
>

To tell if a module is a package, you should do either ``if mod.__name__ ==
mod.__package__`` or ``if hasattr(mod, '__path__')``.


>
> >> So I managed to work out what was needed in the end, but it was a lot
> >> harder than I'd expected. On reflection, getting the finder semantics
> >> right (and in particular the path entry finder semantics) was the hard
> >> bit.
> >
> > Yep, that bit has had the least API tweaks as most people don't muck with
> > finders but with loaders.
>
> Hmm. I'm not sure how you can ever write a loader without needing to
> write an associated finder. The existing finders wouldn't return your
> loader, surely?
>

If you are not changing the storage mechanism you don't need a new finder;
what importlib provides works fine. So if you are, for instance, only
providing a loader which does an AST optimization pass you only need a new
loader. Or if you use a DSL that you compile into Python code then you only
need a new loader.


>
> >> I'm now 100% sure that some cookbook examples would help a lot. I'll
> >> see what I can do.
> >
> > I plan on writing a pure Python zip importer for Python 3.4 which should
> be
> > fairly minimal and work out as a good example chunk of code.  And no one
> > need bother writing it as I'm going to do it myself regardless to make
> sure
> > I plug any missing holes in the API. If you really want something to try
> for
> > fun go for a sqlite3-backed setup (don't see it going in the stdlib but
> it
> > would be a project to have).
>
> I'm pretty sure I'll write a zip importer first - it feels like one of
> those essential but largely useless exercises that people have to
> start with - a bit like scales on the piano :-) But I'd be interested
> in trying a sqlite importer as well. I might well see how I go with
> that.
>

The sqlite3 one is interesting as it does not whatsoever require file paths
to operate; you can easily define a schema specific to source code and
bytecode and really go db-specific and have the loader work from that
(would also make finder lookups dead-simple). Otherwise you will end up
writing a schema for a virtual filesystem which would also work but would
show that people are not respecting abstractions on modules (or that the
API has gaps which need filling in).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130328/6f7e2607/attachment.html>

From walter at livinglogic.de  Thu Mar 28 19:36:05 2013
From: walter at livinglogic.de (=?windows-1252?Q?Walter_D=F6rwald?=)
Date: Thu, 28 Mar 2013 19:36:05 +0100
Subject: [Python-Dev] noob contributions to unit tests
In-Reply-To: <20130327022422.D7ACA250BCA@webabinitio.net>
References: <CAOFf2a1Uf_hP-xHuHbkrXdEk6sYSiqTRiT3nbvo8LZk3AvJGJQ@mail.gmail.com>
	<CAK5idxS_Ze9bTGsdznTjMUr3rQzbLafeZ_ge9R+KjyeC+TMV0A@mail.gmail.com>
	<20130327022422.D7ACA250BCA@webabinitio.net>
Message-ID: <DB7C8ECA-A011-4521-A79A-56F073F57ED4@livinglogic.de>


Am 27.03.2013 um 03:24 schrieb R. David Murray <rdmurray at bitdance.com>:

> On Tue, 26 Mar 2013 16:59:06 -0700, Maciej Fijalkowski <fijall at gmail.com> wrote:
>> On Tue, Mar 26, 2013 at 4:49 PM, Sean Felipe Wolfe <ether.joe at gmail.com> wrote:
>>> Hey everybody how are you all :)
>>> 
>>> I am an intermediate-level python coder looking to get help out. I've
>>> been reading over the dev guide about helping increase test coverage
>>> -->
>>> http://docs.python.org/devguide/coverage.html
>>> 
>>> And also the third-party code coverage referenced in the devguide page:
>>> http://coverage.livinglogic.de/
>>> 
>>> I'm seeing that according to the coverage tool, two of my favorite
>>> libraries, urllib/urllib2, have no unit tests? Is that correct or am I
>>> reading it wrong?
>>> 
>>> If that's correct it seems like a great place perhaps for me to cut my
>>> teeth and I would be excited to learn and help out here.
>>> 
>>> And of course any thoughts or advice for an aspiring Python
>>> contributor would be appreciated. Of course the dev guide gives me
>>> plenty of good info.
>>> 
>>> Thanks!
>> 
>> That looks like an error in the coverage report, there are certainly
>> urllib and urllib2 tests in test/test_urllib*
> 
> The devguide contains instructions for running coverage yourself,
> and if I recall correctly the 'fullcoverage' recipe does a better
> job than what runs at coverage.livinglogic.de.

The job that produces that output has been broken for some time now, and I haven't found the time to look into it. If someone wants to try, here's the code:

   https://pypi.python.org/pypi/pycoco/0.7.2

> [?]

Servus,
   Walter


From pje at telecommunity.com  Thu Mar 28 20:44:10 2013
From: pje at telecommunity.com (PJ Eby)
Date: Thu, 28 Mar 2013 15:44:10 -0400
Subject: [Python-Dev] Can I introspect/reflect to get arguments exec()?
In-Reply-To: <CANCp2gZVkRxiFsYVpRxcpf3H0szZTFPX3Vv-t-jCfabGjndvNg@mail.gmail.com>
References: <CANCp2gbahXra4UqjG36FR5GCTC4a18Qq2q2zUzX5Z2eC93o7Wg@mail.gmail.com>
	<CAPZV6o9Vczz3k5AHwZKs_+46kR23FKrsGqhuK93N6JGeyjeY0Q@mail.gmail.com>
	<CANCp2gYB8w0VyzZ4RVTQ6VQjTx0-xDPGxojpHNoyRqV9n_4y6A@mail.gmail.com>
	<CALeMXf7YoC6JYPc_PkvWrg38uMdJvhJSKJGcsxzzpCcYPd1uSA@mail.gmail.com>
	<CANCp2gZVkRxiFsYVpRxcpf3H0szZTFPX3Vv-t-jCfabGjndvNg@mail.gmail.com>
Message-ID: <CALeMXf44e3Y59vgQ3hkcNycp+si-kM_Qjo0GhtqV+zn1XCvEKQ@mail.gmail.com>

On Thu, Mar 28, 2013 at 6:43 AM, Rocky Bernstein <rocky at gnu.org> wrote:
> Of course the debugger uses sys.settrace too, so the evil-ness of that is
> definitely not a concern. But possibly I need to make sure that since the
> DecoratorTools and the debugger both hook into trace hooks they play nice
> together and fire in the right order.

DecoratorTools' trace hooking is unrelated to its linecache
functionality.  All you need from it is the cache_source() function;
you can pretty much ignore everything else for your purposes.  You'll
just need to give it a phony filename to work with, and the associated
string.

From ezio.melotti at gmail.com  Thu Mar 28 23:56:46 2013
From: ezio.melotti at gmail.com (Ezio Melotti)
Date: Fri, 29 Mar 2013 00:56:46 +0200
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Issue 17538:
	Document XML vulnerabilties
In-Reply-To: <3ZZz1v2N95zQBT@mail.python.org>
References: <3ZZz1v2N95zQBT@mail.python.org>
Message-ID: <CACBhJdH_x+B7i25K77AqsM3wFktjG=qqKAqgcnvVe3i7to=sNw@mail.gmail.com>

Hi,

On Tue, Mar 26, 2013 at 6:53 PM, christian.heimes
<python-checkins at python.org> wrote:
> http://hg.python.org/cpython/rev/e87364449954
> changeset:   82973:e87364449954
> branch:      2.7
> parent:      82963:d321885ff8f3
> user:        Christian Heimes <christian at cheimes.de>
> date:        Tue Mar 26 17:53:05 2013 +0100
> summary:
>   Issue 17538: Document XML vulnerabilties
>
> [...]
>
> diff --git a/Doc/library/xml.rst b/Doc/library/xml.rst
> new file mode 100644
> --- /dev/null
> +++ b/Doc/library/xml.rst
> @@ -0,0 +1,131 @@
> +.. _xml:
> +
> +XML Processing Modules
> +======================
> +
> +.. module:: xml
> +   :synopsis: Package containing XML processing modules
> +.. sectionauthor:: Christian Heimes <christian at python.org>
> +.. sectionauthor:: Georg Brandl <georg at python.org>
> +
> +
> +Python's interfaces for processing XML are grouped in the ``xml`` package.
> +
> +.. warning::
> +
> +   The XML modules are not secure against erroneous or maliciously
> +   constructed data.  If you need to parse untrusted or unauthenticated data see
> +   :ref:`xml-vulnerabilities`.
> +
> +It is important to note that modules in the :mod:`xml` package require that
> +there be at least one SAX-compliant XML parser available. The Expat parser is
> +included with Python, so the :mod:`xml.parsers.expat` module will always be
> +available.
> +
> +The documentation for the :mod:`xml.dom` and :mod:`xml.sax` packages are the
> +definition of the Python bindings for the DOM and SAX interfaces.
> +
> +The XML handling submodules are:
> +
> +* :mod:`xml.etree.ElementTree`: the ElementTree API, a simple and lightweight

Something is missing here ^

> +
> +..
> +
> +* :mod:`xml.dom`: the DOM API definition
> +* :mod:`xml.dom.minidom`: a lightweight DOM implementation
> +* :mod:`xml.dom.pulldom`: support for building partial DOM trees
> +
> +..
> +
> +* :mod:`xml.sax`: SAX2 base classes and convenience functions
> +* :mod:`xml.parsers.expat`: the Expat parser binding
> +
> +
> +.. _xml-vulnerabilities:
> +
> [...]
> +
> +defused packages
> +----------------
> +
> +`defusedxml`_ is a pure Python package with modified subclasses of all stdlib
> +XML parsers that prevent any potentially malicious operation. The courses of
> +action are recommended for any server code that parses untrusted XML data.

This last sentence doesn't make much sense to me.  Is it even correct?

> The
> +package also ships with example exploits and an extended documentation on more
> +XML exploits like xpath injection.
> +
> +`defusedexpat`_ provides a modified libexpat and patched replacment

s/replacment/replacement/

> +:mod:`pyexpat` extension module with countermeasures against entity expansion
> +DoS attacks. Defusedexpat still allows a sane and configurable amount of entity
> +expansions. The modifications will be merged into future releases of Python.
> +
> +The workarounds and modifications are not included in patch releases as they
> +break backward compatibility. After all inline DTD and entity expansion are
> +well-definied XML features.

s/definied/defined/

> +
> +
> +.. _defusedxml: <https://pypi.python.org/pypi/defusedxml/>
> +.. _defusedexpat: <https://pypi.python.org/pypi/defusedexpat/>
> +.. _Billion Laughs: http://en.wikipedia.org/wiki/Billion_laughs
> +.. _ZIP bomb: http://en.wikipedia.org/wiki/Zip_bomb
> +.. _DTD: http://en.wikipedia.org/wiki/Document_Type_Definition
>  [...]

Best Regards,
Ezio Melotti

From greg at krypto.org  Fri Mar 29 02:06:35 2013
From: greg at krypto.org (Gregory P. Smith)
Date: Thu, 28 Mar 2013 18:06:35 -0700
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <CAP1=2W5xxmDNi=5y=f68DKZnX60MZYxSi5+hC+yRKT5fvFauEA@mail.gmail.com>
References: <loom.20130327T203142-944@post.gmane.org>
	<kj1ktg$362$1@ger.gmane.org>
	<CAP1=2W5xxmDNi=5y=f68DKZnX60MZYxSi5+hC+yRKT5fvFauEA@mail.gmail.com>
Message-ID: <CAGE7PN+HrGiiQSpcsqetoErDGh_3btkO7N0GctY-tbLvHYY5Jg@mail.gmail.com>

On Thu, Mar 28, 2013 at 9:09 AM, Brett Cannon <brett at python.org> wrote:

>
>
>
> On Thu, Mar 28, 2013 at 10:44 AM, Thomas Heller <theller at ctypes.org>wrote:
>
>> Am 27.03.2013 20:38, schrieb Vinay Sajip:
>>
>>  This quote is here to stop GMane complaining that I'm top-posting.
>>>> Ignore.
>>>>
>>>
>>> I've already posted this to distutils-sig, but thought that it might be
>>> of
>>> interest to readers here as it relates to importing C extensions ...
>>>
>>> zipimport is great, but there can be issues importing software that
>>> contains
>>> C extensions. But the new wheel format (PEP 427) may give us a better
>>> way of
>>> importing zip files containing C extensions. Since wheels are .zip
>>> files, they
>>> can sometimes be used to provide functionality without needing to be
>>> installed.
>>> But whereas .zip files contain no convention for indicating
>>> compatibility with
>>> a particular Python, wheels do contain this compatibility information.
>>> Thus, it
>>> is possible to check if a wheel can be directly imported from, and the
>>> wheel
>>> support in distlib allows you to take advantage of this using the
>>> mount() and
>>> unmount() methods. When you mount a wheel, its absolute path name is
>>> added to
>>> sys.path, allowing the Python code in it to be imported. (A
>>> DistlibException is
>>> raised if the wheel isn't compatible with the Python which calls the
>>> mount()
>>> method.)
>>>
>>
>> The zip-file itself could support importing compiled extensions when it
>> contains a python-wrapper module that unpacks the .so/.dll file somewhere,
>> and finally calls imp.load_dynamic() to import it and replace itself.
>
>
> Which must be done carefully to prevent a security issue. It shouldn't be
> unzipped anywhere but into a directory only writable by the process.
>
>
Once http://sourceware.org/bugzilla/show_bug.cgi?id=11767 is implemented
and available in libc, no extraction of .so's should be needed (they will
likely need to be stored uncompressed in the .zip file for that though).
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130328/1d252a7d/attachment.html>

From theller at ctypes.org  Fri Mar 29 13:00:29 2013
From: theller at ctypes.org (Thomas Heller)
Date: Fri, 29 Mar 2013 13:00:29 +0100
Subject: [Python-Dev] Safely importing zip files with C extensions
In-Reply-To: <CAGE7PN+HrGiiQSpcsqetoErDGh_3btkO7N0GctY-tbLvHYY5Jg@mail.gmail.com>
References: <loom.20130327T203142-944@post.gmane.org>
	<kj1ktg$362$1@ger.gmane.org>
	<CAP1=2W5xxmDNi=5y=f68DKZnX60MZYxSi5+hC+yRKT5fvFauEA@mail.gmail.com>
	<CAGE7PN+HrGiiQSpcsqetoErDGh_3btkO7N0GctY-tbLvHYY5Jg@mail.gmail.com>
Message-ID: <kj3vmk$ggi$1@ger.gmane.org>

Am 29.03.2013 02:06, schrieb Gregory P. Smith:
>
> On Thu, Mar 28, 2013 at 9:09 AM, Brett Cannon <brett at python.org
> <mailto:brett at python.org>> wrote:
>
>     On Thu, Mar 28, 2013 at 10:44 AM, Thomas Heller <theller at ctypes.org
>     <mailto:theller at ctypes.org>> wrote:
>
>         The zip-file itself could support importing compiled extensions
>         when it contains a python-wrapper module that unpacks the
>         .so/.dll file somewhere, and finally calls imp.load_dynamic() to
>         import it and replace itself.
>
>
>     Which must be done carefully to prevent a security issue. It
>     shouldn't be unzipped anywhere but into a directory only writable by
>     the process.
>
>
> Once http://sourceware.org/bugzilla/show_bug.cgi?id=11767 is implemented
> and available in libc, no extraction of .so's should be needed (they
> will likely need to be stored uncompressed in the .zip file for that
> though).

For windows there is already code that does it:

http://www.py2exe.org/index.cgi/Hacks/ZipExtImporter

This page is not up-to-date, but it describes the idea and the
implementation.  The code currently is 32-bit only and for Python 2
but that probably can be fixed.

It is based on Joachim Bauch's MemoryModule:
https://github.com/fancycode/MemoryModule


Thomas


From status at bugs.python.org  Fri Mar 29 18:07:31 2013
From: status at bugs.python.org (Python tracker)
Date: Fri, 29 Mar 2013 18:07:31 +0100 (CET)
Subject: [Python-Dev] Summary of Python tracker Issues
Message-ID: <20130329170731.7600156915@psf.upfronthosting.co.za>


ACTIVITY SUMMARY (2013-03-22 - 2013-03-29)
Python tracker at http://bugs.python.org/

To view or respond to any of the issues listed below, click on the issue.
Do NOT respond to this message.

Issues counts and deltas:
  open    3887 (-21)
  closed 25461 (+72)
  total  29348 (+51)

Open issues with patches: 1724 


Issues opened (46)
==================

#6671: webbrowser doesn't respect xfce default browser
http://bugs.python.org/issue6671  reopened by eric.araujo

#17295: __slots__ on PyVarObject subclass
http://bugs.python.org/issue17295  reopened by haypo

#17523: Additional tests for the os module.
http://bugs.python.org/issue17523  opened by willweaver

#17525: os.getcwd() fails on cifs share
http://bugs.python.org/issue17525  opened by dcuddihy

#17526: inspect.findsource raises undocumented error for code objects 
http://bugs.python.org/issue17526  opened by Nils.Bruin

#17527: PATCH as valid request method in wsgiref.validator
http://bugs.python.org/issue17527  opened by lsbardel

#17528: Implement dumps/loads for lru_cache
http://bugs.python.org/issue17528  opened by frafra

#17529: fix os.sendfile() documentation regarding the type of file des
http://bugs.python.org/issue17529  opened by neologix

#17530: pprint could use line continuation for long bytes literals
http://bugs.python.org/issue17530  opened by pitrou

#17532: IDLE: Always include "Options" menu on MacOSX
http://bugs.python.org/issue17532  opened by roger.serwy

#17533: test_xpickle fails with "cannot import name precisionbigmemtes
http://bugs.python.org/issue17533  opened by ned.deily

#17534: unittest keeps references to test cases alive
http://bugs.python.org/issue17534  opened by ezio.melotti

#17535: IDLE: Add an option to show line numbers along the left side o
http://bugs.python.org/issue17535  opened by Todd.Rovito

#17536: update browser list with additional browser names
http://bugs.python.org/issue17536  opened by doko

#17537: csv.DictReader should fail if >1 column has the same name
http://bugs.python.org/issue17537  opened by doko

#17538: Document XML Vulnerabilties
http://bugs.python.org/issue17538  opened by dstufft

#17539: Use the builtins module in the unittest.mock.patch example
http://bugs.python.org/issue17539  opened by berker.peksag

#17540: logging formatter support 'style' key in dictionary config
http://bugs.python.org/issue17540  opened by monson

#17544: regex code re-raises exceptions on success
http://bugs.python.org/issue17544  opened by Zden??k.Pavlas

#17545: os.listdir and os.path.join inconsistent on empty path
http://bugs.python.org/issue17545  opened by babou

#17546: Document the circumstances where the locals() dict gets	update
http://bugs.python.org/issue17546  opened by techtonik

#17547: "checking whether gcc supports ParseTuple __format__... " erro
http://bugs.python.org/issue17547  opened by dmalcolm

#17548: unittest.mock: test_create_autospec_unbound_methods is skipped
http://bugs.python.org/issue17548  opened by haypo

#17549: Some exceptions not highlighted in exceptions documentation.
http://bugs.python.org/issue17549  opened by Ramchandra Apte

#17551: Windows - accessing drive with nothing mounted forces user int
http://bugs.python.org/issue17551  opened by bobjalex

#17552: socket.sendfile()
http://bugs.python.org/issue17552  opened by giampaolo.rodola

#17553: Note that distutils??? bdist_rpm command is not used to build 
http://bugs.python.org/issue17553  opened by Sean.Carolan

#17554: Compact output for regrtest
http://bugs.python.org/issue17554  opened by ezio.melotti

#17555: Creating new processes after importing multiprocessing.manager
http://bugs.python.org/issue17555  opened by Marc.Br??nink

#17557: test_getgroups of test_posix can fail on OS X 10.8 if more tha
http://bugs.python.org/issue17557  opened by ned.deily

#17558: gdb debugging python frames in optimised interpreters
http://bugs.python.org/issue17558  opened by mcobden

#17560: problem using multiprocessing with really big objects?
http://bugs.python.org/issue17560  opened by mrjbq7

#17561: Add socket.create_server_sock() convenience function
http://bugs.python.org/issue17561  opened by giampaolo.rodola

#17563: Excessive resizing of dicts when used as a cache
http://bugs.python.org/issue17563  opened by Mark.Shannon

#17564: test_urllib2_localnet fails
http://bugs.python.org/issue17564  opened by Mark.Shannon

#17565: segfaults during serialization
http://bugs.python.org/issue17565  opened by eddiewrc

#17566: Document that importlib.abc.Loader.module_repr is abstract and
http://bugs.python.org/issue17566  opened by brett.cannon

#17567: Clarify importlib.abc.PathEntryFinder.find_loader() docs
http://bugs.python.org/issue17567  opened by brett.cannon

#17568: re: Infinite loop with repeated empty alternative
http://bugs.python.org/issue17568  opened by ericp

#17569: urllib2 urlopen truncates https pages after 32768 characters
http://bugs.python.org/issue17569  opened by jhp7e

#17570: Improve devguide Windows instructions
http://bugs.python.org/issue17570  opened by ezio.melotti

#17571: broken links on Lib/datetime.py docstring
http://bugs.python.org/issue17571  opened by tshepang

#17572: strptime exception context
http://bugs.python.org/issue17572  opened by Claudiu.Popa

#17573: add ElementTree XML processing benchmark to benchmark suite
http://bugs.python.org/issue17573  opened by scoder

#17574: pysetup failing with "OSError: [Errno 18] Invalid cross-device
http://bugs.python.org/issue17574  opened by vaab

#17575: HTTPConnection.send
http://bugs.python.org/issue17575  opened by dspublic at freemail.hu



Most recent 15 issues with no replies (15)
==========================================

#17575: HTTPConnection.send
http://bugs.python.org/issue17575

#17573: add ElementTree XML processing benchmark to benchmark suite
http://bugs.python.org/issue17573

#17572: strptime exception context
http://bugs.python.org/issue17572

#17571: broken links on Lib/datetime.py docstring
http://bugs.python.org/issue17571

#17570: Improve devguide Windows instructions
http://bugs.python.org/issue17570

#17568: re: Infinite loop with repeated empty alternative
http://bugs.python.org/issue17568

#17567: Clarify importlib.abc.PathEntryFinder.find_loader() docs
http://bugs.python.org/issue17567

#17566: Document that importlib.abc.Loader.module_repr is abstract and
http://bugs.python.org/issue17566

#17563: Excessive resizing of dicts when used as a cache
http://bugs.python.org/issue17563

#17558: gdb debugging python frames in optimised interpreters
http://bugs.python.org/issue17558

#17557: test_getgroups of test_posix can fail on OS X 10.8 if more tha
http://bugs.python.org/issue17557

#17547: "checking whether gcc supports ParseTuple __format__... " erro
http://bugs.python.org/issue17547

#17540: logging formatter support 'style' key in dictionary config
http://bugs.python.org/issue17540

#17539: Use the builtins module in the unittest.mock.patch example
http://bugs.python.org/issue17539

#17532: IDLE: Always include "Options" menu on MacOSX
http://bugs.python.org/issue17532



Most recent 15 issues waiting for review (15)
=============================================

#17573: add ElementTree XML processing benchmark to benchmark suite
http://bugs.python.org/issue17573

#17572: strptime exception context
http://bugs.python.org/issue17572

#17563: Excessive resizing of dicts when used as a cache
http://bugs.python.org/issue17563

#17561: Add socket.create_server_sock() convenience function
http://bugs.python.org/issue17561

#17555: Creating new processes after importing multiprocessing.manager
http://bugs.python.org/issue17555

#17554: Compact output for regrtest
http://bugs.python.org/issue17554

#17552: socket.sendfile()
http://bugs.python.org/issue17552

#17547: "checking whether gcc supports ParseTuple __format__... " erro
http://bugs.python.org/issue17547

#17540: logging formatter support 'style' key in dictionary config
http://bugs.python.org/issue17540

#17539: Use the builtins module in the unittest.mock.patch example
http://bugs.python.org/issue17539

#17538: Document XML Vulnerabilties
http://bugs.python.org/issue17538

#17536: update browser list with additional browser names
http://bugs.python.org/issue17536

#17535: IDLE: Add an option to show line numbers along the left side o
http://bugs.python.org/issue17535

#17529: fix os.sendfile() documentation regarding the type of file des
http://bugs.python.org/issue17529

#17527: PATCH as valid request method in wsgiref.validator
http://bugs.python.org/issue17527



Top 10 most discussed issues (10)
=================================

#17546: Document the circumstances where the locals() dict gets	update
http://bugs.python.org/issue17546  24 msgs

#17561: Add socket.create_server_sock() convenience function
http://bugs.python.org/issue17561  14 msgs

#17538: Document XML Vulnerabilties
http://bugs.python.org/issue17538  13 msgs

#17560: problem using multiprocessing with really big objects?
http://bugs.python.org/issue17560  13 msgs

#17554: Compact output for regrtest
http://bugs.python.org/issue17554   9 msgs

#17536: update browser list with additional browser names
http://bugs.python.org/issue17536   8 msgs

#17522: Add api PyGILState_Check
http://bugs.python.org/issue17522   7 msgs

#17564: test_urllib2_localnet fails
http://bugs.python.org/issue17564   7 msgs

#2704: IDLE: Patch to make PyShell behave more like a Terminal interf
http://bugs.python.org/issue2704   6 msgs

#7511: msvc9compiler.py: ValueError when trying to compile with VC Ex
http://bugs.python.org/issue7511   6 msgs



Issues closed (56)
==================

#4022: 2.6 dependent on c:\python26\ on windows
http://bugs.python.org/issue4022  closed by georg.brandl

#4159: Table about Standard Encodings is cut off at the bottom - 35 e
http://bugs.python.org/issue4159  closed by python-dev

#4653: Patch to fix typos in C code
http://bugs.python.org/issue4653  closed by gregory.p.smith

#5135: Expose simplegeneric function in functools module
http://bugs.python.org/issue5135  closed by terry.reedy

#5445: codecs.StreamWriter.writelines problem when passed generator
http://bugs.python.org/issue5445  closed by benjamin.peterson

#5970: sys.exc_info leaks into a generator
http://bugs.python.org/issue5970  closed by georg.brandl

#6310: Windows "App Paths" key is not checked when installed for curr
http://bugs.python.org/issue6310  closed by georg.brandl

#6661: Transient test_multiprocessing failure (test_active_children)
http://bugs.python.org/issue6661  closed by georg.brandl

#7300: Unicode arguments in str.format()
http://bugs.python.org/issue7300  closed by georg.brandl

#7565: Increasing resource.RLIMIT_NOFILE has no effect
http://bugs.python.org/issue7565  closed by georg.brandl

#8552: msilib can't create large CAB files
http://bugs.python.org/issue8552  closed by r.david.murray

#8906: Document TestCase attributes in class docstring
http://bugs.python.org/issue8906  closed by ezio.melotti

#8911: regrtest.main should have a test skipping argument
http://bugs.python.org/issue8911  closed by ezio.melotti

#9672: test_xpickle fails on Windows: invokes pythonx.y instead of py
http://bugs.python.org/issue9672  closed by georg.brandl

#9986: PDF files of python docs have text missing
http://bugs.python.org/issue9986  closed by georg.brandl

#10211: BufferObject doesn't support new buffer interface
http://bugs.python.org/issue10211  closed by kristjan.jonsson

#10359: ISO C cleanup
http://bugs.python.org/issue10359  closed by gregory.p.smith

#11087: Speeding up the interpreter with a few lines of code
http://bugs.python.org/issue11087  closed by pitrou

#12207: Document ast.PyCF_ONLY_AST
http://bugs.python.org/issue12207  closed by eric.araujo

#14468: Update cloning guidelines in devguide
http://bugs.python.org/issue14468  closed by ezio.melotti

#15052: Outdated comments in build_ssl.py
http://bugs.python.org/issue15052  closed by loewis

#15611: devguide: add "core mentors" area to Experts Index
http://bugs.python.org/issue15611  closed by brett.cannon

#16204: PyBuffer_FillInfo returns 'B' buffer, whose behavior has chang
http://bugs.python.org/issue16204  closed by skrah

#16676: Segfault under Python 3.3 after PyType_GenericNew
http://bugs.python.org/issue16676  closed by benjamin.peterson

#16692: Support TLS 1.1 and TLS 1.2
http://bugs.python.org/issue16692  closed by pitrou

#16880: Importing "imp" will fail if dynamic loading not supported
http://bugs.python.org/issue16880  closed by brett.cannon

#17025: reduce multiprocessing.Queue contention
http://bugs.python.org/issue17025  closed by neologix

#17100: rotating an ordereddict
http://bugs.python.org/issue17100  closed by rhettinger

#17150: pprint could use line continuation for long string literals
http://bugs.python.org/issue17150  closed by pitrou

#17316: Add Django 1.5 to benchmarks
http://bugs.python.org/issue17316  closed by brett.cannon

#17317: Benchmark driver should calculate actual benchmark count in -h
http://bugs.python.org/issue17317  closed by brett.cannon

#17323: Disable [X refs, Y blocks] ouput in debug builds
http://bugs.python.org/issue17323  closed by ezio.melotti

#17329: Document unittest.SkipTest
http://bugs.python.org/issue17329  closed by ezio.melotti

#17389: Optimize Event.wait()
http://bugs.python.org/issue17389  closed by pitrou

#17425: Update OpenSSL versions in Windows builds
http://bugs.python.org/issue17425  closed by loewis

#17433: stdlib generator-like iterators don't forward send/throw
http://bugs.python.org/issue17433  closed by terry.reedy

#17438: json.load docs should mention that it always return unicode
http://bugs.python.org/issue17438  closed by ezio.melotti

#17447: str.identifier shouldn't accept Python keywords
http://bugs.python.org/issue17447  closed by rhettinger

#17479: Fix test discovery for test_io.py
http://bugs.python.org/issue17479  closed by ezio.melotti

#17488: subprocess.Popen bufsize=0 parameter behaves differently in Py
http://bugs.python.org/issue17488  closed by gregory.p.smith

#17489: random.Random implements __getstate__() and __reduce__()
http://bugs.python.org/issue17489  closed by rhettinger

#17504: Dropping duplicated docstring explanation of what Mocks' side_
http://bugs.python.org/issue17504  closed by ezio.melotti

#17510: assertEquals deprecated in test_program.py (unittest)
http://bugs.python.org/issue17510  closed by ezio.melotti

#17516: Dead code should be removed
http://bugs.python.org/issue17516  closed by haypo

#17519: unittest should not try to run abstract classes
http://bugs.python.org/issue17519  closed by michael.foord

#17521: fileConfig() disables any previously-used "named" loggers, eve
http://bugs.python.org/issue17521  closed by python-dev

#17524: Problem to run a method
http://bugs.python.org/issue17524  closed by brett.cannon

#17531: test_grp and test_pwd fail with 32-bit builds on OS X systems
http://bugs.python.org/issue17531  closed by ned.deily

#17541: Importing `webbrowser` module gives NameError in Python 2.7.4r
http://bugs.python.org/issue17541  closed by ezio.melotti

#17550: --enable-profiling does nothing (shell syntax bug in configure
http://bugs.python.org/issue17550  closed by python-dev

#17556: os.path.join() converts None to '' by default
http://bugs.python.org/issue17556  closed by georg.brandl

#17559: str.is* implementation seem suboptimal for single character st
http://bugs.python.org/issue17559  closed by georg.brandl

#17562: spam
http://bugs.python.org/issue17562  closed by ezio.melotti

#444582: Finding programs in PATH, adding shutil.which
http://bugs.python.org/issue444582  closed by eric.araujo

#616013: cPickle documentation incomplete
http://bugs.python.org/issue616013  closed by georg.brandl

#886488: popen2 on Windows does not check _fdopen return value
http://bugs.python.org/issue886488  closed by terry.reedy

From ncoghlan at gmail.com  Fri Mar 29 21:41:19 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Mar 2013 06:41:19 +1000
Subject: [Python-Dev] Writing importers and path hooks
In-Reply-To: <CAP1=2W6M97AsA6w1G7ibLhLrbGC4csAmpfsQzSeO10m-wQn4Sw@mail.gmail.com>
References: <CACac1F8DhamOLuugPF4k-2nSN6kztZeohJbP=8SpiDGEG04tLQ@mail.gmail.com>
	<CAP1=2W4BChvsRM0=k+Yr4j8qfZ9hXguv2mAp5UK8o6hHuQBZMg@mail.gmail.com>
	<CACac1F9uV3quw3YZKOQy6jBaoKnbZ5esGDaUe3o7-jh-crfxEA@mail.gmail.com>
	<CAP1=2W5yu3azH7bH4ZRH4esdRfXizKEkvUKaqEuy5oaBumOJWw@mail.gmail.com>
	<CACac1F_kjuYjwogS5akbW4RW1HD3CCsWCfaVDBVJsca5Kbb1vA@mail.gmail.com>
	<CAP1=2W6M97AsA6w1G7ibLhLrbGC4csAmpfsQzSeO10m-wQn4Sw@mail.gmail.com>
Message-ID: <CADiSq7ep+hKbkZt+ZB8aWC7ENRFJnXPJ_gtD1oiBtLyJOgxQBw@mail.gmail.com>

On Fri, Mar 29, 2013 at 3:39 AM, Brett Cannon <brett at python.org> wrote:
> To tell if a module is a package, you should do either ``if mod.__name__ ==
> mod.__package__`` or ``if hasattr(mod, '__path__')``.

The second of those is actually a bit more reliable. As with many
import quirks, the answer to "But why?" is "Because __main__" :P

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From ncoghlan at gmail.com  Sat Mar 30 02:33:59 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Mar 2013 11:33:59 +1000
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
Message-ID: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>

I am accepting Todd Rovito's and Terry Reedy's PEP 434, officially
declaring IDLE to be an application bundled with Python, with the
contents of "Lib/idlelib" exempt from the usual "no new features in
maintenance releases" rule.

As stated in the PEP, this isn't carte blanche to do major rewrites in
maintenance releases, merely acknowledgement that, when in doubt, we
better serve our users by treating IDLE as a bundled application and
making it behave consistently across all supported versions than we do
by treating it as a library first and an application second.

Hopefully this clarification, and the stated goal of supporting IDLE
as a high quality cross-platform default starting point for new Python
users that aren't already accustomed to the command line and editing
text files directly, will make it easier for the IDLE developers to
focus on making IDLE excel at that task.

Regards,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From fijall at gmail.com  Sat Mar 30 03:40:14 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Fri, 29 Mar 2013 19:40:14 -0700
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
Message-ID: <CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>

On Fri, Mar 29, 2013 at 6:33 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
> I am accepting Todd Rovito's and Terry Reedy's PEP 434, officially
> declaring IDLE to be an application bundled with Python, with the
> contents of "Lib/idlelib" exempt from the usual "no new features in
> maintenance releases" rule.
>
> As stated in the PEP, this isn't carte blanche to do major rewrites in
> maintenance releases, merely acknowledgement that, when in doubt, we
> better serve our users by treating IDLE as a bundled application and
> making it behave consistently across all supported versions than we do
> by treating it as a library first and an application second.
>
> Hopefully this clarification, and the stated goal of supporting IDLE
> as a high quality cross-platform default starting point for new Python
> users that aren't already accustomed to the command line and editing
> text files directly, will make it easier for the IDLE developers to
> focus on making IDLE excel at that task.
>
> Regards,
> Nick.

Does that mean that mainstream idle development should move out of the
python tree?

From rovitotv at gmail.com  Sat Mar 30 04:01:09 2013
From: rovitotv at gmail.com (Todd Rovito)
Date: Fri, 29 Mar 2013 23:01:09 -0400
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
Message-ID: <CALZ1apy1o3+VHGkRYsNtesWta2vFTyv7Qi411PzQVp4F6SNfew@mail.gmail.com>

On Fri, Mar 29, 2013 at 10:40 PM, Maciej Fijalkowski <fijall at gmail.com>wrote:

> Does that mean that mainstream idle development should move out of the
> python tree?


No the acceptance of PEP-434 does not mean IDLE development should move out
of the python tree.  The acceptance of PEP-434 means that the restriction
on applying enhancements be relaxed for IDLE code residing in
../Lib/idlelib.  In other words Python Core Developers can apply
enhancements (but not major rewrites) even to the 2.7 branch. The
relaxation was requested in the hope that we can apply many of the already
existing patches quickly and allow IDLE to become a high quality
cross-platform default starting point for new Python users that aren't
already accustomed to the command line and editing text files directly.
 PEP-434 doesn't suggest moving the IDLE code outside of the Python tree.
 Please let me know if you have additional questions, feel free to help us
with IDLE development!
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130329/9624decb/attachment.html>

From ncoghlan at gmail.com  Sat Mar 30 07:21:35 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 30 Mar 2013 16:21:35 +1000
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
Message-ID: <CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>

On Sat, Mar 30, 2013 at 12:40 PM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Fri, Mar 29, 2013 at 6:33 PM, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> I am accepting Todd Rovito's and Terry Reedy's PEP 434, officially
>> declaring IDLE to be an application bundled with Python, with the
>> contents of "Lib/idlelib" exempt from the usual "no new features in
>> maintenance releases" rule.
>>
>> As stated in the PEP, this isn't carte blanche to do major rewrites in
>> maintenance releases, merely acknowledgement that, when in doubt, we
>> better serve our users by treating IDLE as a bundled application and
>> making it behave consistently across all supported versions than we do
>> by treating it as a library first and an application second.
>>
>> Hopefully this clarification, and the stated goal of supporting IDLE
>> as a high quality cross-platform default starting point for new Python
>> users that aren't already accustomed to the command line and editing
>> text files directly, will make it easier for the IDLE developers to
>> focus on making IDLE excel at that task.
>>
>> Regards,
>> Nick.
>
> Does that mean that mainstream idle development should move out of the
> python tree?

That will ultimately be up to the IDLE developers. However, I don't
expect it to happen any time soon, as remaining in the CPython repo
allows them to easily re-use the existing buildbot fleet as they try
to build out a decent test suite, and also means they don't have to
spend their time working out a completely new development workflow
rather than working on IDLE as it exists now.

Past experience also suggests that maintaining things in the CPython
repo and cutting periodic external releases (if the IDLE developers
ever choose to do that) works a *lot* better than trying to
periodically reintegrate an externally maintained tool.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia

From hodgestar+pythondev at gmail.com  Sat Mar 30 07:33:38 2013
From: hodgestar+pythondev at gmail.com (Simon Cross)
Date: Sat, 30 Mar 2013 08:33:38 +0200
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
	<CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
Message-ID: <CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>

Having a standalone version of IDLE might be really useful to
alternative Python implementations.

From solipsis at pitrou.net  Sat Mar 30 13:26:08 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Mar 2013 13:26:08 +0100
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
	<CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
	<CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>
Message-ID: <20130330132608.6ec898bd@pitrou.net>

On Sat, 30 Mar 2013 08:33:38 +0200
Simon Cross <hodgestar+pythondev at gmail.com> wrote:

> Having a standalone version of IDLE might be really useful to
> alternative Python implementations.

Why?



From solipsis at pitrou.net  Sat Mar 30 13:28:00 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Mar 2013 13:28:00 +0100
Subject: [Python-Dev] cpython (2.7): Fix typos and clear up one very odd
 bit of wording as pointed out by
References: <3ZdCsJ4BKJzPyZ@mail.python.org>
Message-ID: <20130330132800.5b3b9454@pitrou.net>

On Sat, 30 Mar 2013 09:39:00 +0100 (CET)
gregory.p.smith <python-checkins at python.org> wrote:
>  
>  The workarounds and modifications are not included in patch releases as they
>  break backward compatibility.

This is not entirely true. It was perfectly possible to give the
*options* to change behaviour, but the patch wasn't ready.

Regards

Antoine.



From fijall at gmail.com  Sat Mar 30 13:35:44 2013
From: fijall at gmail.com (Maciej Fijalkowski)
Date: Sat, 30 Mar 2013 05:35:44 -0700
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <20130330132608.6ec898bd@pitrou.net>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
	<CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
	<CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>
	<20130330132608.6ec898bd@pitrou.net>
Message-ID: <CAK5idxREAryGtWvF_+111ivXT8q8jqK9sA_DaE7q-R7Mn81REg@mail.gmail.com>

On Sat, Mar 30, 2013 at 5:26 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 30 Mar 2013 08:33:38 +0200
> Simon Cross <hodgestar+pythondev at gmail.com> wrote:
>
>> Having a standalone version of IDLE might be really useful to
>> alternative Python implementations.
>
> Why?

I don't think it's worth discussing - tkinter does not work on any
other implementation than CPython and it seems it won't work. It's a
bit pity, but I guess if I felt really bad about it, I should just
make it work.

PS. are there idle projects in SoC? Maybe we should put a more
pypy-friendly one there too?

Cheers,
fijal

From dholth at gmail.com  Sat Mar 30 20:26:51 2013
From: dholth at gmail.com (Daniel Holth)
Date: Sat, 30 Mar 2013 15:26:51 -0400
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <CAK5idxREAryGtWvF_+111ivXT8q8jqK9sA_DaE7q-R7Mn81REg@mail.gmail.com>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
	<CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
	<CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>
	<20130330132608.6ec898bd@pitrou.net>
	<CAK5idxREAryGtWvF_+111ivXT8q8jqK9sA_DaE7q-R7Mn81REg@mail.gmail.com>
Message-ID: <CAG8k2+6MmQ49mcTXuqBq9Fxe5bcV1XSSxrHcdgM2kLVUgn3GbQ@mail.gmail.com>

Yes, it would probably make more sense to split the editor and shell
processes as many Python IDEs do, with IDLE running in CPython and the
user's computation running in the chosen interpreter.

On Sat, Mar 30, 2013 at 8:35 AM, Maciej Fijalkowski <fijall at gmail.com> wrote:
> On Sat, Mar 30, 2013 at 5:26 AM, Antoine Pitrou <solipsis at pitrou.net> wrote:
>> On Sat, 30 Mar 2013 08:33:38 +0200
>> Simon Cross <hodgestar+pythondev at gmail.com> wrote:
>>
>>> Having a standalone version of IDLE might be really useful to
>>> alternative Python implementations.
>>
>> Why?
>
> I don't think it's worth discussing - tkinter does not work on any
> other implementation than CPython and it seems it won't work. It's a
> bit pity, but I guess if I felt really bad about it, I should just
> make it work.
>
> PS. are there idle projects in SoC? Maybe we should put a more
> pypy-friendly one there too?
>
> Cheers,
> fijal
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/dholth%40gmail.com

From fwierzbicki at gmail.com  Sat Mar 30 21:36:39 2013
From: fwierzbicki at gmail.com (fwierzbicki at gmail.com)
Date: Sat, 30 Mar 2013 13:36:39 -0700
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
In-Reply-To: <CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
	<CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
	<CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>
Message-ID: <CADrh4zK=A8gniuNz10=f+ULBcK-WOwLh8ASE_danpZSLcdZLpw@mail.gmail.com>

On Fri, Mar 29, 2013 at 11:33 PM, Simon Cross <hodgestar+pythondev at gmail.com
> wrote:

> Having a standalone version of IDLE might be really useful to
> alternative Python implementations.

I suspect it's too hard. I remember seeing some work on "anygui.py" that
looked like an attempt to make these sorts of things work across various
windowing platforms, but I don't think it made it very far.

-Frank
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130330/f9c1d895/attachment.html>

From solipsis at pitrou.net  Sat Mar 30 21:38:08 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Mar 2013 21:38:08 +0100
Subject: [Python-Dev] Accepting PEP 434, Idle Enhancement Exception
References: <CADiSq7f2m__izZWFWTUA9osUsPm1R1xAYWxJshdTn1TvVL6n9A@mail.gmail.com>
	<CAK5idxRxaKCN5+5U66Ej_hct79f0it3kTGPExrN0Yk+1Qi27MQ@mail.gmail.com>
	<CADiSq7f_nv3PJM1H7N1jM_OkDC0gfH46sz57tRcjEL=1ghUJEg@mail.gmail.com>
	<CAD5NRCFnAyt7AaXXxQMO6D7OJqxENOPODLvyde0Eb=y7ADOokA@mail.gmail.com>
	<CADrh4zK=A8gniuNz10=f+ULBcK-WOwLh8ASE_danpZSLcdZLpw@mail.gmail.com>
Message-ID: <20130330213808.375ee9ee@pitrou.net>

On Sat, 30 Mar 2013 13:36:39 -0700
"fwierzbicki at gmail.com" <fwierzbicki at gmail.com> wrote:
> On Fri, Mar 29, 2013 at 11:33 PM, Simon Cross <hodgestar+pythondev at gmail.com
> > wrote:
> 
> > Having a standalone version of IDLE might be really useful to
> > alternative Python implementations.
> 
> I suspect it's too hard. I remember seeing some work on "anygui.py" that
> looked like an attempt to make these sorts of things work across various
> windowing platforms, but I don't think it made it very far.

Instead of "anygui.py", one could probably start with Qt or wxWidgets.
But that would add a dependency to a very large 3rd party library.

Regards

Antoine.



From mynameisfiber at gmail.com  Sat Mar 30 22:31:26 2013
From: mynameisfiber at gmail.com (Micha Gorelick)
Date: Sat, 30 Mar 2013 17:31:26 -0400
Subject: [Python-Dev] py2.7: dictobject not properly resizing
Message-ID: <CAONs-SfgrTZan=NDZPMYZUn43H5VvcUNZMa6TbuX0WD31rnqWA@mail.gmail.com>

I was taking a look at dictobject.c and realized that the logic
controlling whether a resizedict will occur in
dict_set_item_by_hash_or_entry disallows for the shrinking of a
dictionary.  This is contrary to what the comments directly above say:

(http://hg.python.org/cpython/file/f3032825f637/Objects/dictobject.c#l771)
771 /* If we added a key, we can safely resize. Otherwise just return!
772 * If fill >= 2/3 size, adjust size. Normally, this doubles or
773 * quaduples the size, but it's also possible for the dict to shrink
774 * (if ma_fill is much larger than ma_used, meaning a lot of dict
775 * keys have been * deleted).

The "bug" occures in the following conditional since we exit out of
the function without checking the relative magnitudes of ma_filled to
ma_used. Instead, we only check if we still have a correct loading
factor (and the "don't resize on modification" bit).

This can be fixed by changing the following conditional on line 785 to:

if (mp->ma_used <= n_used || (mp->ma_fill*3 < (mp->ma_mask+1)*2 &&
mp->ma_used*5 > mp->ma_fill))

The factor of 5 was chosen arbitrarily... I'm sure with some
benchmarking we could tune it to an optimal value for the internal use
of dictionaries.  However, before I put this effort in I was wondering if
this is in fact desired behavior or if it is indeed a bug.  At the
very least, the comments should be updated to reflect the actual
resizing dynamics of the dictionary.

Micha
-----------------------------
http://micha.gd/
http://github.com/mynameisfiber/

From solipsis at pitrou.net  Sat Mar 30 22:37:01 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sat, 30 Mar 2013 22:37:01 +0100
Subject: [Python-Dev] py2.7: dictobject not properly resizing
References: <CAONs-SfgrTZan=NDZPMYZUn43H5VvcUNZMa6TbuX0WD31rnqWA@mail.gmail.com>
Message-ID: <20130330223701.6ac267d8@pitrou.net>

On Sat, 30 Mar 2013 17:31:26 -0400
Micha Gorelick <mynameisfiber at gmail.com> wrote:
> I was taking a look at dictobject.c and realized that the logic
> controlling whether a resizedict will occur in
> dict_set_item_by_hash_or_entry disallows for the shrinking of a
> dictionary.  This is contrary to what the comments directly above say:

Also in 3.4:

>>> d = {i: i for i in range(1000)}
>>> sys.getsizeof(d)
49264
>>> for i in range(900): del d[i]
... 
>>> sys.getsizeof(d)
49264
>>> for i in range(900, 1000): del d[i]
... 
>>> sys.getsizeof(d)
49264
>>> len(d)
0
>>> d.clear()
>>> sys.getsizeof(d)
88


Regards

Antoine.



From arigo at tunes.org  Sat Mar 30 22:56:10 2013
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 30 Mar 2013 22:56:10 +0100
Subject: [Python-Dev] py2.7: dictobject not properly resizing
In-Reply-To: <20130330223701.6ac267d8@pitrou.net>
References: <CAONs-SfgrTZan=NDZPMYZUn43H5VvcUNZMa6TbuX0WD31rnqWA@mail.gmail.com>
	<20130330223701.6ac267d8@pitrou.net>
Message-ID: <CAMSv6X366kn4Th+XNh2rXQGrDphAMr9ChxNJ4Lwdt16+QjJ49w@mail.gmail.com>

Hi Antoine,

On Sat, Mar 30, 2013 at 10:37 PM, Antoine Pitrou <solipsis at pitrou.net> wrote:
> On Sat, 30 Mar 2013 17:31:26 -0400
> Micha Gorelick <mynameisfiber at gmail.com> wrote:
>> I was taking a look at dictobject.c and realized that the logic
>> controlling whether a resizedict will occur in
>> dict_set_item_by_hash_or_entry disallows for the shrinking of a
>> dictionary.

It doesn't disallow shrinking.  If you take a dictionary of size 1000,
remove of its elements, and continue to use it (write and delete more
items) for long enough, then eventually it is shrinked.  It just takes
a while because it needs to fill 2/3 of the slots of the big table
with "deleted" markers before it happens.

Python 3.3.0b1 (default:07ddf5ecaafa, Aug 12 2012, 17:47:28)
[GCC 3.4.6 (Gentoo 3.4.6-r2, ssp-3.4.6-1.0, pie-8.7.10)] on linux
Type "help", "copyright", "credits" or "license" for more information.
>>> d = {i:i for i in range(1000)}
>>> sys.getsizeof(d)
24624
>>> for i in range(1000): del d[i]
...
>>> sys.getsizeof(d)
24624
>>> for j in range(1001, 2000):
...    d[j] = 0; del d[j]
...
>>> sys.getsizeof(d)
144
>>>


A bient?t,

Armin.

From mynameisfiber at gmail.com  Sat Mar 30 23:30:38 2013
From: mynameisfiber at gmail.com (Micha Gorelick)
Date: Sat, 30 Mar 2013 18:30:38 -0400
Subject: [Python-Dev] py2.7: dictobject not properly resizing
Message-ID: <CAONs-SegRQOQ8kygJ6MQ+AT0SWDDcJheOP0K31=dY9vFxQttRA@mail.gmail.com>

True, but your example mechanism of getting a shrink event is purely
based on ma_fill.  This is happening because your last loop is
increasing ma_fill to the point where it thinks it needs to resize
because of the load factor and then it calculates the new size based
on ma_used.  The comment that I pointed out from the code seems to
imply that simply having ma_fill >> ma_used will trigger a resize.
The two conditions for a resize are definitely not equivalent!

Micha
-----------------------------
http://micha.gd/
http://github.com/mynameisfiber/

From tjreedy at udel.edu  Sun Mar 31 04:34:32 2013
From: tjreedy at udel.edu (Terry Jan Reedy)
Date: Sat, 30 Mar 2013 22:34:32 -0400
Subject: [Python-Dev] Idle, site.py, and the release candidates
Message-ID: <kj87bl$k0f$1@ger.gmane.org>

While trying to test the patch for
http://bugs.python.org/issue5492
on Windows, I discovered that quit() and exit() in the Idle Shell are 
now disabled, it seems, for all versions on all systems rather than just 
sometimes on Linux.

The problem is a change in idlelib that invalidated an assumption made 
in site.py. Revs 81718-81721 for
http://bugs.python.org/issue9290
changed idlelib.PyShell.PseudoFile (line 1277 in 3.3) to subclass 
io.TextIOBase, which subclasses IOBase. This gave PseudoFile and its 
subclasses a .fileno instance method attribute that raises 
io.UnsupportedOperation: fileno.

This is not a bug since the doc for io.IOBase.fileno says:
"Return the underlying file descriptor (an integer) of the stream if it 
exists. An OSError is raised if the IO object does not use a file 
descriptor."
(the particular error raised is not an issue here).

This is the code for Quitter.__call__ in site.py (line 368 in 3.3):

         def __call__(self, code=None):
             # Shells like IDLE catch the SystemExit, but listen when
             # stdin wrapper is closed.
             try:
                 fd = -1
                 if hasattr(sys.stdin, "fileno"):
                     fd = sys.stdin.fileno()
                 if fd != 0:
                     # Don't close stdin if it wraps fd 0
                     sys.stdin.close()
             except:
                 pass
             raise SystemExit(code)

The incorrect assumption is that if sys.stdin.fileno exits but raises, 
the call did not come from a shell that needs .close called.

I do not know enough about other circumstances in which stdin.fileno 
would do something other than return 0 to be sure of what the proper fix 
would be.  (I increasingly dislike bare excepts as they hide the 
thinking and knowledge of the original programmer. What exception was 
expected that should be passed away?)

Given that the callable constants exit and quit  and are optionally 
suppressed on startup and are not standard builtins
http://docs.python.org/3/library/constants.html#constants-added-by-the-site-module
it would be alright with me to ignore this regression and release as 
scheduled. But I though people should be aware of it.

--
Terry Jan Reedy


From ncoghlan at gmail.com  Sun Mar 31 08:39:49 2013
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 31 Mar 2013 16:39:49 +1000
Subject: [Python-Dev] Idle, site.py, and the release candidates
In-Reply-To: <kj87bl$k0f$1@ger.gmane.org>
References: <kj87bl$k0f$1@ger.gmane.org>
Message-ID: <CADiSq7dCY_8L7cA38Zo-AwS3_hBvuffh-kj5ZNY8oEcWYgxoYw@mail.gmail.com>

On Sun, Mar 31, 2013 at 12:34 PM, Terry Jan Reedy <tjreedy at udel.edu> wrote:

> While trying to test the patch for
> http://bugs.python.org/**issue5492 <http://bugs.python.org/issue5492>
> on Windows, I discovered that quit() and exit() in the Idle Shell are now
> disabled, it seems, for all versions on all systems rather than just
> sometimes on Linux.
>
> The problem is a change in idlelib that invalidated an assumption made in
> site.py. Revs 81718-81721 for
> http://bugs.python.org/**issue9290 <http://bugs.python.org/issue9290>
> changed idlelib.PyShell.PseudoFile (line 1277 in 3.3) to subclass
> io.TextIOBase, which subclasses IOBase. This gave PseudoFile and its
> subclasses a .fileno instance method attribute that raises
> io.UnsupportedOperation: fileno.
>
> This is not a bug since the doc for io.IOBase.fileno says:
> "Return the underlying file descriptor (an integer) of the stream if it
> exists. An OSError is raised if the IO object does not use a file
> descriptor."
> (the particular error raised is not an issue here).
>
> This is the code for Quitter.__call__ in site.py (line 368 in 3.3):
>
>         def __call__(self, code=None):
>             # Shells like IDLE catch the SystemExit, but listen when
>             # stdin wrapper is closed.
>             try:
>                 fd = -1
>                 if hasattr(sys.stdin, "fileno"):
>                     fd = sys.stdin.fileno()
>                 if fd != 0:
>                     # Don't close stdin if it wraps fd 0
>                     sys.stdin.close()
>             except:
>                 pass
>             raise SystemExit(code)
>
> The incorrect assumption is that if sys.stdin.fileno exits but raises, the
> call did not come from a shell that needs .close called.
>
> I do not know enough about other circumstances in which stdin.fileno would
> do something other than return 0 to be sure of what the proper fix would
> be.  (I increasingly dislike bare excepts as they hide the thinking and
> knowledge of the original programmer. What exception was expected that
> should be passed away?)
>

The other problem is that making *two* function calls inside a broad
try/except is almost always a terrible idea. It seems to me that the
intended logic is more like this:

    try:
        # Close stdin if it wraps any fd other than 0
        close_stdin = (sys.stdin.fileno() != 0)
    except (AttributeError, OSError, io.UnsupportedOperation):
        # Also close stdin if it doesn't expose a file descriptor
        close_stdin = True
    if close_stdin:
        try:
            sys.stdin.close()
        except Exception:
            pass
    raise SystemExit(code)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130331/cfed23af/attachment-0001.html>

From tjreedy at udel.edu  Sun Mar 31 09:52:26 2013
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun, 31 Mar 2013 03:52:26 -0400
Subject: [Python-Dev] Idle, site.py, and the release candidates
In-Reply-To: <CADiSq7dCY_8L7cA38Zo-AwS3_hBvuffh-kj5ZNY8oEcWYgxoYw@mail.gmail.com>
References: <kj87bl$k0f$1@ger.gmane.org>
	<CADiSq7dCY_8L7cA38Zo-AwS3_hBvuffh-kj5ZNY8oEcWYgxoYw@mail.gmail.com>
Message-ID: <5157EB3A.4050904@udel.edu>

On 3/31/2013 2:39 AM, Nick Coghlan wrote:
> On Sun, Mar 31, 2013 at 12:34 PM, Terry Jan Reedy <tjreedy at udel.edu 
> <mailto:tjreedy at udel.edu>>
>
>     I do not know enough about other circumstances in which
>     stdin.fileno would do something other than return 0 to be sure of
>     what the proper fix would be.  (I increasingly dislike bare
>     excepts as they hide the thinking and knowledge of the original
>     programmer. What exception was expected that should be passed away?)
>
>
> The other problem is that making *two* function calls inside a broad 
> try/except is almost always a terrible idea.

That too. I could not decide which function an exception was expected from.

> It seems to me that the intended logic is more like this:
>
>     try:
>         # Close stdin if it wraps any fd other than 0
>         close_stdin = (sys.stdin.fileno() != 0)
>     except (AttributeError, OSError, io.UnsupportedOperation):
>         # Also close stdin if it doesn't expose a file descriptor
>         close_stdin = True
>     if close_stdin:
>         try:
>             sys.stdin.close()
>         except Exception:
>             pass
>     raise SystemExit(code)
>
There are 4 possible situations for sys.stdin:
1. No .fileno
2. .fileno raises
3. .fileno() != 0
4. lfileno() == 0
I believe the current code calls .close for 1 and raises SystemExit for 
2-4. Your code calls for 1-3 and raises for 4. I suspect that is 
correct.  For an rc patch, the safest temporary patch would be to start 
.__call__ with

if sys.stdin.__name__ == 'PseudoInputFile': sys.stdin.close()

I would have to check that the name is correct as seen in the user 
process (cannot at moment).

The deeper problem, I think, is that none of sys.version, 
sys.hexversion, sys._version, sys.platform tell a program that it is 
running under Idle. It usually does not matter but there are a few 
situations in which it does.

--
Terry Jan Reedy

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130331/51de28dd/attachment.html>

From solipsis at pitrou.net  Sun Mar 31 12:01:50 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 Mar 2013 12:01:50 +0200
Subject: [Python-Dev] Idle, site.py, and the release candidates
References: <kj87bl$k0f$1@ger.gmane.org>
Message-ID: <20130331120150.4ac38f40@pitrou.net>

On Sat, 30 Mar 2013 22:34:32 -0400
Terry Jan Reedy <tjreedy at udel.edu> wrote:
> 
> I do not know enough about other circumstances in which stdin.fileno 
> would do something other than return 0 to be sure of what the proper fix 
> would be.
> (I increasingly dislike bare excepts as they hide the 
> thinking and knowledge of the original programmer.

You should learn to use the power of version control:
http://docs.python.org/devguide/faq.html#how-do-i-find-out-who-edited-or-what-revision-changed-a-line-last

$ hg ann Lib/site.py

will tell you that the lines you mention come from the following
changeset:

$ hg log -r 86358b43c8bb -vp
changeset:   44390:86358b43c8bb
parent:      44385:5670104acd39
user:        Christian Heimes <christian at cheimes.de>
date:        Mon Dec 31 03:07:24 2007 +0000
files:       Lib/site.py
description:
Don't close sys.stdin with quit() if sys.stdin wraps fd 0. Otherwise it
will raise a warning: Lib/io.py:1221: RuntimeWarning: Trying to close
unclosable fd


diff --git a/Lib/site.py b/Lib/site.py
--- a/Lib/site.py
+++ b/Lib/site.py
@@ -247,7 +247,12 @@
             # Shells like IDLE catch the SystemExit, but listen when
their # stdin wrapper is closed.
             try:
-                sys.stdin.close()
+                fd = -1
+                if hasattr(sys.stdin, "fileno"):
+                    fd = sys.stdin.fileno()
+                if fd != 0:
+                    # Don't close stdin if it wraps fd 0
+                    sys.stdin.close()
             except:
                 pass
             raise SystemExit(code)


In this case the original motivation seems obsolete:

>>> import sys
>>> sys.stdin.fileno()
0
>>> sys.stdin.close()
>>> 


That said, if IDLE users expect those global functions, perhaps IDLE
should define its own ones rather than rely on site.py.

Regards

Antoine.



From mark at hotpy.org  Sun Mar 31 15:29:58 2013
From: mark at hotpy.org (Mark Shannon)
Date: Sun, 31 Mar 2013 14:29:58 +0100
Subject: [Python-Dev] Semantics of __int__(), __index__()
Message-ID: <51583A56.8070404@hotpy.org>

Hi all,

I was looking into http://bugs.python.org/issue17576 and I found that 
the semantics of __int__() and __index__() are not precisely defined in 
the documentation and that the implementation (CPython 3.4a) has some 
odd behaviour.

Defining two classes:

class Int1(int):
     def __init__(self, val=0):
         print("new %s" % self.__class__)

class Int2(Int1):
     def __int__(self):
         return self

and two instances
i1 = Int1()
i2 = Int2()

we get the following behaviour:

 >>> type(int(i1))
<class 'int'>

I would have expected 'Int1'

 >>> type(int(i2))
new <class '__main__.Int2'>
<class '__main__.Int2'>

Why is a new Int2 being created?

operator.index does similar things.

So,
1. Should type(int(x)) be exactly int, or is any subclass OK?
2. Should type(index(x)) be exactly int, or is any subclass OK?
3. Should int(x) be defined as int_check(x.__int__())?
4. Should operator.index(x) be defined as index_check(x.__index__())?
where:
def int_check(x):
     if is_int(x):
         return x
     else:
         raise TypeError(...)
def index_check(x):
     if is_index(x):
         return x
     else:
         raise TypeError(...)

The definition of is_int(x) and is_index(x) follow from the answers to 1 
and 2.

I had previously assumed (and would expect) that the answers were:
1. Any subclass is OK
2. Ditto
3. Yes
4. Yes
Which means that
def is_int(x):
     return int in type(x).__mro__
is_index = is_int

Cheers,
Mark.




From dickinsm at gmail.com  Sun Mar 31 16:28:56 2013
From: dickinsm at gmail.com (Mark Dickinson)
Date: Sun, 31 Mar 2013 15:28:56 +0100
Subject: [Python-Dev] Semantics of __int__(), __index__()
In-Reply-To: <51583A56.8070404@hotpy.org>
References: <51583A56.8070404@hotpy.org>
Message-ID: <CAAu3qLU6Zsq2h0ZUK8J-RphqAzV9S761Pi4JMd6gKZUk49LSkw@mail.gmail.com>

On Sun, Mar 31, 2013 at 2:29 PM, Mark Shannon <mark at hotpy.org> wrote:

> class Int1(int):
>     def __init__(self, val=0):
>         print("new %s" % self.__class__)
>
> class Int2(Int1):
>     def __int__(self):
>         return self
>
> and two instances
> i1 = Int1()
> i2 = Int2()
>
> we get the following behaviour:
>
> >>> type(int(i1))
> <class 'int'>
>
> I would have expected 'Int1'
>

Wouldn't that remove the one obvious way to get an 'int' from an 'Int1'?


> 1. Should type(int(x)) be exactly int, or is any subclass OK?
> 2. Should type(index(x)) be exactly int, or is any subclass OK?
> 3. Should int(x) be defined as int_check(x.__int__())?
> 4. Should operator.index(x) be defined as index_check(x.__index__())?
>

For (1), I'd say yes, it should be exactly an int, so my answer to (3) is
no.

As written, int_check would do the wrong thing for bools, too:  I
definitely want int(True) to be 1, not True.

For (2) and (4), it's not so clear.  Are there use-cases for an __index__
return value that's not directly of type int?  I can't think of any offhand.

Mark
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130331/51c4a9e4/attachment.html>

From tjreedy at udel.edu  Sun Mar 31 17:23:53 2013
From: tjreedy at udel.edu (Terry Jan Reedy)
Date: Sun, 31 Mar 2013 11:23:53 -0400
Subject: [Python-Dev] Idle, site.py, and the release candidates
In-Reply-To: <5157EB3A.4050904@udel.edu>
References: <kj87bl$k0f$1@ger.gmane.org>
	<CADiSq7dCY_8L7cA38Zo-AwS3_hBvuffh-kj5ZNY8oEcWYgxoYw@mail.gmail.com>
	<5157EB3A.4050904@udel.edu>
Message-ID: <kj9ke6$ovp$1@ger.gmane.org>

On 3/31/2013 3:52 AM, Terry Reedy wrote:
> For an rc patch, the safest temporary patch would be to start
> .__call__ with
>
> if sys.stdin.__name__ == 'PseudoInputFile': sys.stdin.close()
>
> I would have to check that the name is correct as seen in the user
> process (cannot at moment).

In addition, idlelib.PyShell.PseudoInputFile needs a .close method

+    def close(self):
+        self.shell.close()
+
http://bugs.python.org/issue17585

--
Terry Jan Reedy



From tjreedy at udel.edu  Sun Mar 31 17:38:07 2013
From: tjreedy at udel.edu (Terry Jan Reedy)
Date: Sun, 31 Mar 2013 11:38:07 -0400
Subject: [Python-Dev] Idle, site.py, and the release candidates
In-Reply-To: <20130331120150.4ac38f40@pitrou.net>
References: <kj87bl$k0f$1@ger.gmane.org> <20130331120150.4ac38f40@pitrou.net>
Message-ID: <kj9l8s$9o$1@ger.gmane.org>

On 3/31/2013 6:01 AM, Antoine Pitrou wrote:

> That said, if IDLE users expect those global functions, perhaps IDLE
> should define its own ones rather than rely on site.py.

I thought of that. Idle would have to check the beginning of every 
statement before sending it to the user process, which would do the same 
thing. I personally think exit/quit should eventually go, as ^d/^z or 
[x] button are as easy.



From solipsis at pitrou.net  Sun Mar 31 17:43:56 2013
From: solipsis at pitrou.net (Antoine Pitrou)
Date: Sun, 31 Mar 2013 17:43:56 +0200
Subject: [Python-Dev] Idle, site.py, and the release candidates
References: <kj87bl$k0f$1@ger.gmane.org> <20130331120150.4ac38f40@pitrou.net>
	<kj9l8s$9o$1@ger.gmane.org>
Message-ID: <20130331174356.51457754@pitrou.net>

On Sun, 31 Mar 2013 11:38:07 -0400
Terry Jan Reedy <tjreedy at udel.edu> wrote:

> On 3/31/2013 6:01 AM, Antoine Pitrou wrote:
> 
> > That said, if IDLE users expect those global functions, perhaps IDLE
> > should define its own ones rather than rely on site.py.
> 
> I thought of that. Idle would have to check the beginning of every 
> statement before sending it to the user process, which would do the same 
> thing. I personally think exit/quit should eventually go, as ^d/^z or 
> [x] button are as easy.

I never use them myself, but they are more discoverable than keyboard
shortcuts, and hence can be useful for beginners or casual users.

Regards

Antoine.



From mynameisfiber at gmail.com  Sun Mar 31 18:51:26 2013
From: mynameisfiber at gmail.com (Micha Gorelick)
Date: Sun, 31 Mar 2013 12:51:26 -0400
Subject: [Python-Dev] py2.7: dictobject not properly resizing
In-Reply-To: <CAONs-SegRQOQ8kygJ6MQ+AT0SWDDcJheOP0K31=dY9vFxQttRA@mail.gmail.com>
References: <CAONs-SegRQOQ8kygJ6MQ+AT0SWDDcJheOP0K31=dY9vFxQttRA@mail.gmail.com>
Message-ID: <CAONs-ScgP0J8t+OTcdnCUTWY_2s=cpiF22H6GNeqWS0TE5=EMg@mail.gmail.com>

So I did a bit of benchmarking and attached is the code I used.  With
downsizing happening when ma_used * 2 <= ma_filled, or the following
for the condition in question:

    if (mp->ma_used <= n_used || (mp->ma_fill*3 < (mp->ma_mask+1)*2 &&
mp->ma_used*2 > mp->ma_fill))

I see marginally faster performance with downsizing.  I chose a factor
of 2x because it will ensure downsizings before the ma_fill load
factor comes into play and will also not cause a resize on the next
insert.  Using the vanilla python2.7.3 code, there are never any
resize events where the new size is smaller than the old size.

Cheers,
Micha
-----------------------------
http://micha.gd/
http://github.com/mynameisfiber/
-------------- next part --------------
A non-text attachment was scrubbed...
Name: test.py
Type: application/octet-stream
Size: 1060 bytes
Desc: not available
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130331/697c6e1d/attachment.obj>

From francismb at email.de  Sun Mar 31 21:13:59 2013
From: francismb at email.de (francis)
Date: Sun, 31 Mar 2013 21:13:59 +0200
Subject: [Python-Dev] Semantics of __int__(), __index__()
In-Reply-To: <51583A56.8070404@hotpy.org>
References: <51583A56.8070404@hotpy.org>
Message-ID: <51588AF7.6000801@email.de>


> and two instances
> i1 = Int1()
> i2 = Int2()
>
> we get the following behaviour:
>
> >>> type(int(i1))
> <class 'int'>
>
> I would have expected 'Int1'
 >>> type(float(i1))
<type 'float'>

 >>> type(float(i2))
<class 'float'>

 >>> isinstance(int(i1), int)
True

 >>> isinstance(int(i2), int)
new <class '__main__.Int2'>
True

 >>> isinstance(float(i1), float)
True
 >>> isinstance(float(i2), float)
True

why is printing new <class '__main__.Int2'>?




From brett at yvrsfo.ca  Sun Mar 31 23:11:35 2013
From: brett at yvrsfo.ca (Brett Cannon)
Date: Sun, 31 Mar 2013 17:11:35 -0400
Subject: [Python-Dev] [Python-checkins] cpython (2.7): Add an itertools
 recipe showing how to use t.__copy__().
In-Reply-To: <3Zdn7J53MYzPt5@mail.python.org>
References: <3Zdn7J53MYzPt5@mail.python.org>
Message-ID: <CAP1=2W4=akgb_M5=+t6d3jghoPK4ecRE_AH8eD8gKjL3cvirxQ@mail.gmail.com>

"Upcomping" -> "upcoming"
On Mar 31, 2013 2:38 AM, "raymond.hettinger" <python-checkins at python.org>
wrote:

> http://hg.python.org/cpython/rev/1026b1d47f30
> changeset:   83037:1026b1d47f30
> branch:      2.7
> parent:      83034:e044d22d2f61
> user:        Raymond Hettinger <python at rcn.com>
> date:        Sat Mar 30 23:37:57 2013 -0700
> summary:
>   Add an itertools recipe showing how to use t.__copy__().
>
> files:
>   Doc/library/itertools.rst |  12 ++++++++++++
>   1 files changed, 12 insertions(+), 0 deletions(-)
>
>
> diff --git a/Doc/library/itertools.rst b/Doc/library/itertools.rst
> --- a/Doc/library/itertools.rst
> +++ b/Doc/library/itertools.rst
> @@ -828,6 +828,18 @@
>         indices = sorted(random.randrange(n) for i in xrange(r))
>         return tuple(pool[i] for i in indices)
>
> +   def tee_lookahead(t, i):
> +       """Inspect the i-th upcomping value from a tee object
> +          while leaving the tee object at its current position.
> +
> +          Raise an IndexError if the underlying iterator doesn't
> +          have enough values.
> +
> +       """
> +       for value in islice(t.__copy__(), i, None):
> +           return value
> +       raise IndexError(i)
> +
>  Note, many of the above recipes can be optimized by replacing global
> lookups
>  with local variables defined as default values.  For example, the
>  *dotproduct* recipe can be written as::
>
> --
> Repository URL: http://hg.python.org/cpython
>
> _______________________________________________
> Python-checkins mailing list
> Python-checkins at python.org
> http://mail.python.org/mailman/listinfo/python-checkins
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://mail.python.org/pipermail/python-dev/attachments/20130331/1a50c4d7/attachment.html>

From bugtrack at roumenpetrov.info  Sun Mar 31 23:26:16 2013
From: bugtrack at roumenpetrov.info (Roumen Petrov)
Date: Mon, 01 Apr 2013 00:26:16 +0300
Subject: [Python-Dev] mingw32 port
In-Reply-To: <5109B192.9050504@ubuntu.com>
References: <5109B192.9050504@ubuntu.com>
Message-ID: <5158A9F8.5000102@roumenpetrov.info>

Matthias Klose wrote:
> [No, I'm not interested in the port myself]
>
> patches for a mingw32 port are floating around on the web and the python bug
> tracker, although most of them as a set of patches in one issue addressing
> several things, and which maybe outdated for the trunk. at least for me
> re-reading a big patch in a new version is more work than having the patches in
> different issues. So proposing to break down the patches in independent ones,
> dealing with:
>
>   - mingw32 support (excluding any cross-build support). tested with
>     a native build with srcdir == builddir. The changes for distutils
>     mingw32 support should be a separate patch. Who could review these?
>     Asked Martin, but didn't get a reply yet.
>
>   - patches to cross-build for mingw32.
>
>   - patches to deal with a srcdir != builddir configuration, where the
>     srcdir is read-only (please don't mix with the cross-build support).
>
> All work should be done on the tip/trunk.
>
> So ok to close issue16526, issue3871, issue3754 and suggest in the reports to
> start over with more granular changes?

I post first part of "split of issue3871" . It is related only to build 
of interpreter core.

Second part consist of 23 patches related to build of standard 
extensions . 3 of them are reported as separate issue by other users. As 
prerequisite is modernization of cygwincompiler.py - ref  issue12641. I 
will post after 2-3 weeks remaining 20 granular updates.

Third part is now with only tree updates related to installation (new).

Unlike issue3871 will be supported posix installation scheme as all 
users refuse to use windows scheme.


>    Matthias
>

Roumen