On 24/09/2010 12:06, Vinay Sajip wrote:
>> Cool, how can I use it in Python 2.6? :-)
> 1. Copy the top part (imports, QueueHandler and QueueListener classes) from the
> Gist linked to in the article - http://gist.github.com/591699 - into a utility
> module you can use again and again.
Although my comment was a light hearted dig from my point of view that
the components of the standard library should be separately packaged and
a python distribution should just be a "known good set" of these packages...
Were that the case, users of Python 2.5, 2.6 *and* 2.7 could benefit
from your good work :-)
Simplistix - Content Management, Batch Processing & Python Consulting
I've been following this thread all week at work, but I thought it might be
time to respond to the different remarks in a single response, given that I
am involved in editing and maintaining the Wiki on python.org, and I had a
strong enough opinion about such things to give a talk about it at EuroPython
that some of you attended:
Guido van Rossum wrote:
> On Thu, Sep 23, 2010 at 3:35 PM, "Martin v. Löwis" <martin at v.loewis.de>
> > There actually is an admin team, and they actually do set ACLs.
> Who are they?
> > IIUC, this is primarily for spam protection, though.
> So would they object against additions to the team?
The administrators are generally the people on the following page:
Someone may have special powers, particularly if they have shell access to the
machine running the Wiki, but there's no "secret brotherhood". In fact, I
recall advocating that Carl Trachte get admin powers given his continuing
high-quality contributions, so we can say that people aren't denying others
administrative privileges if that person is clearly doing good work and would
benefit from being able to have those privileges.
Guido van Rossum wrote:
> On Fri, Sep 24, 2010 at 3:31 AM, Michael Foord
> <fuzzyman at voidspace.org.uk> wrote:
> > Wiki maintenance is discussed, along with other python.org maintenance
> > topics, on the pydotorg-www mailing list:
> > http://mail.python.org/mailman/listinfo/pydotorg-www
> Which has hidden its membership (even to members). Does it really need
> to appear that secretive? At least the message archive is open and has
> all the usual suspects.
As we're now seeing, people don't feel that it's acceptable to publish the
subscribers list, and I think that it's a complete distraction to seek to do
so, anyway. It's not as if everyone on that list has special privileges and
is preventing newcomers from joining it.
> > More wiki and website maintainers needed!
> Maybe a prominent wiki page with info about how to join the list and
> the responsibilities / needs would help?
> Also, I gotta say that the wiki login process is awkward.
MoinMoin supports OpenID, although I did find and report issues with Moin 1.9
in this regard. Something on my now-huge list of things to do is to make Moin
authentication better, especially where there might be a choice of
Georg Brandl wrote:
> Am 23.09.2010 22:25, schrieb anatoly techtonik:
> > On Thu, Sep 23, 2010 at 6:57 PM, Barry Warsaw <barry at python.org> wrote:
> >> I certainly agree with that. So, how can we solve those problems?
> >> Radomir has shell access now so perhaps we can ask him to make the
> >> Python wiki theme more visually appealing. What roadblocks do people
> >> encounter when they want to help garden or reorganize the wiki?
> > First of all Wiki is outdated. Correct me, but python.org specific
> > customizations are not modules, but patches that makes it hard to
> > update the Wiki to latest version or add new customizations.
> That's why we have a Moin core developer on the team. ISTM that Moin 1.x
> is notoriously hard to extend -- once you go beyond plugins adding new wiki
> macros -- which is part of what the team wants to fix in 2.x.
Although I understand the sentiments about "specific customizations",
python.org should only have its theme as something that isn't a generic
extension to MoinMoin, and that theme should be actively maintained. When
python.org switched its architecture a while ago, the special theme
presumably came with the deal, but it's been so out of date for so long that
I switched to one of the standard themes years ago. Fortunately, Radomir's
EuroPython theme is actively maintained, works with recent MoinMoin releases,
looks a lot better than the old theme, and is used elsewhere.
As for Moin 1.x being "notoriously hard to extend" beyond macros, you can get
a long way with macros and actions, although I agree that sometimes it's
possible to hit architectural constraints.
> > Second - ugly Creole syntax. I am for inter-cooperation between wikis,
> > and I understand that for non-developer communities  symbols
> > imposing problems, but as an open source developer I am addicted to
> > Trac and Google Code syntax and never had problems with those.
> This isn't Creole syntax, it's Moin wiki syntax. And face it, it's not
> going to change. It's also not so much different from Trac wiki syntax.
I never agreed with this complaint, anyway. When discussing this previously
with Anatoly, I went as far as to ask on #moin where Radomir actually posted
a good summary of the syntax differences:
I invite anyone to justify claims that the old style (which Trac adopted) was
better. Complaints about double bracketing are specious: MediaWiki has
different bracketing levels and it's really confusing.
> > Fourth. GPL license. I personally don't have interest to waste my time
> > for the code I won't be able to use in some projects, unless the
> > project is either exceptional (Mercurial) or interesting. To make
> > python.org MoinMoin interesting - there should be an inviting
> > entrypoint (see point three above) and the list of tasks to choose
> > form. Something that is better than
> > http://wiki.python.org/moin/SiteImprovements
> Yes, that needs to be updated indeed.
On the matter of the GPL, Anatoly and I more or less agreed to disagree when
we last discussed the Web site. On the matter of site improvements and the
page which attempts to document ideas, we could move such things to a tracker
(where they will probably get as much attention as the Web site bugs get
right now), or we could actually prune a lot of the issues by addressing them
with fixes that are readily available (applying to a lot of "xyz doesn't
work" complaints). Getting people to accept that things aren't going to
change for the sake of it (like the Wiki syntax) is more difficult, but at
some point such discussions just have to end.
anatoly techtonik wrote:
> That's a dead way. Wiki should be open for everyone. Just need more
> people subscribed to revert unwelcome changes. That is to make
> timeline more visible, because on wiki.python.org it is _not_
> intuitive. It may worth to see how Mercurial wiki is managed - I've
> picked up the bookmarks monitoring habit from it. Maybe a design
> change will help, but again - there is no entrypoint for people with
> design skills to start.
The Wiki is generally "open for everyone", but as I noted in my talk you often
get people who decide that their opinion is worth more than the cumulative
knowledge and experience of everyone who has been maintaining a particular
resource . You also have to prevent spamming and vandalism which is quite
well taken care of with the TextCha support, although I accept that
mechanisms should be in place to promote users to higher levels of privilege
after a while - another little project on my long "to do" list - so that they
don't have to answer editing questions.
As for the Mercurial Wiki, I have some involvement with that, and I don't see
how it is significantly differently managed. I've also proposed various
changes to that Wiki including a theme that could be deployed straight away,
but I suspect that it's more a case of people being too busy to take me up on
any such offer than a malicious cabal denying me the chance to improve a
particular Internet resource.
Some final notes on the matter of Wiki editing:
The Wiki has for a long time been a product of many co-existing contributions
that occasionally overlap with each other and are tied together as
respectfully and gently as possible. Although there is a temptation to
overhaul much of the content, respect for the existing structure and content
has mostly restrained any cleaning-up attempts. I wouldn't mind a bit of
reorganisation, but that brings us to the role of the Wiki.
If other resources are meant to provide the bulk of what people consider
the "important" content, then the Wiki will always have to be shaped to
respect that; if the Wiki is to hold much of that content, then issues such
as navigability become even more critical. But what is certain is that
tucking the Wiki away in some obscure location (I had to request that a link
to it be re-added to the front page of python.org recently, as I recall) and
treating it like a playpen will only encourage casual edits with little
concern for the resulting quality. Anyone now on the verge of concluding that
this is purely a Wiki phenomenon should consult my "common myths" slide:
High-quality documentation isn't something you get for nothing, nor is it a
property of some magic solution or tool: there's always some hard work
involved for human beings.
 One "happy" memory of mine involves someone who decided that their style
and terminology edits were so important that they felt the need to tell
me "don't modify it again" when it was they who were doing the modifying.
Done. The other amendments were never actually made, so I just
reverted the Python 3 bit after moving it to the new PEP. I'll make
the changes to 3333 instead as soon as I have another time slot free.
At 01:56 PM 9/26/2010 -0700, Guido van Rossum wrote:
>Since you have commit privileges, just do it. The PEP editor position
>mostly exists to assure non-committers are not prevented from
>Please do add a prominent note at the top of PEP 333 pointing to PEP
>3333 for further information on Python 3 compliance or some such
>words. Add a similar note at the top of PEP 3333 -- maybe mark up the
>differences in PEP 3333 so people can easily tell what was added. And
>move PEP 333 to Final status.
>On Sun, Sep 26, 2010 at 1:50 PM, P.J. Eby <pje(a)telecommunity.com> wrote:
> > At 01:44 PM 9/26/2010 -0700, Guido van Rossum wrote:
> >> On Sun, Sep 26, 2010 at 12:47 PM, Barry Warsaw <barry(a)python.org> wrote:
> >> > On Sep 26, 2010, at 1:33 PM, P.J. Eby wrote:
> >> >
> >> >> At 08:20 AM 9/26/2010 -0700, Guido van Rossum wrote:
> >> >>> I'm happy approving Final status for the
> >> >>> *original* PEP 333 and I'm happy to approve a new PEP which includes
> >> >>> PJE's corrections.
> >> >>
> >> >> Can we make it PEP 3333, then? ;-)
> >> >
> >> > That works for me.
> >> Go for it.
> > Shall I just "svn cp" it, then (to preserve edit history), or wait for the
> > PEP editor do it?
>--Guido van Rossum (python.org/~guido)
>Python-Dev mailing list
At 01:44 PM 9/26/2010 -0700, Guido van Rossum wrote:
>On Sun, Sep 26, 2010 at 12:47 PM, Barry Warsaw <barry(a)python.org> wrote:
> > On Sep 26, 2010, at 1:33 PM, P.J. Eby wrote:
> >> At 08:20 AM 9/26/2010 -0700, Guido van Rossum wrote:
> >>> I'm happy approving Final status for the
> >>> *original* PEP 333 and I'm happy to approve a new PEP which includes
> >>> PJE's corrections.
> >> Can we make it PEP 3333, then? ;-)
> > That works for me.
>Go for it.
Shall I just "svn cp" it, then (to preserve edit history), or wait
for the PEP editor do it?
At 09:22 PM 9/25/2010 -0400, Jesse Noller wrote:
>It seems like it will end up
>different enough to be a different specification, closely related to
>the original, but different enough to trip up all the people
>maintaining current WSGI servers and apps.
The only actual *change* to the spec is mandating the use of the
'bytes' type or equivalent for HTTP bodies when using Python 3.
Seriously, that's *it*.
Everything else that's (planned to be) added is either 100% truly
just clarifications (e.g. nothing in the spec *ever* said SERVER_PORT
could be an int, but apparently some people somehow interpreted it
so), or else best-practice recommendations from people who actually
implemented WSGI servers.
For example, the readline() size hint is "not supported" in the
original spec (meaning clients can't call it and be compliant). The
planned modification is "servers should implement it" (best
practice), but you can't call an implementation that *doesn't*
implement it noncompliant. (This just addresses the fact that most
practical implementations *did* in fact support it, and code out
there relies on this.)
So, no (previously-)compliant implementations were harmed in the
making of the updated spec. If they were compliant before, they're
I'm actually a bit surprised people are bringing this up now, since
when I announced the plan to make these changes, I said that nothing
would be changed that would break anything... even for what I
believe are the only Python 3 WSGI implementations right now (by
Graham Dumpleton and Robert Brewer).
Indeed, all of the changes (except the bytes thing) are stuff
previously discussed endlessly on the Web-SIG (years ago in most
cases) and widely agreed on as, "this should have been made clear in
the original PEP".
And, I also explicitly deferred and/or rejected items that *can't* be
done in a 100% backward-compatible way, and would have to be WSGI 1.1
or higher -- indeed, I have a long list of changes from Graham that
I've pronounced "can't be done without a 1.1".
Indeed, the entire point of the my scope choices were to allow all
this to happen *without* a whole new spec. ;-)
Looks like 2.7 changes introduced to exempt dicts and tuples from
cyclic gc if they obviously can't be in cycles has some unintended
consequences. Specifically, if an extension module calls
PyObject_GC_UnTrack() on a dict it _does not want tracked_, Python can
start tracking the dict again.
I assume this is unintended because (a) the docs weren't changed to
warn about this; and, (b) it's wrong ;-)
There are two main reasons an extension module may have been calling
1. For correctness, if the extension is playing games with reference
counts Python isn't expecting.
2. For speed, if the extension is creating dicts (or tuples) it knows
cannot participate in cycles.
This came up when Jim Fulton asked me for advice about assertion
failures coming out of cyclic gc in a debug build when running ZODB's
tests under 2.7. Turned out to be due to the "#1 reason" above: ZODB
hand-rolled its own version of weak references long before Python had
them, and has a dict mapping strings ("object IDs") to complex objects
where the referenced objects' refcounts intentionally do _not_ account
for the references due to this dict.
This had been working fine for at least 8 years, thanks to calling
PyObject_GC_UnTrack() on this dict right after it's created.
But under 2.7, new code in Python apparently decides to track this
dict again the first time its __setitem__ is called. Cyclic gc then
discovers object references due to this dict that aren't accounted for
in the referenced objects' refcounts, and dies early on with an
assertion failure (which it should do - the refcounts are nuts as far
as Python is concerned).
Jim wormed around that for now by calling PyObject_GC_UnTrack() again
every time the dict's content changes, but that was relatively easy in
this case because the dict is an internal implementation detail all
accesses to which are under ZODB's control.
Best if no changes had been needed. "Better than nothing" if the docs
are changed to warn that the effect of calling PyObject_GC_UnTrack()
may be undone by Python a nanosecond later ;-)
At 07:15 PM 9/25/2010 -0700, Guido van Rossum wrote:
>Don't see this as a new spec. See it as a procedural issue.
As a procedural issue, PEP 333 is an Informational PEP, in "Draft"
status, which I'd like to make Final after these amendments. See
http://www.wsgi.org/wsgi/Amendments_1.0, which Graham created in 2007, stating:
"""This page is intended to collect any ideas related to amendments
to the original WSGI 1.0 so that it can be marked as 'Final'."""
IOW, there is no intention to treat the PEP as "mutable" going
forward; this is just cleanup so we can mark it Final. After that,
it's an ex-parrot.
>Clarifications of ambiguous/unspecified behavior can possibly rule as
>non-conforming implementations that used to get the benefit of the
>doubt. Best-practice recommendations also have the effect of changing
I understand the general principle, but with respect to these
*specific* changes, any perceived-compliance arguments that were
going to happen, already happened years ago. The changes are merely
to officially document the way those arguments already turned out, so
the PEP can become Final.
Specifically, the changes all fall into one of three categories:
1. Textual clarification (SERVER_PORT is not an int, iteration can
stop before all output is consumed)
2. Practical issues with wsgi.input arising from the fact that
real-world programs needed its behavior to be more "file-like" than
the specification required... and which essentially forced servers
that were not using socket.makefile() to make their emulations work
like that, anyway (or else be rejected by users).
3. Clarification of behavior that would break HTTP compliance (apps
or servers sending more than Content-Length bytes) and is therefore
*already a bug* in any implementation that does it.
Since in all three categories any implementation that did not end up
following the recommendations on its own is going to have been
considered buggy by its users (regardless of its formal
"compliance"), and because the changes do not actually declare the
buggy behaviors in categories 2 and 3 to be non-compliant, I do not
see how any of these changes can produce the type of problems you're
worried about here.
Certainly, if I thought such problems were possible, I wouldn't have
accepted these amendments. Likewise, if I thought that changes would
continue to be made to the PEP past this point, the goal wouldn't be
getting it to Final status.
At 02:07 PM 9/25/2010 -0700, Guido van Rossum wrote:
>This is a very laudable initiative and I approve of the changes -- but
>I really think it ought to be a separate PEP rather than pretending it
>is just a set of textual corrections on the existing PEP 333.
With the exception of the bytes change, I ruled out accepting any
proposed amendments that would actually alter the protocol. The
amendments are all either textual clarifications, clarifications of
ambiguous/unspecified areas, or best-practice recommendations by
implementors. (i.e., which are generally already implemented in major servers)
The full list of things Graham and others have asked for or
recommended would indeed require a 1.1 version at minimum, and thus a
new PEP. But I really don't want to start down that road right now,
and therefore hope that I can talk Graham or some other poor soul
into shepherding a 1.1 PEP instead. ;-)
(Seriously: through an ironic twist of fate, I have done nearly
*zero* Python web programming since around the time I drafted the
first spec in 2004, so even if it makes sense for me to finish PEP
333, it makes little sense for me to be starting a *new* one on the topic now!)
I have only done the Python 3-specific changes at this point; the
diff is here if anybody wants to review, nitpick or otherwise comment:
For that matter, if anybody wants to take a crack at updating Python
3's wsgiref based on the above, feel free. ;-) I'll be happy to
answer any questions I can that come up in the process.
(Please note: I went with Ian Bicking's "headers are strings, bodies
are bytes" proposal, rather than my original "bodies and outputs are
bytes" one, as there were not only some good arguments in its favor,
but because it also resulted in fewer changes to the PEP, especially
in the code samples.)
I will continue to work on adding the other addenda/errata mentioned here:
But because these are "shoulds" rather than musts, and apply to both
Python 2 and 3, they are not as high priority for immediate
implementation in wsgiref and do not necessarily need to hold up the
(Nonetheless, if anybody is willing to implement them in the Python 3
version, I will happily review the changes for backport into the
Python 2 standalone version of wsgiref, and issue an updated release
to include them.)