From noamraph at gmail.com  Sun Jan  1 00:08:26 2006
From: noamraph at gmail.com (Noam Raphael)
Date: Sun, 1 Jan 2006 01:08:26 +0200
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <002f01c60e4c$23b97d60$2827a044@oemcomputer>
References: <002f01c60e4c$23b97d60$2827a044@oemcomputer>
Message-ID: <b348a0850512311508s4120da84p748c88ed5d01a2c7@mail.gmail.com>

On 12/31/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> [Noam]
> > For example, iteration over a set which once had
> > 1,000,000 members and now has 2 can take 1,000,000 operations every
> > time you traverse all the (2) elements.
>
> Do you find that to be a common or plausible use case?

I don't have a concrete example in this minute, but a set which is
repeatedly filled with elements and then emptied by pop operations
doesn't seem to me that far-fetched.
>
> Was Guido's suggestion of s=set(s) unworkable for some reason?  dicts
> and sets emphasize fast lookups over fast iteration -- apps requiring
> many iterations over a collection may be better off converting to a list
> (which has no dummy entries or empty gaps between entries).

It's workable, but I think that most Python programmers haven't read
that specific message, and are expecting operations which should take
a short time to take a short time. Converting to a list won't help the
use-case above, and anyway, it's another thing that I wouldn't expect
anyone to do - there's no reason that iteration over a set should take
a long time.

(I'm speaking of my point of view, which I believe is common. I don't
expect programs I write in Python to be super-fast - if that were the
case, I would write them in C. I do expect them to take a reasonable
amount of time, and in the case of iteration over a set, that means a
time proportional to the number of elements in the set.)
>
> Would the case be improved by incurring the time cost of 999,998 tests
> for possible resizing (one for each pop) and some non-trivial number of
> resize operations along the way (each requiring a full-iteration over
> the then current size)?
>
I believe it would. It seems to me that those 999,998 tests take not
much more than a machine clock, which means about 1 milisecond on
todays computers. Those resize operations will take some more
miliseconds. It all doesn't really matter, since probably all other
things will take much more. I now run this code

>>> s = set()
>>> for j in xrange(1000000):
...     s.add(j)
...
>>> while s:
...     tmp = s.pop()
...

And it took 2.4 seconds to finish. And it's okay - I'm just saying
that a few additional clock ticks per operation will usually not
matter when the overall complexity is the same, but changes in order
of complexity can matter much more.

> Even if this unique case could be improved, what is the impact on common
> cases?  Would a downsizing scheme risk thrashing with the
> over-allocation scheme in cases with mixed adds and pops?
>
I think that there shouldn't be additional damage beyond those clock
ticks. The simple method I took from "Introduction to Algorithms"
works no matter what sequence of adds and pops you have.

> Is there any new information/research beyond what has been obvious from
> the moment the dict resizing scheme was born?

I wanted to say that there isn't any new information, and yet I don't
think that I have to assume that everything in current Python is the
best that can be. All I did was finding another reason why a
downsizing scheme might be good, and posting it to ask if people have
thought about it. If you have a document listing all the design
decisions that went into dict implementation, then please send it to
me and I won't ask about things that were already thought about.

But the answer is, yes. I beleive that the current dict resizing
scheme was born before the iterator protocol was introduced, and it
may be a reason why the current scheme doesn't try to minimize the
number of empty hash table entries.

Noam

From fperez.net at gmail.com  Sun Jan  1 01:27:50 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Sat, 31 Dec 2005 17:27:50 -0700
Subject: [Python-Dev] When do sets shrink?
References: <002f01c60e4c$23b97d60$2827a044@oemcomputer>
Message-ID: <dp77m7$qso$1@sea.gmane.org>

Raymond Hettinger wrote:

> Was Guido's suggestion of s=set(s) unworkable for some reason?  dicts
> and sets emphasize fast lookups over fast iteration -- apps requiring
> many iterations over a collection may be better off converting to a list
> (which has no dummy entries or empty gaps between entries).
> 
> Would the case be improved by incurring the time cost of 999,998 tests
> for possible resizing (one for each pop) and some non-trivial number of
> resize operations along the way (each requiring a full-iteration over
> the then current size)?

Note that this is not a comment on the current discussion per se, but rather a
small request/idea in the docs department: I think it would be a really useful
thing to have a summary page/table indicating the complexities of the various
operations on all the builtin types, including at least _mention_ of subtleties
and semi-gotchas.

Python is growing in popularity, and it is being used for more and more
demanding tasks all the time.  Such a 'complexity overview' of the language's
performance would, I think, be very valuable to many.   I know that much of
this information is available, but I'm talking about a specific summary, which
also discusses things like Noam's issue.  

For example, I had never realized that on dicts, for some O(N) operations, N
would mean "largest N in the dict's history" instead of "current number of
elements".  While I'm not arguing for any changes, I think it's good to _know_
this, so I can plan for it if I am ever in a situation where it may be a
problem.

Just my 1e-2.

And Happy New Year to the python-dev team, with many thanks for all your
fantastic work on making the most pleasant, useful programming language out
there.

Cheers,

f


From jcarlson at uci.edu  Sun Jan  1 01:42:19 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sat, 31 Dec 2005 16:42:19 -0800
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512311508s4120da84p748c88ed5d01a2c7@mail.gmail.com>
References: <002f01c60e4c$23b97d60$2827a044@oemcomputer>
	<b348a0850512311508s4120da84p748c88ed5d01a2c7@mail.gmail.com>
Message-ID: <20051231155926.BF5D.JCARLSON@uci.edu>


Noam Raphael <noamraph at gmail.com> wrote:
> 
> On 12/31/05, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> > [Noam]
> > > For example, iteration over a set which once had
> > > 1,000,000 members and now has 2 can take 1,000,000 operations every
> > > time you traverse all the (2) elements.
> >
> > Do you find that to be a common or plausible use case?
> 
> I don't have a concrete example in this minute, but a set which is
> repeatedly filled with elements and then emptied by pop operations
> doesn't seem to me that far-fetched.

It doesn't seem far-fetched, but I've not seen anything like it.  List
appending and popping, yeah, set differences and intersections and
unions, yeah, but not set insertion then removal for large numbers of
items.

Note that you provide insertion into a set then repeated popping as an
example, which is done faster by other methods.

> (I'm speaking of my point of view, which I believe is common. I don't
> expect programs I write in Python to be super-fast - if that were the
> case, I would write them in C. I do expect them to take a reasonable
> amount of time, and in the case of iteration over a set, that means a
> time proportional to the number of elements in the set.)

That is a reasonable point of view.  But realize that depending on the
shrinking strategy, popping/deletion will take ammortized 2+ times
longer than they do now, and whose benefits include (and are basically
limited to): memory cam be freed to the operating system, repeated
iteration over a resized-smaller dictionary can be faster.

> > Would the case be improved by incurring the time cost of 999,998 tests
> > for possible resizing (one for each pop) and some non-trivial number of
> > resize operations along the way (each requiring a full-iteration over
> > the then current size)?
> >
> I believe it would. It seems to me that those 999,998 tests take not
> much more than a machine clock, which means about 1 milisecond on
> todays computers. Those resize operations will take some more
> miliseconds. It all doesn't really matter, since probably all other
> things will take much more. I now run this code
> 
> >>> s = set()
> >>> for j in xrange(1000000):
> ...     s.add(j)
> ...
> >>> while s:
> ...     tmp = s.pop()
> ...
> 
> And it took 2.4 seconds to finish. And it's okay - I'm just saying
> that a few additional clock ticks per operation will usually not
> matter when the overall complexity is the same, but changes in order
> of complexity can matter much more.

Doing that while loop will take _longer_ with a constantly resizing set. 
The only way that resizing a dict/set as it gets smaller will increase
overall running speed is if iteration over the dict/set occurs anywhere
between 2-100 times (depending on the resizing factor)


> > Even if this unique case could be improved, what is the impact on common
> > cases?  Would a downsizing scheme risk thrashing with the
> > over-allocation scheme in cases with mixed adds and pops?
> >
> I think that there shouldn't be additional damage beyond those clock
> ticks. The simple method I took from "Introduction to Algorithms"
> works no matter what sequence of adds and pops you have.

You may get more memory fragmentation, depending on the underlying
memory manager.


> > Is there any new information/research beyond what has been obvious from
> > the moment the dict resizing scheme was born?
> 
> I wanted to say that there isn't any new information, and yet I don't
> think that I have to assume that everything in current Python is the
> best that can be. All I did was finding another reason why a
> downsizing scheme might be good, and posting it to ask if people have
> thought about it. If you have a document listing all the design
> decisions that went into dict implementation, then please send it to
> me and I won't ask about things that were already thought about.

See the source for dictobject.c and dictnotes.txt:
http://svn.python.org/view/python/trunk/Objects/dictobject.c?rev=39608&view=auto
http://svn.python.org/view/python/trunk/Objects/dictnotes.txt?rev=35428&view=auto


 - Josiah


From tim.peters at gmail.com  Sun Jan  1 02:21:23 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 31 Dec 2005 20:21:23 -0500
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <b348a0850512311508s4120da84p748c88ed5d01a2c7@mail.gmail.com>
References: <002f01c60e4c$23b97d60$2827a044@oemcomputer>
	<b348a0850512311508s4120da84p748c88ed5d01a2c7@mail.gmail.com>
Message-ID: <1f7befae0512311721k27ba3582p7ad64d6f37a66f74@mail.gmail.com>

[Noam Raphael]
>>> For example, iteration over a set which once had
>>> 1,000,000 members and now has 2 can take 1,000,000 operations every
>>> time you traverse all the (2) elements.

[Raymond Hettinger]
>> Do you find that to be a common or plausible use case?

[Naom]
> I don't have a concrete example in this minute, but a set which is
> repeatedly filled with elements and then emptied by pop operations
> doesn't seem to me that far-fetched.

Ah, but that's an entirely different example than the one you started
with:  every detail counts when you're looking for "bad" cases.  In
this new example, the set _does_ get resized, as soon as you start
adding elements again.  OTOH, in the absence of repeated iteration
too, it's not clear that this resizing helps more than it hurts.

...

> I wanted to say that there isn't any new information, and yet I don't
> think that I have to assume that everything in current Python is the
> best that can be.

It was in 2005; 2006 is an entirely different year ;-)

> All I did was finding another reason why a downsizing scheme might
> be good, and posting it to ask if people have thought about it.

Not all that much -- sets whose sizes bounce around a lot, and which
are also iterated over a lot, haven't stuck out as an important use
case.  Typically, if a set or dict gets iterated over at all, that
happens once near the end of its life.

> If you have a document listing all the design decisions that went into
> dict implementation, then please send it to me and I won't ask about
> things that were already thought about.

Lots of info in the source; Josiah already pointed at the most useful dict docs.

> But the answer is, yes. I beleive that the current dict resizing
> scheme was born before the iterator protocol was introduced, and it
> may be a reason why the current scheme doesn't try to minimize the
> number of empty hash table entries.

Dict resizing was designed before the Python-level iteration protocol,
but under the covers dicts offered the PyDict_Next() C-level iteration
protocol "forever".  It's not the iteration protocol (or lack thereof)
that drives this.

Far more important is that dicts have always been heavily used by
Python itself, in its own implementation, for a variety of namespaces:
 the global dict, originally the local dict too, for class dicts and
instance dicts, and to pass keyword arguments.  Note that all those
cases use strings for keys, and in fact Python originally supported
only string-keyed dicts.  In all those cases too, deletions are at
worst very rare, and iteration a minor use case.

From raymond.hettinger at verizon.net  Sun Jan  1 02:23:06 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat, 31 Dec 2005 20:23:06 -0500
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <20051231155926.BF5D.JCARLSON@uci.edu>
Message-ID: <000101c60e71$eda5d7c0$e916c797@oemcomputer>

> > > [Noam]
> > > > For example, iteration over a set which once had
> > > > 1,000,000 members and now has 2 can take 1,000,000 operations
every
> > > > time you traverse all the (2) elements.
> > >
> > > Do you find that to be a common or plausible use case?
> >
> > I don't have a concrete example in this minute, but a set which is
> > repeatedly filled with elements and then emptied by pop operations
> > doesn't seem to me that far-fetched.
> 
> It doesn't seem far-fetched, but I've not seen anything like it.

It's more far-fetched when fully spelled-out:

Build a VERY large list, ALMOST empty it with pop operations, then
iterate over it MANY times (enough to offset the cost of multiple resize
operations with their attendant memory allocator interactions and the
expensive block copies (cache misses are a certitude and each miss is as
expensive as a floating point divide)).

Also note, this example was not selected from a real-world use-case; it
was contrived for purposes of supporting an otherwise weak proposal.




> > > Would the case be improved by incurring the time cost of 999,998
tests
> > > for possible resizing (one for each pop) and some non-trivial
number
> of
> > > resize operations along the way (each requiring a full-iteration
over
> > > the then current size)?
> > >
> > I believe it would. It seems to me that those 999,998 tests take not
> > much more than a machine clock, which means about 1 milisecond on
> > todays computers. Those resize operations will take some more
> > miliseconds. It all doesn't really matter, since probably all other
> > things will take much more. I now run this code
> >
> Doing that while loop will take _longer_ with a constantly resizing
set.
> The only way that resizing a dict/set as it gets smaller will increase
> overall running speed is if iteration over the dict/set occurs
anywhere
> between 2-100 times (depending on the resizing factor)

Josiah is exactly correct.  The resize operations are enormously
expensive compared to the cost of an iteration.  You would have to do
the latter many times to make up for the costs of repeatedly downsizing
the set.




> > > Even if this unique case could be improved, what is the impact on
> common
> > > cases?  Would a downsizing scheme risk thrashing with the
> > > over-allocation scheme in cases with mixed adds and pops?
> > >
> > I think that there shouldn't be additional damage beyond those clock
> > ticks. The simple method I took from "Introduction to Algorithms"
> > works no matter what sequence of adds and pops you have.
> 
> You may get more memory fragmentation, depending on the underlying
> memory manager.

There's more a risk than fragmentation.  Trashing is a basic concern.
There is no way around it -- some combination of adds and pops always
triggers it when both upsizing and downsizing logic are present.  The
code in listobject.c works hard to avoid this but there are still
patterns which would trigger horrid behavior with a resize occurring
every few steps.



> > > Is there any new information/research beyond what has been obvious
> from
> > > the moment the dict resizing scheme was born?
> >
> > I wanted to say that there isn't any new information, and yet I
don't
> > think that I have to assume that everything in current Python is the
> > best that can be. All I did was finding another reason why a
> > downsizing scheme might be good, and posting it to ask if people
have
> > thought about it. If you have a document listing all the design
> > decisions that went into dict implementation, then please send it to
> > me and I won't ask about things that were already thought about.
> 
> See the source for dictobject.c and dictnotes.txt:
>
http://svn.python.org/view/python/trunk/Objects/dictobject.c?rev=39608&v
ie
> w=auto
> http://svn

Those are both good references.  

The code for general purpose dicts has been fine-tuned, reviewed, and
field-tested to a highly polished level.  It is at a point where most
attempts to improve it will make it worse-off.

There may be some room for development in special versions of
dictionaries for specific use cases.  For instance, it may be worthwhile
to create a version emphasizing size or access speed over insertion time
(using Brent's variation to optimizing search order).



Raymond


From raymond.hettinger at verizon.net  Sun Jan  1 04:09:22 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat, 31 Dec 2005 22:09:22 -0500
Subject: [Python-Dev] When do sets shrink?
In-Reply-To: <dp77m7$qso$1@sea.gmane.org>
Message-ID: <000201c60e80$c3c35900$e916c797@oemcomputer>

[Fernando Perez]
> Note that this is not a comment on the current discussion per se, but
> rather a
> small request/idea in the docs department: I think it would be a
really
> useful
> thing to have a summary page/table indicating the complexities of the
> various
> operations on all the builtin types, including at least _mention_ of
> subtleties
> and semi-gotchas.

The wiki might be the place to cover this sort of thing.  Unlike
infrequent doc releases, the medium is good at being responsive to
whatever someone thought important enough to write an entry for.  Also,
it is more easily kept up-to-date for variations between versions
(Py2.4, Py2.5, etc.) and implementations (CPython, Jython, etc.).

The relevant list of these ideas may be somewhat short:
* mystring += frag          # use ''.join() instead
* mylist.insert(0, obj)     # takes O(n) time to move all the elements
* if x in y:                # runs in O(n) time if y is a sequence

I think the number one performance gotcha is adopting a COBOL-like code
writing mentality and failing to creatively use Python's powerful
collection types:  list, tuple, dict, set, str and an occasional array,
deque, or cStringIO object.



> For example, I had never realized that on dicts, for some O(N)
operations,
> N
> would mean "largest N in the dict's history" instead of "current
number of
> elements".

It might be better to give more generic advice that tends to be true
across implementations and versions:  "Dense collections like lists
tuples iterate faster than sparse structures like dicts and sets.
Whenever repeated iteration starts to dominate application run-time,
consider converting to a dense representation for faster iteration and
better memory/cache utilization."  A statement like this remains true
whether or not a down-sizing algorithm is present.



> Cheers,
> 
> f

Hmm, your initial may be infringing on another developer's trademarked
signature ;-)



Raymond



Side note:  To some degree, ignorance is bliss.  Most of my code was
written in AWK and I was content to know only one non-algorithmic
optimization ("exp" vs /exp/).  Time was spent thinking about the
problem at hand rather than how to outsmart the interpreter.  Knowing
too much about the implementation can be a distraction.  Besides, when
timing does become critical, people seem to suddenly become
spontaneously ingenious.




From kbk at shore.net  Sun Jan  1 06:08:59 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sun, 1 Jan 2006 00:08:59 -0500 (EST)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200601010508.k0158xMh031318@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  382 open ( +3) /  3003 closed ( +1) /  3385 total ( +4)
Bugs    :  903 open (-11) /  5479 closed (+27) /  6382 total (+16)
RFE     :  203 open ( -1) /   195 closed ( +2) /   398 total ( +1)

New / Reopened Patches
______________________

NotImplemented->TypeError in __add__ and __mul__  (2005-12-26)
CLOSED http://python.org/sf/1390657  opened by  Armin Rigo

dict.merge  (2005-12-27)
       http://python.org/sf/1391204  opened by  Nicolas Lehuen

cookielib LWPCookieJar and MozillaCookieJar exceptions  (2005-02-06)
       http://python.org/sf/1117398  reopened by  jjlee

Optional second argument for startfile  (2005-12-29)
       http://python.org/sf/1393157  opened by  Thomas Heller

Add restart debugger command to pdb.py  (2005-12-30)
       http://python.org/sf/1393667  opened by  Rocky Bernstein

Patches Closed
______________

NotImplemented->TypeError in __add__ and __mul__  (2005-12-26)
       http://python.org/sf/1390657  closed by  arigo

weakref callbacks are called only if the weakref is alive  (2005-12-12)
       http://python.org/sf/1379023  closed by  arigo

New / Reopened Bugs
___________________

Incorrectly docs for return values of set update methods  (2005-12-24)
CLOSED http://python.org/sf/1389673  opened by  Collin Winter

Fxn call in _elementtree.c has incorrect signedness  (2005-12-24)
CLOSED http://python.org/sf/1389809  opened by  Brett Cannon

_winreg specifies EnvironmentError instead of WindowsError  (2005-12-21)
CLOSED http://python.org/sf/1386675  reopened by  birkenfeld

ScrolledText hungs up in some conditions  (2005-12-25)
       http://python.org/sf/1390086  opened by  dani_filth

README mention --without-cxx  (2005-12-25)
       http://python.org/sf/1390321  opened by  Aahz

time docs lack %F in strftime!  (2005-12-26)
CLOSED http://python.org/sf/1390605  opened by  Nikos Kouremenos

split() breaks no-break spaces  (2005-12-26)
CLOSED http://python.org/sf/1390608  opened by  MvR

time.strftime('%F', local_time) is okay but time.strptime no  (2005-12-26)
CLOSED http://python.org/sf/1390629  opened by  Nikos Kouremenos

lambda functions confused when mapped in dictionary object  (2005-12-27)
CLOSED http://python.org/sf/1390991  opened by  Samuel Hsiung

missing module names in email package  (2005-12-27)
       http://python.org/sf/1391608  opened by  Gabriel Genellina

floating point literals don't work in non-US locale in 2.5  (2005-12-28)
       http://python.org/sf/1391872  opened by  Fredrik Lundh

build fails on BSD 3.8  (2005-12-30)
       http://python.org/sf/1392915  opened by  George Yoshida

cannot build SVN trunk on old systems  (2005-12-29)
       http://python.org/sf/1393109  opened by  Fredrik Lundh

Deleting first item causes anydbm.first() to fail  (2005-12-30)
       http://python.org/sf/1394135  opened by  Dan Bisalputra

urllib2 raises exception when page redirects to itself  (2005-12-31)
CLOSED http://python.org/sf/1394453  opened by  Ren?? Pijlman

SimpleHTTPServer doesn't understand query arguments  (2005-12-31)
       http://python.org/sf/1394565  opened by  Aaron Swartz

'Plus' filemode exposes uninitialized memory on win32  (2005-12-31)
       http://python.org/sf/1394612  opened by  Cory Dodt

Bugs Closed
___________

Decimal sqrt() ignores rounding  (2005-12-23)
       http://python.org/sf/1388949  closed by  facundobatista

Incorrect docs for return values of set update methods  (2005-12-24)
       http://python.org/sf/1389673  closed by  birkenfeld

Fxn call in _elementtree.c has incorrect signedness  (2005-12-25)
       http://python.org/sf/1389809  closed by  effbot

_winreg specifies EnvironmentError instead of WindowsError  (2005-12-21)
       http://python.org/sf/1386675  closed by  birkenfeld

time docs lack %F in strftime!  (2005-12-26)
       http://python.org/sf/1390605  closed by  birkenfeld

split() breaks no-break spaces  (2005-12-26)
       http://python.org/sf/1390608  closed by  lemburg

time.strftime('%F', local_time) is okay but time.strptime no  (2005-12-26)
       http://python.org/sf/1390629  closed by  birkenfeld

metaclasses, __getattr__, and special methods  (2003-04-29)
       http://python.org/sf/729913  closed by  arigo

special methods become static  (2004-11-15)
       http://python.org/sf/1066490  closed by  arigo

len() on class broken  (2005-12-16)
       http://python.org/sf/1382740  closed by  arigo

urllib.url2pathname, pathname2url doc strings inconsistent  (2002-12-07)
       http://python.org/sf/649974  closed by  birkenfeld

PyLong_AsVoidPtr()/PyLong_FromVoidPtr()  (2002-12-14)
       http://python.org/sf/653542  closed by  birkenfeld

Acrobat Reader 5 compatibility  (2003-04-14)
       http://python.org/sf/721160  closed by  birkenfeld

Calling socket.recv() with a large number breaks  (2003-06-17)
       http://python.org/sf/756104  closed by  birkenfeld

Automated daily documentation builds  (2002-06-26)
       http://python.org/sf/574241  closed by  birkenfeld

String formatting operator % badly documented  (2003-11-10)
       http://python.org/sf/839585  closed by  birkenfeld

Document that highly recursive data cannot be pickled  (2003-11-10)
       http://python.org/sf/839075  closed by  birkenfeld

os.exec* and first 'arg'  (2003-11-19)
       http://python.org/sf/845342  closed by  birkenfeld

Error in urllib2 Examples  (2004-05-16)
       http://python.org/sf/954981  closed by  birkenfeld

Lib/profile.doc should be updated  (2001-12-05)
       http://python.org/sf/489256  closed by  birkenfeld

exec statement link in index broken  (2005-12-19)
       http://python.org/sf/1385004  closed by  birkenfeld

Tutorial errors  (2005-12-07)
       http://python.org/sf/1375599  closed by  birkenfeld

Dict docstring error Python-2.3.5  (2005-02-26)
       http://python.org/sf/1152424  closed by  birkenfeld

lambda functions confused when mapped in dictionary object  (2005-12-27)
       http://python.org/sf/1390991  closed by  birkenfeld

Can't build Python on POSIX w/o $HOME  (2004-05-24)
       http://python.org/sf/959576  closed by  birkenfeld

Setup.local ignored by setup.py  (2004-07-29)
       http://python.org/sf/999767  closed by  birkenfeld

PyXxx_Check(x) trusts x->ob_type->tp_mro  (2005-02-27)
       http://python.org/sf/1153075  closed by  arigo

urllib2 raises exception when page redirects to itself  (2005-12-31)
       http://python.org/sf/1394453  closed by  rpijlman

New / Reopened RFE
__________________

tempfile misses usecase which requirs renaming  (2005-12-25)
       http://python.org/sf/1390197  opened by  Dennis Allison

RFE Closed
__________

split() string method has two splitting algorithms  (2005-11-28)
       http://python.org/sf/1367936  closed by  rhettinger

future warning in commets  (2005-02-18)
       http://python.org/sf/1144057  closed by  mcherm


From stephen at xemacs.org  Sun Jan  1 20:19:45 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 02 Jan 2006 04:19:45 +0900
Subject: [Python-Dev] [Doc-SIG]   that library reference, again
In-Reply-To: <43B700A2.70701@colorstudy.com> (Ian Bicking's message of "Sat,
	31 Dec 2005 16:05:22 -0600")
References: <dope79$flv$2@sea.gmane.org> <dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org> <43B4A238.2030101@python.org>
	<dp2qt7$dib$1@sea.gmane.org> <43B55FE8.1040401@python.org>
	<43B58F09.6090803@colorstudy.com> <43B61A1E.9080300@gmail.com>
	<43B700A2.70701@colorstudy.com>
Message-ID: <87ek3rpxgu.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Ian" == Ian Bicking <ianb at colorstudy.com> writes:

    Ian> Nick Coghlan wrote:

    >> While I quite like this idea, would it make it more difficult
    >> when the bug tracking for the main source code is eventually
    >> migrated off SF? And what would happen to existing
    >> documentation bug reports/patches on the SF trackers?

    Ian> I think the requirements for documentation are a bit lighter,
    Ian> so it's not as big a deal.  E.g., the history of bug reports
    Ian> on documentation isn't as important, either the ones from SF,
    Ian> or if all of Python moves to a new system then the history of
    Ian> the transitional system.

This is almost true, but

    Ian> Documentation is mostly self-describing.

is not.  Python maintains several "supported" releases
simultaneously.  There are important differences between the Python
language and the implementation being discussed on this list
(cPython).  These issues (and their historical tracking) are not
particularly important in the current workflow because most of the
people actually touching the docs are highly experienced experts.
They will get this stuff right more or less naturally, in part because
they're working on documents that are directly associated with a
particular version of a particular implementation.

But as the documentation process moves to a more fluid, user-driven
process, that expertise will be diluted (even with proposals that the
last phase be edited by a dedicated expert, that expert's time
commitment will surely be strained).  That means that history tracking
will become more important, and documentation meta-data (like which
version of which implementation does this change apply to) much more
so.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From fperez.net at gmail.com  Sun Jan  1 21:06:03 2006
From: fperez.net at gmail.com (Fernando Perez)
Date: Sun, 01 Jan 2006 13:06:03 -0700
Subject: [Python-Dev] When do sets shrink?
References: <dp77m7$qso$1@sea.gmane.org>
	<000201c60e80$c3c35900$e916c797@oemcomputer>
Message-ID: <dp9cnc$cu4$1@sea.gmane.org>

Raymond Hettinger wrote:

> It might be better to give more generic advice that tends to be true
> across implementations and versions:  "Dense collections like lists
> tuples iterate faster than sparse structures like dicts and sets.
> Whenever repeated iteration starts to dominate application run-time,
> consider converting to a dense representation for faster iteration and
> better memory/cache utilization."  A statement like this remains true
> whether or not a down-sizing algorithm is present.

Thanks.  While I certainly wasn't advocating an early optimization approach, I
think that part of using a tool effectively is also knowing its dark corners. 
Sometimes you _do_ need them, so it's handy to have the little 'break the glass
in case of an emergency' box :)

>> Cheers,
>> 
>> f
> 
> Hmm, your initial may be infringing on another developer's trademarked
> signature ;-)

Well, tough.  It happens to be my name, and I've been signing like this since
long before I started using python.  I'll think about changing when the
lawsuits come knocking, if I can't get the EFF to defend me ;-)


Thanks again for your feedback.  Until a suitable wiki comes along, I've kept
your message in my python-info folder as a potentially useful nugget.

Regards,

f 


From martin at v.loewis.de  Mon Jan  2 15:14:04 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 02 Jan 2006 15:14:04 +0100
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
 lines
In-Reply-To: <20051230184534.GC30751@ActiveState.com>
References: <20051230184534.GC30751@ActiveState.com>
Message-ID: <43B9352C.9080800@v.loewis.de>

Trent Mick wrote:
> Is this intentional? If not, could someone point me to where the svn
> trigger scripts are maintained so I could poke around for a fix? (Or
> just fix it themselves. :)

It was not my intention. They are in
dinsdale.python.org:/data/repos/projects

Regards,
Martin

From barry at python.org  Tue Jan  3 00:03:04 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 02 Jan 2006 18:03:04 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41880
	-	python/trunk/Python/Python-ast.c
In-Reply-To: <20060102211856.1D1F61E4002@bag.python.org>
References: <20060102211856.1D1F61E4002@bag.python.org>
Message-ID: <1136242984.2309.109.camel@geddy.wooz.org>

I think we have a fundamental problem with Python-ast.c and
Python-ast.h.  These files should not be both auto-generated and checked
into Subversion.  The problem is that if you do a "make distclean",
these files will get removed and "svn stat" will give you a ! flag.  Of
course, you can "svn up" to get them back, but that just masks the
problem.  

Worse, if you distclean, then configure, then make, it is possible these
files will change.  Say you're working on a patch that touches a lot of
files and you do an "svn commit".  It's certainly possible that you'll
accidentally commit new versions of the Python-ast.* files, possibly
even broken ones.

The general rule should be that no file that is ever generated can be
checked into Subversion.  Probably the right approach is to check in a
template file that will not get removed by a distclean, and modify the
build process to generate Python-ast.* from those template files.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060102/4624949e/attachment.pgp 

From nnorwitz at gmail.com  Tue Jan  3 00:16:19 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 2 Jan 2006 15:16:19 -0800
Subject: [Python-Dev] [Python-checkins] commit of r41880 -
	python/trunk/Python/Python-ast.c
In-Reply-To: <1136242984.2309.109.camel@geddy.wooz.org>
References: <20060102211856.1D1F61E4002@bag.python.org>
	<1136242984.2309.109.camel@geddy.wooz.org>
Message-ID: <ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>

On 1/2/06, Barry Warsaw <barry at python.org> wrote:
> I think we have a fundamental problem with Python-ast.c and
> Python-ast.h.  These files should not be both auto-generated and checked
> into Subversion.

I agree with the problem statement.

> The general rule should be that no file that is ever generated can be
> checked into Subversion.  Probably the right approach is to check in a
> template file that will not get removed by a distclean, and modify the
> build process to generate Python-ast.* from those template files.

I'm not sure about your proposed solution, though.

There's a bootstrapping issue.  Python-ast.[ch] are generated by a
python 2.2+ script.  /f created a bug report if only 2.1 is available.

The Python-ast.[ch] should probably not be removed by distclean.  This
is similar to configure.  Would that make you happy?  What else would
improve the current situation?

If you go the template route, you would just copy the files. That
doesn't seem to gain anything.

n

From barry at python.org  Tue Jan  3 00:22:50 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 02 Jan 2006 18:22:50 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41880
	-	python/trunk/Python/Python-ast.c
In-Reply-To: <ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
References: <20060102211856.1D1F61E4002@bag.python.org>
	<1136242984.2309.109.camel@geddy.wooz.org>
	<ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
Message-ID: <1136244170.2309.112.camel@geddy.wooz.org>

On Mon, 2006-01-02 at 15:16 -0800, Neal Norwitz wrote:

> The Python-ast.[ch] should probably not be removed by distclean.  This
> is similar to configure.  Would that make you happy?  What else would
> improve the current situation?

I think it would, without causing bootstrapping issues.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060102/e981d725/attachment.pgp 

From nnorwitz at gmail.com  Tue Jan  3 02:06:36 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 2 Jan 2006 17:06:36 -0800
Subject: [Python-Dev] buildbot
Message-ID: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>

Here's some info on setting up buildbot
<http://www.python.org/dev/buildbot/>.  These instructions should be
cleaned up and put in the developers faq.  Right now, the only person
that can generate/assign buildbot names/passwords is Martin AFAIK.

We currently have 4 slaves setup:
  Linux: x86 and amd64 (gentoo 2.6, glibc 2.3)
  Solaris10 sparc
  OS X: G5 (not yet working, need to setup path to svn)

It would be nice to get HP-UX, AIX, and Windows in various flavors. 
Ideally, any common platform we support should be added.  This
includes O/S, version, H/W, and various configurations.  Not sure we
care about much more Linux variants.  Note the x86 is the same one I
use for generating doc and more detailed build results.  I'll put that
info into a separate mail.

We also are only running the slaves on the trunk.

Thanks to Martin for setting this up.

n
--

Instructions:

Request a buildbot name and password from python-dev at python.org.
  Please state the machine type, operating system, compiler, and branch.
  For example, "amd64 gentoo gcc trunk" or "powerpc64 aix43 xlC 2.4".

Download buildbot  (we currently use 0.7.1)
Install buildbot (and dependencies like Twisted-1.3 if necesary)
Create buildslave account [optional]

If adding a slave for the trunk:

   # example directory, choose whatever you like
   DIR=`pwd`/buildslave/python-trunk
   mkdir -p $DIR
   cd $DIR
   buildbot slave `pwd` www.python.org:9020 "assigned-name" "assigned-password"

If adding a slave for the 2.4 branch:

   # example directory, choose whatever you like
   DIR=buildslave/python-2.4
   mkdir -p $DIR
   cd $DIR
   buildbot slave `pwd` www.python.org:9024 "assigned-name" "assigned-password"

Edit info/admin and info/host appropriately.  You can use this command
for info/host if you'd like:

   uname -a > info/host

To start the buildbot:

   buildbot start `pwd`

To ensure buildbot is started on reboot:

   (crontab -l | egrep -v '^#' ; echo "@reboot /usr/bin/buildbot start
$DIR") | crontab -

Make sure the buildslave account is able run cron jobs.

From nnorwitz at gmail.com  Tue Jan  3 02:09:08 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 2 Jan 2006 17:09:08 -0800
Subject: [Python-Dev] current test problems
Message-ID: <ee2a432c0601021709u6fadf600h65773f626ae96b9@mail.gmail.com>

test_bsddb3 fails for me with BSD DB 4.1 and 4.2 on gentoo Linux.  I
think it has for a long time.  Is anyone else seeing these failures? 
You must use -u bsddb when running regrtest.

4.2 has 2 failures:

  test02_associateAfterDB
(bsddb.test.test_associate.ShelveAssociateRecnoTestCase)
  test02_associateAfterDB
(bsddb.test.test_associate.ThreadedAssociateRecnoTestCase)

4.1 fails like this:

Traceback (most recent call last):
  File "Lib/threading.py", line 444, in __bootstrap
    self.run()
  File "Lib/threading.py", line 424, in run
    self.__target(*self.__args, **self.__kwargs)
  File "Lib/bsddb/test/test_thread.py", line 275, in readerThread
    rec = dbutils.DeadlockWrap(c.next, max_retries=10)
  File "Lib/bsddb/dbutils.py", line 62, in DeadlockWrap
    return function(*_args, **_kwargs)
DBLockDeadlockError: (-30996, 'DB_LOCK_DEADLOCK: Locker killed to
resolve a deadlock')


When looking for leaks, the following tests fail:

     test_decimal test_difflib test_optparse test_warnings test_zipimport

To test:
     ./python -E -tt ./Lib/test/regrtest.py -R 3:2: \
       test_decimal test_difflib test_optparse test_warnings test_zipimport

The following test has actual leaks which was reported a while ago.  Armin
found the problem to be that itertools.tee needs to participate with GC IIRC:

      test_generators

The following tests do not report stable leaks (at least sometimes):

    test_capi test_cfgparser test_charmapcodec
    test_cmd_line test_compiler test_filecmp test_logging
    test_threadedtempfile test_threading
    test_threading_local test_urllib2

n

From jeremy at alum.mit.edu  Tue Jan  3 02:23:20 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 2 Jan 2006 20:23:20 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41880 -
	python/trunk/Python/Python-ast.c
In-Reply-To: <ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
References: <20060102211856.1D1F61E4002@bag.python.org>
	<1136242984.2309.109.camel@geddy.wooz.org>
	<ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
Message-ID: <e8bf7a530601021723k7e194b99he2ec1ef03bf53443@mail.gmail.com>

On 1/2/06, Neal Norwitz <nnorwitz at gmail.com> wrote:
> On 1/2/06, Barry Warsaw <barry at python.org> wrote:
> > I think we have a fundamental problem with Python-ast.c and
> > Python-ast.h.  These files should not be both auto-generated and checked
> > into Subversion.
>
> I agree with the problem statement.
>
> > The general rule should be that no file that is ever generated can be
> > checked into Subversion.  Probably the right approach is to check in a
> > template file that will not get removed by a distclean, and modify the
> > build process to generate Python-ast.* from those template files.
>
> I'm not sure about your proposed solution, though.
>
> There's a bootstrapping issue.  Python-ast.[ch] are generated by a
> python 2.2+ script.  /f created a bug report if only 2.1 is available.
>
> The Python-ast.[ch] should probably not be removed by distclean.  This
> is similar to configure.  Would that make you happy?  What else would
> improve the current situation?

I think this solution is better.  It's relatively rare for people to
change the ast definition, so for most purposes these should be static
files.

Jeremy

From nnorwitz at gmail.com  Tue Jan  3 02:27:13 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 2 Jan 2006 17:27:13 -0800
Subject: [Python-Dev] automated builds and tests
Message-ID: <ee2a432c0601021727n461af2e6x7f5843fba5d82142@mail.gmail.com>

I've updated the script that builds python and the docs, runs the
tests, and copies everything up to docs.python.org.  Currently, this
only works for the trunk.  I'm not sure if there's much of a benefit
to set this up for branch(es) also.

The docs are here:
  http://docs.python.org/dev/

The results of the build are here:
  http://docs.python.org/dev/results/

The script is checked in to Misc/build.sh.

If the script is modified, it doesn't pick up the changes until the
next run.  There is a hook to pickup the changes on each run, but I
didn't finish that.  Look for one of the several FIXMEs.

A brief description of what the script does is:
  svn up ; ./configure ; make ; make test ; make install ; cd Doc ; make ; rsync

Below is a description of everything that's done in the script.  Most
commands are checked for errors and the results are emailed iff there
are problems.  Right now I'm the only one that gets spammed.  This ran
all last week without any problems.  I made a bunch of changes and
probably sometime this week, I'll change the mail to go to
python-checkins.

svn update
svn status
./configure --prefix=$INSTALL_DIR --with-pydebug
make | grep warning
if make fails, goto building docs
make install
make test | grep fail
./python ./Lib/test/regrtest.py -R 4:3:$REFLOG -u network | grep leak
./python -E -tt ./Lib/test/regrtest.py -uall -x test_curses
test_linuxaudiodev test_ossaudiodev | grep fail

cd Doc ; make

rsync doc
rsync build results

---

test_curses is ignored because this script is run from cron and there
is no terminal.  The test stops prematurely.  The sound tests are
ignored because the sound device is not configured on this computer.

Some of the failure counts are off because it does a simple grep and
make test runs everything twice.  I've generally tried to filter out
stuff we don't care about.

There is potential for the tests to fail due to them running at the
same time as the buildbot test run.  These tests are run every 12
hours.  It currently takes over 1.25 hours to build and run
everything.  There is no make clean step for either the code or doc,
perhaps there should be?

n

From skip at pobox.com  Tue Jan  3 04:12:27 2006
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 2 Jan 2006 21:12:27 -0600
Subject: [Python-Dev] buildbot
In-Reply-To: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
Message-ID: <17337.60315.618431.480677@montanaro.dyndns.org>

    Neal> We currently have 4 slaves setup:
    Neal>   Linux: x86 and amd64 (gentoo 2.6, glibc 2.3)
    Neal>   Solaris10 sparc
    Neal>   OS X: G5 (not yet working, need to setup path to svn)

The G5 *was* working.  I changed nothing at my end.  Got a mail yesterday
from Martin.  It looks like PATH lost /usr/local/bin (where the Metissian
installer puts the svn executable).  I added a /usr/bin/svn symlink, but
would really rather see /usr/local/bin in PATH.

Skip

From nnorwitz at gmail.com  Tue Jan  3 07:30:01 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 2 Jan 2006 22:30:01 -0800
Subject: [Python-Dev] buildbot
In-Reply-To: <17337.60315.618431.480677@montanaro.dyndns.org>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
Message-ID: <ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>

On 1/2/06, skip at pobox.com <skip at pobox.com> wrote:
>
> The G5 *was* working.  I changed nothing at my end.  Got a mail yesterday
> from Martin.  It looks like PATH lost /usr/local/bin (where the Metissian
> installer puts the svn executable).  I added a /usr/bin/svn symlink, but
> would really rather see /usr/local/bin in PATH.

It seems reasonable for /usr/local/bin to be in PATH.  But isn't this
be done on the slave (ie your) side?  It looks like the PATH comes
from your env't based on how my PATH in twistd.log.  So if you started
this from cron, you may need to add it there.  If nothing else works,
you could probably update PATH in buildbot.tac.

I forced a build, but it failed since svnversion was not found.

BTW, I noticed this warning:
  Modules/getpath.c:405: warning: passing arg 2 of
`_NSGetExecutablePath' from incompatible pointer type

Can somebody with a Mac fix this warning?

n

From bob at redivi.com  Tue Jan  3 07:39:25 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 2 Jan 2006 22:39:25 -0800
Subject: [Python-Dev] buildbot
In-Reply-To: <ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
Message-ID: <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>


On Jan 2, 2006, at 10:30 PM, Neal Norwitz wrote:

> On 1/2/06, skip at pobox.com <skip at pobox.com> wrote:
>>
>> The G5 *was* working.  I changed nothing at my end.  Got a mail  
>> yesterday
>> from Martin.  It looks like PATH lost /usr/local/bin (where the  
>> Metissian
>> installer puts the svn executable).  I added a /usr/bin/svn  
>> symlink, but
>> would really rather see /usr/local/bin in PATH.
>
> It seems reasonable for /usr/local/bin to be in PATH.  But isn't this
> be done on the slave (ie your) side?  It looks like the PATH comes
> from your env't based on how my PATH in twistd.log.  So if you started
> this from cron, you may need to add it there.  If nothing else works,
> you could probably update PATH in buildbot.tac.
>
> I forced a build, but it failed since svnversion was not found.
>
> BTW, I noticed this warning:
>   Modules/getpath.c:405: warning: passing arg 2 of
> `_NSGetExecutablePath' from incompatible pointer type
>
> Can somebody with a Mac fix this warning?

This warning depends on the version of Mac OS X you're using.  The  
type signature for arg 2 is "unsigned long" up to 10.3, and  
"uint32_t" on 10.4.  The version I see in svn is current with Mac OS  
X 10.4.

The easy fix is to upgrade your OS.  I don't think anyone is going to  
bother with the preprocessor hackery necessary to make that  
(harmless) warning go away on older versions of the OS.

-bob


From ianb at colorstudy.com  Tue Jan  3 08:08:26 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 03 Jan 2006 01:08:26 -0600
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <43B58F09.6090803@colorstudy.com>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<dp2qt7$dib$1@sea.gmane.org>	<43B55FE8.1040401@python.org>
	<43B58F09.6090803@colorstudy.com>
Message-ID: <43BA22EA.5010603@colorstudy.com>

I've put an instance of Commentary up against the full 2.4 
documentation: http://pythonpaste.org/comment/python24/

It writes to a Subversion repository so you can see what the backend 
data looks like: http://pythonpaste.org/comment/svn/python24/ -- not 
much there yet.  Obviously things like notification and reports would be 
useful, but I don't know what the state of the art for Subversion is 
either, so potentially these tools already exist.  But last I really 
looked at the tools for Subversion, I wasn't overly impressed.  Things 
like self-signup for pages you were interested in would be useful.  But 
anyway, there's nothing too difficult about those features.

Anyway, it's just meant as a demo for now, but give it a look, and if 
you have problems or suggestions write a ticket: 
http://pythonpaste.org/commentary/trac/newticket

Cheers.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From nnorwitz at gmail.com  Tue Jan  3 08:58:31 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 2 Jan 2006 23:58:31 -0800
Subject: [Python-Dev] mac memory leaks
Message-ID: <ee2a432c0601022358sb787c68o938ac08f950b9715@mail.gmail.com>

I had a friend run regrtest -L on Mac OSX a while ago.  There are
several memory leaks which still appear to be an issue.  There are a
bunch of leaks reported from  putenv which I'm not sure how to fix

The attached patch should correct one of the problems.  Can someone
with a Mac test it?  I'll add to SF later if I don't get a response.

n
--

I'm not sure about how to fix these or if they are real problems:

Leak: 0x02541470  size=368      instance of 'NSCFData'
Call stack: call_function | ici_ICFindPrefHandle | ICFindPrefHandle |
OpaqueICInstance::FindPrefHandle(ICKeyInfo const&, unsigned long*,
char**) | OpaqueICInstance::GetPref(ICKeyInfo const&, long, void*,
long*, unsigned long*) | OpaqueICInstance::GetPrefCore(ICKeyInfo
const&, long, void*, long*, unsigned long*) |
OpaqueICInstance::LazyLoad() | ICCFPrefWrapper::ContainsPrefs() |
ICCFPrefWrapper::GetPrefDictionary() | fetchXMLValue |
_loadXMLDomainIfStale | _CFPropertyListCreateFromXMLData |
__CFTryParseBinaryPlist | __CFBinaryPlistCreateObject |
__CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject |
__CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject |
__CFDataInit | _CFRuntimeCreateInstance | CFAllocatorAllocate

Leak: 0x02506640  size=256
Call stack: call_function | do_call | PyObject_Call | parser_st2tuple
| node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
node2tuple | PyTuple_New | _PyObject_GC_NewVar | _PyObject_GC_Malloc |
PyObject_Malloc | new_arena

Leak: 0x0118ad10  size=80
Call stack: call_function | AE_AECreateDesc | AECreateDesc | operator
new(unsigned long)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: mac-leak.diff
Type: text/x-patch
Size: 812 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060102/49128aa2/attachment.bin 

From martin at v.loewis.de  Tue Jan  3 09:23:23 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jan 2006 09:23:23 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
Message-ID: <43BA347B.90002@v.loewis.de>

Neal Norwitz wrote:
> If adding a slave for the trunk:
[...]
> If adding a slave for the 2.4 branch:

Currently, branches are not supported, because buildbot is
somewhat limited. When I get a solution for this problem,
I plan to have all buildbots build both the trunk and the
2.4 branch; such builds would only be initiated whenever
something is committed on the branch.

> To ensure buildbot is started on reboot:
> 
>    (crontab -l | egrep -v '^#' ; echo "@reboot /usr/bin/buildbot start
> $DIR") | crontab -

AFAICT, this assumes Vixie cron, which is fine for Linux,
but I don't think it would work on, say, Solaris (atleast
I couldn't find anything in the documentation of Solaris
cron that suggests "@reboot" lines are supported).

Regards,
Martin

From martin at v.loewis.de  Tue Jan  3 09:38:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jan 2006 09:38:59 +0100
Subject: [Python-Dev] [Python-checkins] commit of r41880
	-	python/trunk/Python/Python-ast.c
In-Reply-To: <e8bf7a530601021723k7e194b99he2ec1ef03bf53443@mail.gmail.com>
References: <20060102211856.1D1F61E4002@bag.python.org>	<1136242984.2309.109.camel@geddy.wooz.org>	<ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
	<e8bf7a530601021723k7e194b99he2ec1ef03bf53443@mail.gmail.com>
Message-ID: <43BA3823.9000305@v.loewis.de>

Jeremy Hylton wrote:
> I think this solution is better.  It's relatively rare for people to
> change the ast definition, so for most purposes these should be static
> files.

Interestingly enough, I found yesterday that Python-ast.c did change for
me, even though I had not touched the AST grammar at all. Assuming
somebody forgot to commit this file, I just did.

I now investigated where this change originated from, and it was this
commit

r41812 | tim.peters | 2005-12-26 00:18:31 +0100 (Mo, 26 Dez 2005) | 2 lines

Whitespace normalization.

This indicates two problems: apparently, the whitespace normalization
also normalizes string literals; not sure whether this is a good idea
or not. Furthermore, people should be made aware that something
deserves their attention when they find that Python-ast.c has changed
for them (but I guess Tim never saw Python-ast.c change, because this
build step isn't executed on Windows).

Regards,
Martin

From abo at minkirri.apana.org.au  Tue Jan  3 13:13:23 2006
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Tue, 03 Jan 2006 12:13:23 +0000
Subject: [Python-Dev] [Python-checkins] commit of r41880
	-	python/trunk/Python/Python-ast.c
In-Reply-To: <ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
References: <20060102211856.1D1F61E4002@bag.python.org>
	<1136242984.2309.109.camel@geddy.wooz.org>
	<ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
Message-ID: <1136290403.18340.22.camel@warna.dub.corp.google.com>

On Mon, 2006-01-02 at 15:16 -0800, Neal Norwitz wrote:
> On 1/2/06, Barry Warsaw <barry at python.org> wrote:
> > I think we have a fundamental problem with Python-ast.c and
> > Python-ast.h.  These files should not be both auto-generated and checked
> > into Subversion.
> 
> I agree with the problem statement.
> 
> > The general rule should be that no file that is ever generated can be
> > checked into Subversion.  Probably the right approach is to check in a
> > template file that will not get removed by a distclean, and modify the
> > build process to generate Python-ast.* from those template files.
> 
> I'm not sure about your proposed solution, though.
> 
> There's a bootstrapping issue.  Python-ast.[ch] are generated by a
> python 2.2+ script.  /f created a bug report if only 2.1 is available.
> 
> The Python-ast.[ch] should probably not be removed by distclean.  This
> is similar to configure.  Would that make you happy?  What else would
> improve the current situation?
> 
> If you go the template route, you would just copy the files. That
> doesn't seem to gain anything.

The solution I use is to never have anything auto-generated in CVS/SVN,
but have "make dist" generate and include anything needed for
bootstrapping in the distribution tarball (or whatever). Doing "make
distclean" should delete enough to bring you back to a freshly extracted
distribution tarball, and "make maintainer-clean" should delete all
auto-generated files to bring you back to a clean CVS/SVN checkout.

I tend to include quite a few generated files in the distribution
tarball that are not in CVS/RCS. Things like ChangeList (generated by
cvs2cl), all the autotools autogen'ed files, generated datafiles, etc.

This way your source distributions don't have any bootstrap problems,
but you also don't have any auto-generated files in CVS/SVN and the
problems they create. It does however mean that a fresh CVS/SVN checkout
does have additional build requirements above and beyond building from a
source distribution.

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From Jack.Jansen at cwi.nl  Tue Jan  3 15:55:38 2006
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue, 3 Jan 2006 15:55:38 +0100
Subject: [Python-Dev] mac memory leaks
In-Reply-To: <ee2a432c0601022358sb787c68o938ac08f950b9715@mail.gmail.com>
References: <ee2a432c0601022358sb787c68o938ac08f950b9715@mail.gmail.com>
Message-ID: <357EDE66-9CE4-4E11-99EC-565D7E6B239D@cwi.nl>


On 3 Jan 2006, at 08:58, Neal Norwitz wrote:

> I had a friend run regrtest -L on Mac OSX a while ago.  There are
> several memory leaks which still appear to be an issue.  There are a
> bunch of leaks reported from  putenv which I'm not sure how to fix
>
> The attached patch should correct one of the problems.  Can someone
> with a Mac test it?  I'll add to SF later if I don't get a response.
>
> n
> --
>
> I'm not sure about how to fix these or if they are real problems:
>
> Leak: 0x02541470  size=368      instance of 'NSCFData'
> Call stack: call_function | ici_ICFindPrefHandle | ICFindPrefHandle |
> OpaqueICInstance::FindPrefHandle(ICKeyInfo const&, unsigned long*,
> char**) | OpaqueICInstance::GetPref(ICKeyInfo const&, long, void*,
> long*, unsigned long*) | OpaqueICInstance::GetPrefCore(ICKeyInfo
> const&, long, void*, long*, unsigned long*) |
> OpaqueICInstance::LazyLoad() | ICCFPrefWrapper::ContainsPrefs() |
> ICCFPrefWrapper::GetPrefDictionary() | fetchXMLValue |
> _loadXMLDomainIfStale | _CFPropertyListCreateFromXMLData |
> __CFTryParseBinaryPlist | __CFBinaryPlistCreateObject |
> __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject |
> __CFBinaryPlistCreateObject | __CFBinaryPlistCreateObject |
> __CFDataInit | _CFRuntimeCreateInstance | CFAllocatorAllocate

This is somewhere down in the Internet Config builtin module, which  
would point to the webbrowser library module. My guess is that it's a  
singleton leak.
>
> Leak: 0x02506640  size=256
> Call stack: call_function | do_call | PyObject_Call | parser_st2tuple
> | node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
> node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
> node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
> node2tuple | node2tuple | node2tuple | node2tuple | node2tuple |
> node2tuple | PyTuple_New | _PyObject_GC_NewVar | _PyObject_GC_Malloc |
> PyObject_Malloc | new_arena

No idea. There don't seem to be any mac-specific modules involved...

> Leak: 0x0118ad10  size=80
> Call stack: call_function | AE_AECreateDesc | AECreateDesc | operator
> new(unsigned long)

Hmm, the first candidates here would be test_aepack and  
test_scriptpackages, but neither one has an obvious leak (on cursory  
inspection). And actually, if there was a leak problem in either I  
would expect more than one AEDesc to be leaked.
--
Jack Jansen, <Jack.Jansen at cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma  
Goldman



From skip at pobox.com  Tue Jan  3 16:39:03 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 3 Jan 2006 09:39:03 -0600
Subject: [Python-Dev] Including zlib...
In-Reply-To: <20060103064313.DA3A21E4002@bag.python.org>
References: <20060103064313.DA3A21E4002@bag.python.org>
Message-ID: <17338.39575.904122.821741@montanaro.dyndns.org>


Martin checked in zlib to the Python svn repository.  Are we really sure
that including zlib is the only path to whatever it is that it achieves?  If
security holes in zlib turn up (they have in the past), new Python releases
will have to be released quickly.

Skip


From skip at pobox.com  Tue Jan  3 16:46:52 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 3 Jan 2006 09:46:52 -0600
Subject: [Python-Dev] buildbot
In-Reply-To: <727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
Message-ID: <17338.40044.896652.583651@montanaro.dyndns.org>


    Bob> The easy fix is to upgrade your OS.  I don't think anyone is going
    Bob> to bother with the preprocessor hackery necessary to make that
    Bob> (harmless) warning go away on older versions of the OS.

Excuse me, but this really pisses me off.  I delayed upgrading from 10.2 for
awhile and was given the same "advice".  I was further told (check the mac
sig archives) that "you'd be able to stick with 10.3 for much longer"
because new apps wouldn't need to require 10.4.  So I upgraded.  Now you're
telling me that it's somehow "obsolete" and that I should upgrade because
"we can't be bothered to fix a compiler warning"?  Python supports much
older versions of other platforms.  What makes Mac OSX so special in this
regard?

Skip

From theller at python.net  Tue Jan  3 16:58:37 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 03 Jan 2006 16:58:37 +0100
Subject: [Python-Dev] API changes
Message-ID: <4q4l9uc2.fsf@python.net>

The ast-branch merge apparently changed some formerly public functions
to macros.  The two that I found out about are PyRun_SimpleString and
PyRun_InteractiveLoop, I have not checked if there are more or not.

This breaks apps which dynamically link at runtime to the Python dll
(the latest py2exe does this).

Was this change intentional, or can it be reverted?

Thanks,

Thomas


From ronaldoussoren at mac.com  Tue Jan  3 17:04:05 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Tue, 3 Jan 2006 17:04:05 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <17338.40044.896652.583651@montanaro.dyndns.org>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
Message-ID: <BA31271E-3623-44B7-BEDD-1E4AE5838B3C@mac.com>


On 3-jan-2006, at 16:46, skip at pobox.com wrote:

>
>     Bob> The easy fix is to upgrade your OS.  I don't think anyone  
> is going
>     Bob> to bother with the preprocessor hackery necessary to make  
> that
>     Bob> (harmless) warning go away on older versions of the OS.
>
> Excuse me, but this really pisses me off.  I delayed upgrading from  
> 10.2 for
> awhile and was given the same "advice".  I was further told (check  
> the mac
> sig archives) that "you'd be able to stick with 10.3 for much longer"
> because new apps wouldn't need to require 10.4.  So I upgraded.   
> Now you're
> telling me that it's somehow "obsolete" and that I should upgrade  
> because
> "we can't be bothered to fix a compiler warning"?  Python supports  
> much
> older versions of other platforms.  What makes Mac OSX so special  
> in this
> regard?

This is a completely harmless warning, but I agree it is annoying  
that older
versions of the OS are neglected.

At least for me, supporting older versions of OSX is hard. Both my  
system at
work and my private system run 10.4 by default, which means I'll have  
to reboot
and leave my normal work environment to work on older versions.  
Thanks to the
age of my machines "older versions" is limited to 10.3.0 or later,  
neither machine
can boot 10.2.

Ronald

>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com


From jeremy at alum.mit.edu  Tue Jan  3 17:18:56 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 3 Jan 2006 11:18:56 -0500
Subject: [Python-Dev] API changes
In-Reply-To: <4q4l9uc2.fsf@python.net>
References: <4q4l9uc2.fsf@python.net>
Message-ID: <e8bf7a530601030818w29e5c775r7967337fd1ee0f26@mail.gmail.com>

The intent was to provide binary compatibility, but redirect all newly
linked code to the newer variants.  We did this correctly for
PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do
it for the rest of the changed functions.  Can you file a bug report? 
(Or just fix the ones that bother you.)

We ought to update the docs, too.

Jeremy

On 1/3/06, Thomas Heller <theller at python.net> wrote:
> The ast-branch merge apparently changed some formerly public functions
> to macros.  The two that I found out about are PyRun_SimpleString and
> PyRun_InteractiveLoop, I have not checked if there are more or not.
>
> This breaks apps which dynamically link at runtime to the Python dll
> (the latest py2exe does this).
>
> Was this change intentional, or can it be reverted?
>
> Thanks,
>
> Thomas
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From stephen at xemacs.org  Tue Jan  3 17:27:43 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 04 Jan 2006 01:27:43 +0900
Subject: [Python-Dev] buildbot
In-Reply-To: <17338.40044.896652.583651@montanaro.dyndns.org> (skip@pobox.com's
	message of "Tue, 3 Jan 2006 09:46:52 -0600")
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
Message-ID: <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "skip" == skip  <skip at pobox.com> writes:

    Bob> The easy fix is to upgrade your OS.  I don't think anyone is going
    Bob> to bother with the preprocessor hackery necessary to make that
    Bob> (harmless) warning go away on older versions of the OS.

    skip> Excuse me, but this really pisses me off.

Oh, c'mon, Skip.  I'm a Panther hold-out, too, but this is just a
(soi-disant harmless) warning.  Have you tried to build anything from
DarwinPorts recently?  I have at least 20 modified PortFiles plus at
least two patched source files because they can't be bothered to make
sure things _build_ on Panther, let alone eliminate warnings.

Sure, Python can be expected to do much better than random port
maintainers, but....  Discussions with the port maintainers makes it
hard to blame them.  They say the development environment has changed
enormously---you really need multiple systems because of significant
changes in the standard headers, it's nowhere near enough to just keep
a couple or six versions of GCC around.  And you need multiple dev
environments, because Xcode 1.5 doesn't cut it on Tiger, and Xcode 2.x
doesn't work at all on Panther.

Yes, it would be nice to make the warnings go away, but in view of all
that, is it something to get one's shorts in a knot about?

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From theller at python.net  Tue Jan  3 17:44:47 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 03 Jan 2006 17:44:47 +0100
Subject: [Python-Dev] API changes
References: <4q4l9uc2.fsf@python.net>
	<e8bf7a530601030818w29e5c775r7967337fd1ee0f26@mail.gmail.com>
Message-ID: <wthh8dmo.fsf@python.net>

Jeremy Hylton <jeremy at alum.mit.edu> writes:
> On 1/3/06, Thomas Heller <theller at python.net> wrote:
>> The ast-branch merge apparently changed some formerly public functions
>> to macros.  The two that I found out about are PyRun_SimpleString and
>> PyRun_InteractiveLoop, I have not checked if there are more or not.
>>
>> This breaks apps which dynamically link at runtime to the Python dll
>> (the latest py2exe does this).
>>
>> Was this change intentional, or can it be reverted?
>>
>> Thanks,
>>
>> Thomas
>>

> The intent was to provide binary compatibility, but redirect all newly
> linked code to the newer variants.  We did this correctly for
> PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do
> it for the rest of the changed functions.  Can you file a bug report? 
> (Or just fix the ones that bother you.)
>
> We ought to update the docs, too.

I'm not sure I understand the 'we did this correctly for ...'.
PyParser_SimpleParseFile is a function in 2.4, and a macro in 2.5.

Thomas


From jeremy at alum.mit.edu  Tue Jan  3 18:40:14 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 3 Jan 2006 12:40:14 -0500
Subject: [Python-Dev] API changes
In-Reply-To: <wthh8dmo.fsf@python.net>
References: <4q4l9uc2.fsf@python.net>
	<e8bf7a530601030818w29e5c775r7967337fd1ee0f26@mail.gmail.com>
	<wthh8dmo.fsf@python.net>
Message-ID: <e8bf7a530601030940i53a6482eq636083ad17dc0958@mail.gmail.com>

On 1/3/06, Thomas Heller <theller at python.net> wrote:
> Jeremy Hylton <jeremy at alum.mit.edu> writes:
> > On 1/3/06, Thomas Heller <theller at python.net> wrote:
> >> The ast-branch merge apparently changed some formerly public functions
> >> to macros.  The two that I found out about are PyRun_SimpleString and
> >> PyRun_InteractiveLoop, I have not checked if there are more or not.
> >>
> >> This breaks apps which dynamically link at runtime to the Python dll
> >> (the latest py2exe does this).
> >>
> >> Was this change intentional, or can it be reverted?
> >>
> >> Thanks,
> >>
> >> Thomas
> >>
>
> > The intent was to provide binary compatibility, but redirect all newly
> > linked code to the newer variants.  We did this correctly for
> > PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do
> > it for the rest of the changed functions.  Can you file a bug report?
> > (Or just fix the ones that bother you.)
> >
> > We ought to update the docs, too.
>
> I'm not sure I understand the 'we did this correctly for ...'.
> PyParser_SimpleParseFile is a function in 2.4, and a macro in 2.5.

There's a function by the same name at the end of pythonrun.c. 
Nothing in the Python core should link to it, but it's still available
for third-party code.  Now we can mark PyParser_SimpleParseFile as
deprecated remove it in some future release.

Jeremy

From theller at python.net  Tue Jan  3 19:18:21 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 03 Jan 2006 19:18:21 +0100
Subject: [Python-Dev] API changes
References: <4q4l9uc2.fsf@python.net>
	<e8bf7a530601030818w29e5c775r7967337fd1ee0f26@mail.gmail.com>
	<wthh8dmo.fsf@python.net>
	<e8bf7a530601030940i53a6482eq636083ad17dc0958@mail.gmail.com>
Message-ID: <fyo589aq.fsf@python.net>

Jeremy Hylton <jeremy at alum.mit.edu> writes:

> On 1/3/06, Thomas Heller <theller at python.net> wrote:
>> Jeremy Hylton <jeremy at alum.mit.edu> writes:
>> > On 1/3/06, Thomas Heller <theller at python.net> wrote:
>> >> The ast-branch merge apparently changed some formerly public functions
>> >> to macros.  The two that I found out about are PyRun_SimpleString and
>> >> PyRun_InteractiveLoop, I have not checked if there are more or not.
>> >>
>> >> This breaks apps which dynamically link at runtime to the Python dll
>> >> (the latest py2exe does this).
>>
>> > The intent was to provide binary compatibility, but redirect all newly
>> > linked code to the newer variants.  We did this correctly for
>> > PyParser_SimpleParseFile and PyParser_SimpleParseString, but didn't do
>> > it for the rest of the changed functions.  Can you file a bug report?
>> > (Or just fix the ones that bother you.)
>>
>> I'm not sure I understand the 'we did this correctly for ...'.
>> PyParser_SimpleParseFile is a function in 2.4, and a macro in 2.5.
>
> There's a function by the same name at the end of pythonrun.c. 
> Nothing in the Python core should link to it, but it's still available
> for third-party code.  Now we can mark PyParser_SimpleParseFile as
> deprecated remove it in some future release.

I see.  I doesn't help on Windows, though, since the functions are not
declared DL_EXPORT().

All in all, I'm unsure if it is worth fixing this - maybe it would be
better to change py2exe.

Thomas


From tim.peters at gmail.com  Tue Jan  3 19:38:03 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 3 Jan 2006 13:38:03 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41906 -
	python/branches/ssize_t/Objects/obmalloc.c
In-Reply-To: <20060103131655.201471E400A@bag.python.org>
References: <20060103131655.201471E400A@bag.python.org>
Message-ID: <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com>

> Author: martin.v.loewis
> Date: Tue Jan  3 14:16:53 2006
> New Revision: 41906
>
> Modified:
>    python/branches/ssize_t/Objects/obmalloc.c
> Log:
> Disable 32-bit size limitation for 64-bit mode.
>
>
> Modified: python/branches/ssize_t/Objects/obmalloc.c
> ==============================================================================
> --- python/branches/ssize_t/Objects/obmalloc.c  (original)
> +++ python/branches/ssize_t/Objects/obmalloc.c  Tue Jan  3 14:16:53 2006
> @@ -1005,6 +1005,8 @@
>
>         bumpserialno();
>         total = nbytes + 16;
> +#if SIZEOF_SIZE_T < 8
> +       /* XXX do this check only on 32-bit machines */
>         if (total < nbytes || (total >> 31) > 1) {
>                 /* overflow, or we can't represent it in 4 bytes */
>                 /* Obscure:  can't do (total >> 32) != 0 instead, because
> @@ -1013,6 +1015,7 @@
>                    size_t is an unsigned type. */
>                 return NULL;
>         }
> +#endif

This checkin should be reverted for now.  It's in
_PyObject_DebugMalloc, and at present the layout of the extra debug
info in a PYMALLOC_DEBUG build is hard-coded to use 4-byte fields, no
matter what sizeof(size_t) may be.  One of the extra fields recorded
in a PYMALLOC_DEBUG build is the number of bytes requested, and at
present it's simply not capable of recording a value that doesn't fit
in 4 bytes.

Even after (if ever ;-)) this is changed to support recording 8-byte
values on a box where sizeof(size_t) == 8, the "total < nbytes" part
of the test would still be appropriate:  PyObject_DebugMalloc requests
more memory (`total`) than the user asked for (`nbytes`), and the
computation of `total` may have overflowed.  That's what "total <
nbytes" is checking, and that's the right way to spell the overflow
check on any box.

From tim.peters at gmail.com  Tue Jan  3 19:58:52 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 3 Jan 2006 13:58:52 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41880 -
	python/trunk/Python/Python-ast.c
In-Reply-To: <43BA3823.9000305@v.loewis.de>
References: <20060102211856.1D1F61E4002@bag.python.org>
	<1136242984.2309.109.camel@geddy.wooz.org>
	<ee2a432c0601021516u45ef9ea1v72615bab35f7683d@mail.gmail.com>
	<e8bf7a530601021723k7e194b99he2ec1ef03bf53443@mail.gmail.com>
	<43BA3823.9000305@v.loewis.de>
Message-ID: <1f7befae0601031058h6f5b1615tbea25bf7acc4835c@mail.gmail.com>

[Jeremy Hylton]
>> I think this solution is better.  It's relatively rare for people to
>> change the ast definition, so for most purposes these should be static
>> files.

[Martin v. L?wis]
> Interestingly enough, I found yesterday that Python-ast.c did change for
> me, even though I had not touched the AST grammar at all. Assuming
> somebody forgot to commit this file, I just did.
>
> I now investigated where this change originated from, and it was this
> commit
>
> r41812 | tim.peters | 2005-12-26 00:18:31 +0100 (Mo, 26 Dez 2005) | 2 lines
>
> Whitespace normalization.
>
> This indicates two problems: apparently, the whitespace normalization
> also normalizes string literals; not sure whether this is a good idea
> or not.

reindent.py never leaves a hard tab character behind, or a line with
trailing whitespace.  Those are features, because they're invisible,
and _someone's_ editor is going to destroy either or both of those
sooner or later anyway.  No core code should rely on them (inside
string literals or not).

I don't believe that asdl_c.py requires either hard tab characters or
trailing whitespace on lines (both of which reindent.py
replaced/removed), and reindent.py's output is a fixed-point for
reindent.py, so those shouldn't happen again (unless/until someone
sticks hard tabs or trailing whitespace into asdl_c.py again).

If someone is really keen to have hard tab characters inside the CODE
strings written by this file, then "the right" way to do that is to
use \t escapes inside the triple-quoted strings (instead of using
embedded hard tab characters).

> Furthermore, people should be made aware that something
> deserves their attention when they find that Python-ast.c has changed
> for them (but I guess Tim never saw Python-ast.c change, because this
> build step isn't executed on Windows).

I can do better than guess about that ;-)

From bob at redivi.com  Tue Jan  3 20:49:09 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 3 Jan 2006 11:49:09 -0800
Subject: [Python-Dev] buildbot
In-Reply-To: <17338.40044.896652.583651@montanaro.dyndns.org>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
Message-ID: <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>

On Jan 3, 2006, at 7:46 AM, skip at pobox.com wrote:

>     Bob> The easy fix is to upgrade your OS.  I don't think anyone  
> is going
>     Bob> to bother with the preprocessor hackery necessary to make  
> that
>     Bob> (harmless) warning go away on older versions of the OS.
>
> Excuse me, but this really pisses me off.  I delayed upgrading from  
> 10.2 for
> awhile and was given the same "advice".  I was further told (check  
> the mac
> sig archives) that "you'd be able to stick with 10.3 for much longer"
> because new apps wouldn't need to require 10.4.  So I upgraded.   
> Now you're
> telling me that it's somehow "obsolete" and that I should upgrade  
> because
> "we can't be bothered to fix a compiler warning"?  Python supports  
> much
> older versions of other platforms.  What makes Mac OSX so special  
> in this
> regard?

The compiler warning is harmless.  It's probably about 10 lines of  
code and an hour of testing to fix it, then you'd need to backport to  
2.4, etc.  Also, in order to do it properly you need to have access  
to older versions of the OS in order to see the warning and test to  
make sure you didn't break anything.

Who's going to bother?  It's just not practical to fix something that  
isn't broken.

The thing that makes Mac OS X "special" is that there don't seem to  
be Python developers who are both knowledgeable and willing to make  
sure everything works on all versions of the OS.  Since you seem to  
care, why don't you take up the task?

-bob


From gmccaughan at synaptics-uk.com  Tue Jan  3 19:43:10 2006
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Tue, 3 Jan 2006 18:43:10 +0000
Subject: [Python-Dev] a quit that actually quits
In-Reply-To: <dp1jet$p9m$1@sea.gmane.org>
References: <dor982$qrc$1@sea.gmane.org> <20051229051008.GA2459@panix.com>
	<dp1jet$p9m$1@sea.gmane.org>
Message-ID: <200601031843.11183.gmccaughan@synaptics-uk.com>

>      class _Quitter(str):
>          def __call__(self): raise SystemExit
>      quit = _Quitter('The quit command.  Type "quit()" to exit')
>      exit = _Quitter('The exit command.  Type "exit()" to exit')

I think you meant

    class _Quitter(str):
        def __call__(self): raise SystemExit
    quit = _Quitter('The quit command.  Type "quit()" to quit')
    exit = _Quitter('The exit command.  Type "exit()" to exit')

:-)

-- 
g


From trentm at ActiveState.com  Tue Jan  3 21:02:29 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Tue, 3 Jan 2006 12:02:29 -0800
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
	lines
In-Reply-To: <43B9352C.9080800@v.loewis.de>
References: <20051230184534.GC30751@ActiveState.com>
	<43B9352C.9080800@v.loewis.de>
Message-ID: <20060103200229.GC31965@activestate.com>

[Martin v. Loewis wrote]
> Trent Mick wrote:
> > Is this intentional? If not, could someone point me to where the svn
> > trigger scripts are maintained so I could poke around for a fix? (Or
> > just fix it themselves. :)
> 
> It was not my intention. They are in
> dinsdale.python.org:/data/repos/projects

I don't have access to that box. Is this a machine used for maintaining
the pydotorg stuff? Do you want me in that "club"? Or would it be easier
for you the fix this little inconsistency... or have me make a bug on SF
and assign to you (or someone else)?

Thanks,
Trent

-- 
Trent Mick
trentm at activestate.com

From guido at python.org  Tue Jan  3 21:35:03 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jan 2006 12:35:03 -0800
Subject: [Python-Dev] Including zlib...
In-Reply-To: <17338.39575.904122.821741@montanaro.dyndns.org>
References: <20060103064313.DA3A21E4002@bag.python.org>
	<17338.39575.904122.821741@montanaro.dyndns.org>
Message-ID: <ca471dc20601031235o449e76c5w42f162d0b3295978@mail.gmail.com>

Hear, hear.

On 1/3/06, skip at pobox.com <skip at pobox.com> wrote:
>
> Martin checked in zlib to the Python svn repository.  Are we really sure
> that including zlib is the only path to whatever it is that it achieves?  If
> security holes in zlib turn up (they have in the past), new Python releases
> will have to be released quickly.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at python.net  Tue Jan  3 21:36:39 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 03 Jan 2006 21:36:39 +0100
Subject: [Python-Dev] buildbot
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
Message-ID: <7j9h82w8.fsf@python.net>

skip at pobox.com writes:

> I delayed upgrading from 10.2 for awhile and was given the same
> "advice".  I was further told (check the mac sig archives) that "you'd
> be able to stick with 10.3 for much longer" because new apps wouldn't
> need to require 10.4.  So I upgraded.  Now you're telling me that it's
> somehow "obsolete" and that I should upgrade because "we can't be
> bothered to fix a compiler warning"?  Python supports much older
> versions of other platforms.  What makes Mac OSX so special in this
> regard?

There are 10.1 and 10.2 boxes in the sourceforge compile farm which
could probably used for testing.

Thomas


From tim.peters at gmail.com  Tue Jan  3 21:58:10 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 3 Jan 2006 15:58:10 -0500
Subject: [Python-Dev] Including zlib...
In-Reply-To: <ca471dc20601031235o449e76c5w42f162d0b3295978@mail.gmail.com>
References: <20060103064313.DA3A21E4002@bag.python.org>
	<17338.39575.904122.821741@montanaro.dyndns.org>
	<ca471dc20601031235o449e76c5w42f162d0b3295978@mail.gmail.com>
Message-ID: <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com>

[Guido]
> Hear, hear.

[Skip]
>> Martin checked in zlib to the Python svn repository.  Are we really sure
>> that including zlib is the only path to whatever it is that it achieves?  If
>> security holes in zlib turn up (they have in the past), new Python releases
>> will have to be released quickly.

Martin & Thomas Heller proposed it here about two weeks ago:

    http://mail.python.org/pipermail/python-dev/2005-December/058873.html

It's pimarily for the benefit of py2exe on Windows.  AFAICT, he didn't
change Python's treatment of zlib on non-Windows boxes; on Windows
zlib is now compiled in to the core DLL.

From guido at python.org  Tue Jan  3 22:13:00 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 3 Jan 2006 13:13:00 -0800
Subject: [Python-Dev] Including zlib...
In-Reply-To: <1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com>
References: <20060103064313.DA3A21E4002@bag.python.org>
	<17338.39575.904122.821741@montanaro.dyndns.org>
	<ca471dc20601031235o449e76c5w42f162d0b3295978@mail.gmail.com>
	<1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com>
Message-ID: <ca471dc20601031313r590813f3jf8037be1db138ab0@mail.gmail.com>

OK. As long a typical Unix build still links with whatever shared zlib
is present on the box I'm fine with this.

--Guido

On 1/3/06, Tim Peters <tim.peters at gmail.com> wrote:
> [Guido]
> > Hear, hear.
>
> [Skip]
> >> Martin checked in zlib to the Python svn repository.  Are we really sure
> >> that including zlib is the only path to whatever it is that it achieves?  If
> >> security holes in zlib turn up (they have in the past), new Python releases
> >> will have to be released quickly.
>
> Martin & Thomas Heller proposed it here about two weeks ago:
>
>     http://mail.python.org/pipermail/python-dev/2005-December/058873.html
>
> It's pimarily for the benefit of py2exe on Windows.  AFAICT, he didn't
> change Python's treatment of zlib on non-Windows boxes; on Windows
> zlib is now compiled in to the core DLL.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.peters at gmail.com  Tue Jan  3 22:22:09 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 3 Jan 2006 16:22:09 -0500
Subject: [Python-Dev] Including zlib...
In-Reply-To: <ca471dc20601031313r590813f3jf8037be1db138ab0@mail.gmail.com>
References: <20060103064313.DA3A21E4002@bag.python.org>
	<17338.39575.904122.821741@montanaro.dyndns.org>
	<ca471dc20601031235o449e76c5w42f162d0b3295978@mail.gmail.com>
	<1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com>
	<ca471dc20601031313r590813f3jf8037be1db138ab0@mail.gmail.com>
Message-ID: <1f7befae0601031322u2b3ac77ej195141b1e112d559@mail.gmail.com>

[Guido]
> OK. As long a typical Unix build still links with whatever shared zlib
> is present on the box I'm fine with this.

 [Guido]
>>> Hear, hear.

[Skip]
>>>> Martin checked in zlib to the Python svn repository.  Are we really sure
>>>> that including zlib is the only path to whatever it is that it
achieves?  If
>>>> security holes in zlib turn up (they have in the past), new Python releases
>>>> will have to be released quickly.

[Tim]
>> Martin & Thomas Heller proposed it here about two weeks ago:
>>
>>     http://mail.python.org/pipermail/python-dev/2005-December/058873.html
>>
>> It's pimarily for the benefit of py2exe on Windows.  AFAICT, he didn't
>> change Python's treatment of zlib on non-Windows boxes; on Windows
>> zlib is now compiled in to the core DLL.

I figure that if we keep this conversation going long enough, we can
eventually shame Guido out of top-posting before it becomes a habit
;-).

From nnorwitz at gmail.com  Tue Jan  3 22:27:11 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 3 Jan 2006 13:27:11 -0800
Subject: [Python-Dev] Including zlib...
In-Reply-To: <1f7befae0601031322u2b3ac77ej195141b1e112d559@mail.gmail.com>
References: <20060103064313.DA3A21E4002@bag.python.org>
	<17338.39575.904122.821741@montanaro.dyndns.org>
	<ca471dc20601031235o449e76c5w42f162d0b3295978@mail.gmail.com>
	<1f7befae0601031258t20401cc6q2c4e676dedc09d2d@mail.gmail.com>
	<ca471dc20601031313r590813f3jf8037be1db138ab0@mail.gmail.com>
	<1f7befae0601031322u2b3ac77ej195141b1e112d559@mail.gmail.com>
Message-ID: <ee2a432c0601031327t709a146bw2e6d4b9826f92e07@mail.gmail.com>

Hear, hear. Er....

On 1/3/06, Tim Peters <tim.peters at gmail.com> wrote:
> [Guido]
> > OK. As long a typical Unix build still links with whatever shared zlib
> > is present on the box I'm fine with this.
>
>  [Guido]
> >>> Hear, hear.
>
> [Skip]
> ...
>
> [Tim]
> ...
> I figure that if we keep this conversation going long enough, we can
> eventually shame Guido out of top-posting before it becomes a habit
> ;-).

Here, here!  Google has already taught him bad habits. :-)

n

From martin at v.loewis.de  Tue Jan  3 22:59:42 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jan 2006 22:59:42 +0100
Subject: [Python-Dev] [Python-checkins] commit of r41906
	-	python/branches/ssize_t/Objects/obmalloc.c
In-Reply-To: <1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com>
References: <20060103131655.201471E400A@bag.python.org>
	<1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com>
Message-ID: <43BAF3CE.7040506@v.loewis.de>

Tim Peters wrote:
>>Modified: python/branches/ssize_t/Objects/obmalloc.c
[...]
> This checkin should be reverted for now.

Not sure whether you've noticed this is "just" on the ssize_t branch.
Without this patch, it is not possible to allocate 4GiB or more for
a string object in debug mode, which kind of defeated my attempts to
test that.

I certainly plan to remove all XXX marks I have introduced in that
branch before suggesting to integrate it back into the trunk.

So "for now", I would prefer to keep it, and only revert it if
I have a complete fix.

> It's in
> _PyObject_DebugMalloc, and at present the layout of the extra debug
> info in a PYMALLOC_DEBUG build is hard-coded to use 4-byte fields, no
> matter what sizeof(size_t) may be.  One of the extra fields recorded
> in a PYMALLOC_DEBUG build is the number of bytes requested, and at
> present it's simply not capable of recording a value that doesn't fit
> in 4 bytes.

Well, AFAICT, it "works" even if it records only records the lower 4
bytes of the requested size. Upon freeing, it just won't put enough
DEADBYTEs in, which I cannot see having further unfortunate consequences
(except that it won't diagnose errors as good anymore as it could).

> Even after (if ever ;-)) this is changed to support recording 8-byte
> values on a box where sizeof(size_t) == 8, the "total < nbytes" part
> of the test would still be appropriate:  PyObject_DebugMalloc requests
> more memory (`total`) than the user asked for (`nbytes`), and the
> computation of `total` may have overflowed.  That's what "total <
> nbytes" is checking, and that's the right way to spell the overflow
> check on any box.

Certainly; I did not mean to completely disable this test.

Regards,
Martin

From tim.peters at gmail.com  Tue Jan  3 23:13:21 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 3 Jan 2006 17:13:21 -0500
Subject: [Python-Dev] [Python-checkins] commit of r41906 -
	python/branches/ssize_t/Objects/obmalloc.c
In-Reply-To: <43BAF3CE.7040506@v.loewis.de>
References: <20060103131655.201471E400A@bag.python.org>
	<1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com>
	<43BAF3CE.7040506@v.loewis.de>
Message-ID: <1f7befae0601031413hc93e132gb64022f7e8eb727b@mail.gmail.com>

[Tim]
> >>Modified: python/branches/ssize_t/Objects/obmalloc.c
> [...]
>> This checkin should be reverted for now.

[Martin]
> Not sure whether you've noticed this is "just" on the ssize_t branch.

Right, I noticed.

> Without this patch, it is not possible to allocate 4GiB or more for
> a string object in debug mode, which kind of defeated my attempts to
> test that.
>
> I certainly plan to remove all XXX marks I have introduced in that
> branch before suggesting to integrate it back into the trunk.
>
> So "for now", I would prefer to keep it, and only revert it if
> I have a complete fix.

If it's expedient, that's fine for now.  I'd like to help fix it "for
real", but time is a problem for now.  In a week or two I may have
more to give to this.  Would probably also like to change the dict
object's ma_{fill,used,mask} members to Py_ssize_t (while other people
don't seem to do this much, I routinely write first-stab programs
where a single dict blows more than 2GB of VM ;-)).

> ...

From martin at v.loewis.de  Tue Jan  3 23:14:42 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jan 2006 23:14:42 +0100
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
 lines
In-Reply-To: <20060103200229.GC31965@activestate.com>
References: <20051230184534.GC30751@ActiveState.com>
	<43B9352C.9080800@v.loewis.de>
	<20060103200229.GC31965@activestate.com>
Message-ID: <43BAF752.4040504@v.loewis.de>

Trent Mick wrote:
> I don't have access to that box. Is this a machine used for maintaining
> the pydotorg stuff?

Yes, it's also www.python.org (but not mail.python.org)

> Or would it be easier
> for you the fix this little inconsistency... or have me make a bug on SF
> and assign to you (or someone else)?

I tried to, but couldn't. I'll be sending the mailer.py and mailer.conf
files in a private mail. I can't see anything wrong in that code.

Perhaps mailman is adding the spaces?

Regards,
Martin


From nnorwitz at gmail.com  Tue Jan  3 23:25:22 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 3 Jan 2006 14:25:22 -0800
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
	lines
In-Reply-To: <43BAF752.4040504@v.loewis.de>
References: <20051230184534.GC30751@ActiveState.com>
	<43B9352C.9080800@v.loewis.de>
	<20060103200229.GC31965@activestate.com>
	<43BAF752.4040504@v.loewis.de>
Message-ID: <ee2a432c0601031425r29552468g726aaf8dab34146@mail.gmail.com>

On 1/3/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>
> I tried to, but couldn't. I'll be sending the mailer.py and mailer.conf
> files in a private mail. I can't see anything wrong in that code.
>
> Perhaps mailman is adding the spaces?

I looked into this briefly.  I saw something like:

if prefix:
  subject = prefix + ' ' + self.subject
else:
  subject = self.subject

My guess is that there needs to be stripping for prefix and
self.subject.  I don't know the code well, but that's all I could come
up with.  I didn't see any extra spaces in mailer.conf IIRC.

n

From martin at v.loewis.de  Tue Jan  3 23:45:03 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 03 Jan 2006 23:45:03 +0100
Subject: [Python-Dev] [Python-checkins] commit of r41906 -
	python/branches/ssize_t/Objects/obmalloc.c
In-Reply-To: <1f7befae0601031413hc93e132gb64022f7e8eb727b@mail.gmail.com>
References: <20060103131655.201471E400A@bag.python.org>	
	<1f7befae0601031038w6f6c6f03o4549e039dafd19a8@mail.gmail.com>	
	<43BAF3CE.7040506@v.loewis.de>
	<1f7befae0601031413hc93e132gb64022f7e8eb727b@mail.gmail.com>
Message-ID: <43BAFE6F.5010305@v.loewis.de>

Tim Peters wrote:
> If it's expedient, that's fine for now.  I'd like to help fix it "for
> real", but time is a problem for now.  In a week or two I may have
> more to give to this.  Would probably also like to change the dict
> object's ma_{fill,used,mask} members to Py_ssize_t (while other people
> don't seem to do this much, I routinely write first-stab programs
> where a single dict blows more than 2GB of VM ;-)).

Yes, I just noticed I mostly ignored dicts so far; that should also
be part of the final patch.

Regards,
Martin

From martin at v.loewis.de  Wed Jan  4 00:01:30 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Jan 2006 00:01:30 +0100
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
 lines
In-Reply-To: <ee2a432c0601031425r29552468g726aaf8dab34146@mail.gmail.com>
References: <20051230184534.GC30751@ActiveState.com>	
	<43B9352C.9080800@v.loewis.de>	
	<20060103200229.GC31965@activestate.com>	
	<43BAF752.4040504@v.loewis.de>
	<ee2a432c0601031425r29552468g726aaf8dab34146@mail.gmail.com>
Message-ID: <43BB024A.6030109@v.loewis.de>

Neal Norwitz wrote:
> I looked into this briefly.  I saw something like:
> 
> if prefix:
>   subject = prefix + ' ' + self.subject
> else:
>   subject = self.subject

Hmm - this shouldn't matter: There is only a single prefix defined,
in [defaults]:

commit_subject_prefix = [Python-checkins] commit of

There is no other occurrence of that string in any other spelling
elsewhere, and neither of the "commit of" part. So if prefix was
empty, the entire "commit of" should be absent.

Anyway, I just killed the "[Python-checkins]" part from the prefix
(expecting that mailman would add it, anyway), and regenerated
the r41848 commit message, and it seem that worked this time.

So I still suspect this to be a mailman bug.

There is just too much Python code involved in delivering Python
commit messages :-)

Regards,
Martin

From martin at v.loewis.de  Wed Jan  4 00:11:07 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Jan 2006 00:11:07 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>	<17337.60315.618431.480677@montanaro.dyndns.org>	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>	<17338.40044.896652.583651@montanaro.dyndns.org>
	<871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <43BB048B.7080909@v.loewis.de>

Stephen J. Turnbull wrote:
> Sure, Python can be expected to do much better than random port
> maintainers, but....  Discussions with the port maintainers makes it
> hard to blame them.

Well, it's the same attitude that Fredrik Lundh just critized. It's
ok for a port maintainer to say "I don't have any energy to fix it
myself" (although I would hope he still says "patches will be
accepted"). It's not ok for a port maintainer to say "it's not
a bug - just upgrade your system". It *is* a bug for Python to
emit warnings on "major platforms" (PEP 7).

Also, it would be good if PEP 11 was considered before breaking
platforms. If some version of the system is not supported anymore,
fine - but *atleast* that should be mentioned in PEP 11, and better,
the process suggested in PEP 11 should be followed.

Regards,
Martin

From martin at v.loewis.de  Wed Jan  4 00:12:22 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Jan 2006 00:12:22 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>	<17337.60315.618431.480677@montanaro.dyndns.org>	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
Message-ID: <43BB04D6.5080302@v.loewis.de>

Bob Ippolito wrote:
> Who's going to bother? 

It violates PEP 7, unless you argue that OS X/gcc is not
a "major compiler".

Regards,
Martin

From martin at v.loewis.de  Wed Jan  4 00:17:50 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Jan 2006 00:17:50 +0100
Subject: [Python-Dev] Including zlib...
In-Reply-To: <17338.39575.904122.821741@montanaro.dyndns.org>
References: <20060103064313.DA3A21E4002@bag.python.org>
	<17338.39575.904122.821741@montanaro.dyndns.org>
Message-ID: <43BB061E.5020401@v.loewis.de>

skip at pobox.com wrote:
> Martin checked in zlib to the Python svn repository.  Are we really sure
> that including zlib is the only path to whatever it is that it achieves?  If
> security holes in zlib turn up (they have in the past), new Python releases
> will have to be released quickly.

As Tim explains: this change doesn't change anything wrt. having to
rerelease. On Unix, nothing changes because the included zlib
isn't used (but, personally, I would accept contributions of a patch
to use it if there is no platform zlib). On Windows, nothing changes
really because now you have to update python24.dll, instead of updating
zlib.pyd - either requires a recompilation of the Python sources.

Regards,
Martin

From bob at redivi.com  Wed Jan  4 00:51:39 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 3 Jan 2006 15:51:39 -0800
Subject: [Python-Dev] buildbot
In-Reply-To: <43BB04D6.5080302@v.loewis.de>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>	<17337.60315.618431.480677@montanaro.dyndns.org>	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
	<43BB04D6.5080302@v.loewis.de>
Message-ID: <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>


On Jan 3, 2006, at 3:12 PM, Martin v. L?wis wrote:

> Bob Ippolito wrote:
>> Who's going to bother?
>
> It violates PEP 7, unless you argue that OS X/gcc is not
> a "major compiler".

Clearly, but that still doesn't answer the question of who's going to  
do it.  Writing two code paths with availability macros and all the  
testing involved is a whole lot of work to fix a harmless warning on  
older versions of the OS.  If it actually caused a problem I'm sure  
someone would go through the trouble, but in this case it's probably  
not worth the effort.

-bob


From martin at v.loewis.de  Wed Jan  4 01:00:09 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Jan 2006 01:00:09 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>	<17337.60315.618431.480677@montanaro.dyndns.org>	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
	<43BB04D6.5080302@v.loewis.de>
	<3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
Message-ID: <43BB1009.7080008@v.loewis.de>

Bob Ippolito wrote:
> Clearly, but that still doesn't answer the question of who's going to 
> do it.  Writing two code paths with availability macros and all the 
> testing involved is a whole lot of work to fix a harmless warning on 
> older versions of the OS.  If it actually caused a problem I'm sure 
> someone would go through the trouble, but in this case it's probably 
> not worth the effort.

I can fully understand that it is not worth your effort (it isn't
worth mine, either). However, if Skip (or somebody else) wanted to
fix it, I wouldn't mind. So I wouldn't agree to "not worth the
effort".

Regards,
Martin

From skip at pobox.com  Wed Jan  4 04:15:03 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 3 Jan 2006 21:15:03 -0600
Subject: [Python-Dev] buildbot
In-Reply-To: <43BB048B.7080909@v.loewis.de>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
	<871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43BB048B.7080909@v.loewis.de>
Message-ID: <17339.15799.408323.601253@montanaro.dyndns.org>

>>>>> "Martin" == Martin v L?wis <martin at v.loewis.de> writes:

    Martin> Stephen J. Turnbull wrote:
    >> Sure, Python can be expected to do much better than random port
    >> maintainers, but....  Discussions with the port maintainers makes it
    >> hard to blame them.

    Martin> Well, it's the same attitude that Fredrik Lundh just critized.
    Martin> It's ok for a port maintainer to say "I don't have any energy to
    Martin> fix it myself" (although I would hope he still says "patches
    Martin> will be accepted").  It's not ok for a port maintainer to say
    Martin> "it's not a bug - just upgrade your system". It *is* a bug for
    Martin> Python to emit warnings on "major platforms" (PEP 7).

Thank you.  I was beginning to think I was going crazy.

Skip

From walter at livinglogic.de  Wed Jan  4 07:27:03 2006
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed, 04 Jan 2006 07:27:03 +0100
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
 lines
In-Reply-To: <43BB024A.6030109@v.loewis.de>
References: <20051230184534.GC30751@ActiveState.com>		<43B9352C.9080800@v.loewis.de>		<20060103200229.GC31965@activestate.com>		<43BAF752.4040504@v.loewis.de>	<ee2a432c0601031425r29552468g726aaf8dab34146@mail.gmail.com>
	<43BB024A.6030109@v.loewis.de>
Message-ID: <43BB6AB7.8000301@livinglogic.de>

Martin v. L?wis wrote:

> [...] 
> commit_subject_prefix = [Python-checkins] commit of
> 
> [...]
> 
> Anyway, I just killed the "[Python-checkins]" part from the prefix
> (expecting that mailman would add it, anyway), and regenerated
> the r41848 commit message, and it seem that worked this time.

And while we're at it, could you remove the "commit of" too? IMHO it 
just obscures the real content of the subject.

Bye,
    Walter D?rwald


From raymond.hettinger at verizon.net  Wed Jan  4 07:37:51 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 04 Jan 2006 01:37:51 -0500
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
 lines
In-Reply-To: <43BB6AB7.8000301@livinglogic.de>
Message-ID: <000801c610f9$63acadc0$741cc797@oemcomputer>

> And while we're at it, could you remove the "commit of" too? IMHO it
> just obscures the real content of the subject.

+1


Raymond


From warner at lothar.com  Wed Jan  4 09:30:58 2006
From: warner at lothar.com (Brian Warner)
Date: Wed, 04 Jan 2006 00:30:58 -0800 (PST)
Subject: [Python-Dev]  buildbot
In-Reply-To: ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com
Message-ID: <20060104.003058.48524852.warner@lothar.com>

> Currently, branches are not supported, because buildbot is
> somewhat limited. When I get a solution for this problem,
> I plan to have all buildbots build both the trunk and the
> 2.4 branch; such builds would only be initiated whenever
> something is committed on the branch.

Branch support appeared fairly recently (buildbot-0.7.0, about two months
ago), and I believe it should be sufficient to trigger builds for both the
trunk and the 2.4 branch. The last part of the setup involves modifying the
svn_buildbot.py hook script, though, so I won't claim this support is
complete yet.

The basic idea is that you create multiple Scheduler instances, one for each
branch you want to monitor. Both Schedulers are configured to trigger all the
Builders. You configure the SVN-checkout step of the build process with a
baseURL= argument to provide the common portion of the SVN URL, and add a
defaultBranch= argument to point to the trunk. Each Build is done for a
specific branch, and the SVN checkout step just appends the branch name to
the baseURL to figure out what to pass to 'svn co %s'

Assuming the ChangeSource is providing branch information properly, this will
result in any change to the trunk causing a trunk build, and any change to
the 2.4 branch causing a 2.4 build.

Your master.cfg file will probably want to have some of the following
elements:

---BEGIN---
from buildbot.changes.pb import PBChangeSource
c['sources'] = PBChangeSource() # N.B.: don't set prefix=

from buildbot.scheduler import Scheduler
all_builders=["sparc solaris10 gcc trunk",
              "g5 osx trunk",
              "amd64 gentoo trunk",
              "x86 gentoo trunk"]
s1 = Scheduler("trunk scheduler", "trunk", 2*60, all_builders)
s2 = Scheduler("2.4 scheduler", "branches/release24-maint", 2*60, all_builders)
c['schedulers'] = [s1, s2]

...
from buildbot.process.factory import s
from buildbot.process.step import SVN
checkoutstep = s(SVN, baseURL="http://svn.python.org/projects/python/",
                 defaultBranch="trunk", mode="copy")
steps = [checkoutstep, ...]

---END---


The tricky part is getting branch-annotated Changes into the buildbot in the
first place. I assume you're using the svn_buildbot.py script. The version
provided in the current release always sets the branch to 'None', which
implies the default branch. You will have to edit this script to give it the
ability to decide which branch each change is for (by adding some code which
does a test like filename.startswith("trunk/"), perhaps), then include the
branch name in the RPC call that informs the buildmaster about the new
Change. Basically you'll need to change this call:

        return persp.callRemote('addChange', {'who': who,
                                              'files': changed,
                                              'comments': message,
                                              'revision': revision})

into one like:

        branchname = "trunk" # or "branches/release24-maint"
        return persp.callRemote('addChange', {'who': who,
                                              'branch': branchname,
                                              'files': changed,
                                              'comments': message,
                                              'revision': revision})

There's been some discussion on the buildbot-devel list about the best way to
implement branch-determination. One school of thought says it belongs close
to the repository (hence inside a hook script like svn_buildbot.py). Another
says it is easier to configure from the buildbot side, in which case you'd
probably want to modify buildbot.changes.pb.PBChangeSource to analyze the
filenames and decide which branch they live on there.

There was also a patch[1] to add some regexps to svn_buildbot.py to do this
kind of branch determination. I haven't been able to review it properly yet,
but it may give you some ideas. svn_buildbot.py certainly needs help here.
The rest of the Buildbot is branch-enabled and ready to go. The only
lingering issues are with status delivery: the main HTML "Waterfall" HTML
will interleave builds of both branches on the same page, which could be a
bit confusing (if the top line is red, does that mean the trunk is broken, or
the 2.4 branch?). Fixing this (by creating separate pages for each branch) is
high on the TODO list.


If there's any way I can help out further, please let me know. I'm ecstatic
that Python is using a buildbot.. if you have any feature suggestions or bug
reports, please send them to me, to the buildbot-devel mailing list[2], or
file them in the sf.net bugtracker[3]. Or corner me at PyCon, or in #twisted
(where I can sometimes be found as <warner>).

thanks!
 -Brian  (author of Buildbot)


[1]: http://sourceforge.net/mailarchive/message.php?msg_id=13977004
[2]: http://sourceforge.net/mailarchive/forum.php?forum=buildbot-devel
[3]: http://sourceforge.net/projects/buildbot/

From martin at v.loewis.de  Wed Jan  4 09:47:02 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 04 Jan 2006 09:47:02 +0100
Subject: [Python-Dev] slight inconsistency in svn checkin email subject
 lines
In-Reply-To: <43BB6AB7.8000301@livinglogic.de>
References: <20051230184534.GC30751@ActiveState.com>		<43B9352C.9080800@v.loewis.de>		<20060103200229.GC31965@activestate.com>		<43BAF752.4040504@v.loewis.de>	<ee2a432c0601031425r29552468g726aaf8dab34146@mail.gmail.com>
	<43BB024A.6030109@v.loewis.de> <43BB6AB7.8000301@livinglogic.de>
Message-ID: <43BB8B86.7000509@v.loewis.de>

Walter D?rwald wrote:
> And while we're at it, could you remove the "commit of" too? IMHO it
> just obscures the real content of the subject.

Done. FYI, the rationale for this prefix was that post-commit
distinguishes between "commit" and "revprop", where revprop
would indicate that properties changed, not the contents.

I just noticed that this is not actually post-commit, but that
there is a second hook, post-revprop-change, which I hadn't
filled out yet. I just created the hook; because I dropped the
prefix, commit and revprop mails will have the same subject.

Regards,
Martin

From gregoire.weber at schmid-telecom.ch  Wed Jan  4 12:17:32 2006
From: gregoire.weber at schmid-telecom.ch (Weber, Gregoire)
Date: Wed, 4 Jan 2006 12:17:32 +0100
Subject: [Python-Dev] Real time behaviour of Pythons memory management;
	WAS: RE: Documentation about Python's GC,python-dev list messages
	referenced in Modules/gcmodule.c notreachable anymore
Message-ID: <52D00EBDF4ECC74389E3B4F3D39F8A290FC575@ZHEXCHANGE.schmid-telecom.com>

Garbage Collector findings

To understand pythons garbage collector better and to get
a picture about the runtime behaviour and performance 
I did some experiments:

The attached script allocates a lot of circularly self 
referencing lists. Then it instantiates a one item list.

I tuned the loop counter in such a way that the subsequent  
instantiation of the one item list triggers the garbage 
collector. The number of the circularly self referencing 
objects is around 90000.


Results (Pentium4 3GHz, Python 2.4.1 on cygwin/Win2k)
-----------------------------------------------------

 gc gen 1) | time msec 2) | unreachable 3)
-----------+--------------+----------------
 None      | 0.000        | All
 0         | 0.002        | All
 1         | 0.007        | All
 2         | 0.065        | All
 None      | 0.000        | None
 0         | 0.001        | None
 1         | 0.003        | None
 2         | 0.018        | None

--> Collecting a self referencing list costs about 
    0.7 usec (micro seconds)
--> Checking a non collectable self ref list costs 
    about 0.2 usec (micro seconds)

Legend:

1) the generation the garbage collector has been triggered 
   to collect (None means that GC wasn't triggered)
2) time for instantiating a list with one entry in msec
3) All: All of the circularly self referencing lists were 
        **unreachable** (thus got collected by the gc)
   None: None of the circularly self referencing lists were 
         **unreachable** (no garbage available to collect)


Questions
---------

1. Am I correct that in a system which instantiates a lot*) 
   of containerish objects without destructing them again the 
   GC may be triggered to evaluate all generations which may 
   be very costy (see above)?

2. In a system where only generation 0 and 1 get evaluated
   (because not so much object got instantiated without beeing 
   destructed) the costs are relatively low. Correct?

*) a lot means here: 
   more than ``threshold0 * threshold1 * threshold2`` objects, 
   Python 2.4 default values are: 700*10*10 = 70000


Gregoire




P.S.: Interestingely the counters found by experimenting 
      seem to depend on the platform (my platform: 
	Python 2.4.1, cygwin under Win2k, the counters aren't 
	valid for native Python 2.4.2 under Win2k).


-------------- next part --------------
A non-text attachment was scrubbed...
Name: gctest.py
Type: application/octet-stream
Size: 3936 bytes
Desc: gctest.py
Url : http://mail.python.org/pipermail/python-dev/attachments/20060104/a19e29c2/attachment.obj 

From stephen at xemacs.org  Wed Jan  4 13:47:41 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 04 Jan 2006 21:47:41 +0900
Subject: [Python-Dev] buildbot
In-Reply-To: =?iso-8859-1?q?=3C43BB048B=2E7080909=40v=2Eloewis=2Ede=3E_?=
	=?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Wed=2C_04_Jan_200?=
	=?iso-8859-1?q?6_00=3A11=3A07_+0100=22=29?=
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
	<871wzpmg3k.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43BB048B.7080909@v.loewis.de>
Message-ID: <87vex0jh1u.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v L?wis <martin at v.loewis.de> writes:

    Martin> It *is* a bug for Python to emit warnings on "major
    Martin> platforms" (PEP 7).

OK, I'm as big a standards bigot as the next guy, you hooked me.
After some consideration, I can't write the patch, though.  I'm sorry
that all I can offer is time and trouble testing on a Mac G4 running
Panther (10.3.9), but for what it's worth, I offer it.  I'll get the
workspace ready, and when there's a patch, I'll test it.

    Martin> Also, it would be good if PEP 11 was considered before
    Martin> breaking platforms.

I don't think Apple is ever likely to consider Python PEPs when
revising its standard headers in incompatible ways.<0.5 wink>

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From thomas at xs4all.net  Wed Jan  4 17:34:19 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 4 Jan 2006 17:34:19 +0100
Subject: [Python-Dev] file.next() vs. file.readline()
Message-ID: <20060104163419.GF18916@xs4all.nl>


Twice now, I've been bitten by the 'mixing file-iteration with readline'
issue. (Yeah, twice.. Good thing I don't write anything important in
Python ;) I've also seen a number of newbies bitten by the same thing. The
issue, for those that aren't aware, is that when mixing file-iteration and
readline (or readlines or such), you run the risk of getting data in the
wrong order. This is because file-iteration uses an 8k buffer, while
readline doesn't. It's not really a big deal, once you understand it's not      
wise to mix iteration and the readline(s) methods.                              
                                                                                
I do wonder, though, why Python doesn't take care to raise an exception when
readline is called with 'stuff' in the iteration-buffer. A cursory look at
the source doesn't reveal any glaring problems. It's a single check,
possibly two, with good locality of reference. Also, raising an exception
when there is stuff in the buffer would only make mixing iteration/readline
an error when you would actually, in fact, lose or mess up data. In other
words, it would only raise exceptions for existing cases that are already
broken.                                                     

Is there something I've missed that makes the check undesireable or
unfeasible? Or should I just assume no on has gotten around to it (or could
be bothered), and just submit a patch? :)                
                                                        
(Making readline and friends use the same buffer may seem the better
solution to some, but I'm sure there's a whole discussion behind that, about
whether to buffer or not. All non-iteration routines in fileobject.c take
pretty good care not to read too much, and I choose to see that as        
explicitly designed that way.)                                          

Absent-ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From guido at python.org  Wed Jan  4 19:10:07 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jan 2006 10:10:07 -0800
Subject: [Python-Dev] file.next() vs. file.readline()
In-Reply-To: <20060104163419.GF18916@xs4all.nl>
References: <20060104163419.GF18916@xs4all.nl>
Message-ID: <ca471dc20601041010r68d7bcd8q5d9bb9ae0c664a27@mail.gmail.com>

I'd say go right ahead and submit a change to SF (and then after it's
reviewed you can check it in yourself :-).

The only reason I can think of why this isn't done yet would be that
nobody thought of it.

Of course there are other ill-advised combinations, like write
followed by read for which stdio doesn't guarantee any result in
particular. In Py3K I want to revise the whole I/O stack to be
independent from C stdio (except on those platforms where that's the
only I/O you have.)

--Guido

On 1/4/06, Thomas Wouters <thomas at xs4all.net> wrote:
>
> Twice now, I've been bitten by the 'mixing file-iteration with readline'
> issue. (Yeah, twice.. Good thing I don't write anything important in
> Python ;) I've also seen a number of newbies bitten by the same thing. The
> issue, for those that aren't aware, is that when mixing file-iteration and
> readline (or readlines or such), you run the risk of getting data in the
> wrong order. This is because file-iteration uses an 8k buffer, while
> readline doesn't. It's not really a big deal, once you understand it's not
> wise to mix iteration and the readline(s) methods.
>
> I do wonder, though, why Python doesn't take care to raise an exception when
> readline is called with 'stuff' in the iteration-buffer. A cursory look at
> the source doesn't reveal any glaring problems. It's a single check,
> possibly two, with good locality of reference. Also, raising an exception
> when there is stuff in the buffer would only make mixing iteration/readline
> an error when you would actually, in fact, lose or mess up data. In other
> words, it would only raise exceptions for existing cases that are already
> broken.
>
> Is there something I've missed that makes the check undesireable or
> unfeasible? Or should I just assume no on has gotten around to it (or could
> be bothered), and just submit a patch? :)
>
> (Making readline and friends use the same buffer may seem the better
> solution to some, but I'm sure there's a whole discussion behind that, about
> whether to buffer or not. All non-iteration routines in fileobject.c take
> pretty good care not to read too much, and I choose to see that as
> explicitly designed that way.)
>
> Absent-ly y'rs,
> --
> Thomas Wouters <thomas at xs4all.net>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at core-sdi.com  Wed Jan  4 18:43:15 2006
From: lists at core-sdi.com (Gabriel Becedillas)
Date: Wed, 04 Jan 2006 14:43:15 -0300
Subject: [Python-Dev] Sharing between multiple interpreters and restricted
	mode
Message-ID: <43BC0933.2060705@core-sdi.com>

Hi,
At the company I work for we've embedded Python 2.4.1 in a C++ 
application. We execute multiple scripts concurrenlty, each one in its 
own interpreter (created using Py_NewInterpreter()).
We are sharing a certain instance between interpreters because its to
expensive to instantiate that class every time an interpreter is
created. The class is instantiated in the main interpreter (that is
always alive) and every time a new interpreter is created, that
instance is added to the interpreter's __builtins__ dict.
The problem I'm having is that some methods of that class (when
executed in an interpreter different from the one it was created in)
complain about being in restricted execution mode.
Assuming that there are no issues with this instance lifetime (coz it
will always be referenced by the main interpreter), is there a safe way
to do some sharing between interpreters ?.
Thanks.

-- 

Gabriel Becedillas
Developer
CORE SECURITY TECHNOLOGIES

Florida 141 - 2? cuerpo - 7? piso
C1005AAC Buenos Aires - Argentina
Tel/Fax: (54 11) 5032-CORE (2673)
http://www.corest.com

From guido at python.org  Wed Jan  4 19:29:09 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 4 Jan 2006 10:29:09 -0800
Subject: [Python-Dev] Sharing between multiple interpreters and
	restricted mode
In-Reply-To: <43BC0933.2060705@core-sdi.com>
References: <43BC0933.2060705@core-sdi.com>
Message-ID: <ca471dc20601041029l38cae0bbtaaa1206c0e9d118d@mail.gmail.com>

This is more a question for c.l.py. My own suggestion is to go back to
a single shared interpreter; what's the value of separate interpreters
if you're sharing objects anyway? Sharing Python code between
interpreters is definitely not supported. If you insist on separate
interpreters, an alternative solution to your heavyweight shared
object might be a lighter-weight facade for that object which can be
instantiated separately in each interpreter?

--Guido

On 1/4/06, Gabriel Becedillas <lists at core-sdi.com> wrote:
> Hi,
> At the company I work for we've embedded Python 2.4.1 in a C++
> application. We execute multiple scripts concurrenlty, each one in its
> own interpreter (created using Py_NewInterpreter()).
> We are sharing a certain instance between interpreters because its to
> expensive to instantiate that class every time an interpreter is
> created. The class is instantiated in the main interpreter (that is
> always alive) and every time a new interpreter is created, that
> instance is added to the interpreter's __builtins__ dict.
> The problem I'm having is that some methods of that class (when
> executed in an interpreter different from the one it was created in)
> complain about being in restricted execution mode.
> Assuming that there are no issues with this instance lifetime (coz it
> will always be referenced by the main interpreter), is there a safe way
> to do some sharing between interpreters ?.
> Thanks.
>
> --
>
> Gabriel Becedillas
> Developer
> CORE SECURITY TECHNOLOGIES
>
> Florida 141 - 2? cuerpo - 7? piso
> C1005AAC Buenos Aires - Argentina
> Tel/Fax: (54 11) 5032-CORE (2673)
> http://www.corest.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ronaldoussoren at mac.com  Wed Jan  4 21:10:44 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 4 Jan 2006 21:10:44 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
	<43BB04D6.5080302@v.loewis.de>
	<3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
Message-ID: <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com>


On 4-jan-2006, at 0:51, Bob Ippolito wrote:

>
> On Jan 3, 2006, at 3:12 PM, Martin v. L?wis wrote:
>
>> Bob Ippolito wrote:
>>> Who's going to bother?
>>
>> It violates PEP 7, unless you argue that OS X/gcc is not
>> a "major compiler".
>
> Clearly, but that still doesn't answer the question of who's going to
> do it.  Writing two code paths with availability macros and all the
> testing involved is a whole lot of work to fix a harmless warning on
> older versions of the OS.  If it actually caused a problem I'm sure
> someone would go through the trouble, but in this case it's probably
> not worth the effort.

More energy seems to be spent on talking about this than implementing  
the required patch :-)

This should do it, although I haven't tested this on OSX 10.3:


diff -u Python-2.4.2/Modules/getpath.c Python-2.4.2-orig/Modules/ 
getpath.c
--- Python-2.4.2/Modules/getpath.c      2006-01-04 21:06:17.000000000  
+0100
+++ Python-2.4.2-orig/Modules/getpath.c 2004-08-08 03:00:47.000000000  
+0200
@@ -381,12 +381,8 @@
      NSModule pythonModule;
#endif
#ifdef __APPLE__
-#ifdef MAC_OS_X_VERSION_10_4
-    uint32_t nsexeclength = MAXPATHLEN;
-#else
      unsigned long nsexeclength = MAXPATHLEN;
#endif
-#endif
         /* If there is no slash in the argv0 path, then we have to
          * assume python is on the user's $PATH, since there's no


>
> -bob
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com


From reinhold-birkenfeld-nospam at wolke7.net  Wed Jan  4 22:01:21 2006
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Wed, 04 Jan 2006 22:01:21 +0100
Subject: [Python-Dev] TAR problems under Solaris
Message-ID: <dphd31$8av$1@sea.gmane.org>

Recently, someone on dclpy posted about an error he got
when he tried to unpack the Python distribution tarball
with Sparc Solaris 9's tar:

tar: directory checksum error

With GNU tar, it worked correctly.

Is this a known issue, or is it irrelevant?

Reinhold

-- 
Mail address is perfectly valid!


From tim.peters at gmail.com  Wed Jan  4 22:07:37 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 4 Jan 2006 16:07:37 -0500
Subject: [Python-Dev] TAR problems under Solaris
In-Reply-To: <dphd31$8av$1@sea.gmane.org>
References: <dphd31$8av$1@sea.gmane.org>
Message-ID: <1f7befae0601041307u3720fe18p485ee3ed2239aab7@mail.gmail.com>

[Reinhold Birkenfeld]
> Recently, someone on dclpy posted about an error he got
> when he tried to unpack the Python distribution tarball
> with Sparc Solaris 9's tar:
>
> tar: directory checksum error
>
> With GNU tar, it worked correctly.
>
> Is this a known issue, or is it irrelevant?

It's a known issue for just about every tarball-based distribution of
every project on the planet.  For Python, Solaris tar woes are noted
on this page:

    http://www.python.org/2.4.2/bugs.html

From reinhold-birkenfeld-nospam at wolke7.net  Wed Jan  4 22:10:58 2006
From: reinhold-birkenfeld-nospam at wolke7.net (Reinhold Birkenfeld)
Date: Wed, 04 Jan 2006 22:10:58 +0100
Subject: [Python-Dev] TAR problems under Solaris
In-Reply-To: <1f7befae0601041307u3720fe18p485ee3ed2239aab7@mail.gmail.com>
References: <dphd31$8av$1@sea.gmane.org>
	<1f7befae0601041307u3720fe18p485ee3ed2239aab7@mail.gmail.com>
Message-ID: <dphdl2$atq$1@sea.gmane.org>

Tim Peters wrote:
> [Reinhold Birkenfeld]
>> Recently, someone on dclpy posted about an error he got
>> when he tried to unpack the Python distribution tarball
>> with Sparc Solaris 9's tar:
>>
>> tar: directory checksum error
>>
>> With GNU tar, it worked correctly.
>>
>> Is this a known issue, or is it irrelevant?
> 
> It's a known issue for just about every tarball-based distribution of
> every project on the planet.  For Python, Solaris tar woes are noted
> on this page:
> 
>     http://www.python.org/2.4.2/bugs.html

Ah ok, so never mind.

Reinhold

-- 
Mail address is perfectly valid!


From barry at python.org  Wed Jan  4 22:23:41 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 04 Jan 2006 16:23:41 -0500
Subject: [Python-Dev] TAR problems under Solaris
In-Reply-To: <dphd31$8av$1@sea.gmane.org>
References: <dphd31$8av$1@sea.gmane.org>
Message-ID: <1136409821.10363.59.camel@geddy.wooz.org>

On Wed, 2006-01-04 at 22:01 +0100, Reinhold Birkenfeld wrote:
> Recently, someone on dclpy posted about an error he got
> when he tried to unpack the Python distribution tarball
> with Sparc Solaris 9's tar:
> 
> tar: directory checksum error
> 
> With GNU tar, it worked correctly.
> 
> Is this a known issue, or is it irrelevant?

Dunno, but I'm always having problems w/ Solaris tar, so I just use GNU
tar on Solaris. ;)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060104/5249bed0/attachment.pgp 

From martin at v.loewis.de  Thu Jan  5 00:36:40 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 00:36:40 +0100
Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907-
	python/trunk/Makefile.pre.in)
In-Reply-To: <1136406410.10342.36.camel@geddy.wooz.org>
References: <20060103143057.4DB8E1E401C@bag.python.org>	<ee2a432c0601032205s684f18f8o95d540345eaadacf@mail.gmail.com>	<43BB9562.2070404@acm.org>	<200601050238.51766.anthony@interlink.com.au>
	<1136406410.10342.36.camel@geddy.wooz.org>
Message-ID: <43BC5C08.1010600@v.loewis.de>

Barry Warsaw wrote:
> Unfortunately, /usr/bin/type doesn't seem to accept the -t flag for me
> on Solaris 9.  Okay, so what's the best (read: portable) way to do this?

The portable way would be to check for svnversion in configure, and then
only use it if it was found. You could also check for .svn in configure,
and generate the entire buildno generation.

OTOH, I also think we should get rid of buildno entirely. Instead,
svnversion should be compiled into the object file, or, if it is absent,
$Revision$ should be used; the release process should be updated to
force a commit to the tag/Modules/buildno.c right after creating the
tag. sys.build_number should go, and be replaced with sys.svn_info,
which should also include the branch from which the checkout/export
was made. $Revision$ should only be trusted if it comes from a
tag/.

Should I write a PEP for that?

Regards,
Martin

From martin at v.loewis.de  Thu Jan  5 00:45:34 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 00:45:34 +0100
Subject: [Python-Dev] Real time behaviour of Pythons memory management;
 WAS: RE: Documentation about Python's GC,python-dev list
 messages	referenced in Modules/gcmodule.c notreachable anymore
In-Reply-To: <52D00EBDF4ECC74389E3B4F3D39F8A290FC575@ZHEXCHANGE.schmid-telecom.com>
References: <52D00EBDF4ECC74389E3B4F3D39F8A290FC575@ZHEXCHANGE.schmid-telecom.com>
Message-ID: <43BC5E1E.5020304@v.loewis.de>

Weber, Gregoire wrote:
> Questions
> ---------

Notice that this article is not appropriate for python-dev.
If you ask "is it the case that Python behaves such and such?",
you have a python-list (comp.lang.python) question. For
python-dev, the question should be formulated as "because
it is such and such, would it be acceptable to change it
so and so?"

> 1. Am I correct that in a system which instantiates a lot*) 
>    of containerish objects without destructing them again the 
>    GC may be triggered to evaluate all generations which may 
>    be very costy (see above)?

Correct.

> 2. In a system where only generation 0 and 1 get evaluated
>    (because not so much object got instantiated without beeing 
>    destructed) the costs are relatively low. Correct?

Correct. If the objects are *really* short-lived, a generation
0 collection might never get initiated, even though millions
of containers are created.

Regards,
Martin

From trentm at ActiveState.com  Thu Jan  5 03:28:55 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 4 Jan 2006 18:28:55 -0800
Subject: [Python-Dev] buildbot
In-Reply-To: <20060104.003058.48524852.warner@lothar.com>
References: <20060104.003058.48524852.warner@lothar.com>
Message-ID: <20060105022855.GB5183@activestate.com>

[Brian Warner wrote]
>...
> The only lingering issues are with status delivery: the main HTML
> "Waterfall" HTML will interleave builds of both branches on the same
> page, which could be a bit confusing (if the top line is red, does
> that mean the trunk is broken, or the 2.4 branch?). 

Or for separate logic projects being built with the same builtbot
master. For example, say Python's buildbot wanted to do regular builds
and tests of the distutils tree
(http://svn.python.org/view/distutils/trunk/).

> Fixing this (by creating separate pages for each branch) is high on
> the TODO list.

I'm keen to help with that if I can. I'm now subscribed to
buildbot-devel (as trentm at gmail) so I can pester you about that there
further.

Cheers,
Trent

-- 
Trent Mick
trentm at activestate.com

From trentm at ActiveState.com  Thu Jan  5 03:35:55 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 4 Jan 2006 18:35:55 -0800
Subject: [Python-Dev] buildbot
In-Reply-To: <20060105022855.GB5183@activestate.com>
References: <20060104.003058.48524852.warner@lothar.com>
	<20060105022855.GB5183@activestate.com>
Message-ID: <20060105023555.GC5183@activestate.com>

[Trent Mick wrote]
> Or for separate logic projects being built with the same builtbot

s/logic/logical/


Trent

-- 
Trent Mick
trentm at activestate.com

From skip at pobox.com  Thu Jan  5 04:23:07 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 4 Jan 2006 21:23:07 -0600
Subject: [Python-Dev] buildbot
In-Reply-To: <5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
	<43BB04D6.5080302@v.loewis.de>
	<3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
	<5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com>
Message-ID: <17340.37147.878933.687304@montanaro.dyndns.org>


    Ronald> This should do it, although I haven't tested this on OSX 10.3:

Not quite.  On my 10.3 system MAC_OS_X_VERSION_10_<N> for <N> in 0, 1, 2, 3,
4 is defined.  However, MAC_OS_X_VERSION_MAX_ALLOWED is defined to be
MAC_OS_X_VERSION_10_3.

This works for me (compiles with no warnings, passes all tests).

Skip

% svn diff Modules/getpath.c
Index: Modules/getpath.c
===================================================================
--- Modules/getpath.c   (revision 41914)
+++ Modules/getpath.c   (working copy)
@@ -381,8 +381,12 @@
     NSModule pythonModule;
 #endif
 #ifdef __APPLE__
+#if MAC_OS_X_VERSION_MAX_ALLOWED >= MAC_OS_X_VERSION_10_4
     uint32_t nsexeclength = MAXPATHLEN;
+#else
+    unsigned long nsexeclength = MAXPATHLEN;
 #endif
+#endif
 
        /* If there is no slash in the argv0 path, then we have to
         * assume python is on the user's $PATH, since there's no

From anthony at interlink.com.au  Thu Jan  5 04:54:44 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu, 5 Jan 2006 14:54:44 +1100
Subject: [Python-Dev] TAR problems under Solaris
In-Reply-To: <1136409821.10363.59.camel@geddy.wooz.org>
References: <dphd31$8av$1@sea.gmane.org>
	<1136409821.10363.59.camel@geddy.wooz.org>
Message-ID: <200601051454.46472.anthony@interlink.com.au>

On Thursday 05 January 2006 08:23, Barry Warsaw wrote:
> Dunno, but I'm always having problems w/ Solaris tar, so I just use
> GNU tar on Solaris. ;)

Maybe we should switch to cpio-based distributions?

Anthony

From nnorwitz at gmail.com  Thu Jan  5 09:39:39 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 5 Jan 2006 00:39:39 -0800
Subject: [Python-Dev] bsddb broken
Message-ID: <ee2a432c0601050039u5f7a0b1dkbf0f2ce847220014@mail.gmail.com>

Anyone wanna give bsddb some tlc?  Head and 2.4 work with 4.2 on amd64
and x86. Neither python version works when using BSD db 4.1 or 3.2.  I
don't know anything about bsddb, so any help fixing this would be
appreciated.

In 4.1 it seems associate doesn't work.

  http://python.org/sf/1332873

3.2 has other problems.

n

From martin at v.loewis.de  Thu Jan  5 10:33:29 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 10:33:29 +0100
Subject: [Python-Dev] TAR problems under Solaris
In-Reply-To: <200601051454.46472.anthony@interlink.com.au>
References: <dphd31$8av$1@sea.gmane.org>	<1136409821.10363.59.camel@geddy.wooz.org>
	<200601051454.46472.anthony@interlink.com.au>
Message-ID: <43BCE7E9.90602@v.loewis.de>

Anthony Baxter wrote:
>>Dunno, but I'm always having problems w/ Solaris tar, so I just use
>>GNU tar on Solaris. ;)
> 
> 
> Maybe we should switch to cpio-based distributions?

Peace, brother... Err, pax(1).

-bash-3.00$ uname -a
SunOS tb3 5.10 Generic sun4u sparc SUNW,Sun-Fire-V440
-bash-3.00$ bunzip2 -c Python-2.4.2.tar.bz2 |pax -r
pax: ././@LongLink : Unknown filetype

This message refers to the links in
Mac/OSXResources/app/Resources/English.lproj/Documentation/ide

Apart from that, pax extracts it the same way as gtar.

Regards,
Martin

P.S. I always ROTFL when I see mentioning of the "tar wars"
in the POSIX standard.

From skip at pobox.com  Thu Jan  5 11:53:00 2006
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 5 Jan 2006 04:53:00 -0600
Subject: [Python-Dev] buildbot
In-Reply-To: <17340.37147.878933.687304@montanaro.dyndns.org>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
	<43BB04D6.5080302@v.loewis.de>
	<3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
	<5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com>
	<17340.37147.878933.687304@montanaro.dyndns.org>
Message-ID: <17340.64140.68114.605704@montanaro.dyndns.org>


    me> This works for me (compiles with no warnings, passes all tests).
    ...

The bagpipe didn't say "no", so I checked this in on trunk and the 2.4
branch.

Skip



From ronaldoussoren at mac.com  Thu Jan  5 12:07:46 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 5 Jan 2006 12:07:46 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <17340.64140.68114.605704@montanaro.dyndns.org>
References: <ee2a432c0601021706p43cd8001j1b7e1b1547a7bc2a@mail.gmail.com>
	<17337.60315.618431.480677@montanaro.dyndns.org>
	<ee2a432c0601022230q628acbd9i5c4669a8cc4b5e53@mail.gmail.com>
	<727E07C9-AE95-4FE4-A4AA-7AFD6044990C@redivi.com>
	<17338.40044.896652.583651@montanaro.dyndns.org>
	<1F024C8F-E68E-40F7-991A-AE25B6616CFF@redivi.com>
	<43BB04D6.5080302@v.loewis.de>
	<3E7B7F37-D4F0-4835-9323-89BFA100149B@redivi.com>
	<5D707323-97AF-4939-A95F-EFC48DBC6D3D@mac.com>
	<17340.37147.878933.687304@montanaro.dyndns.org>
	<17340.64140.68114.605704@montanaro.dyndns.org>
Message-ID: <66D28935-C68C-4E92-9DDE-D41B308A5CC3@mac.com>


On 5-jan-2006, at 11:53, skip at pobox.com wrote:

>
>     me> This works for me (compiles with no warnings, passes all  
> tests).
>     ...
>
> The bagpipe didn't say "no", so I checked this in on trunk and the 2.4
> branch.

I haven't tested this on 10.4 yet, but it should work. The heuristic  
is false for anyone that will try to build a 10.3-only binary on  
10.4, but I'd be surprised if that would work anyway.

Ronald

>
> Skip
>
>


From xavier.morel at masklinn.net  Wed Jan  4 00:31:49 2006
From: xavier.morel at masklinn.net (Morel Xavier)
Date: Wed, 04 Jan 2006 00:31:49 +0100
Subject: [Python-Dev] Buildbot questions
Message-ID: <43BB0965.5080301@masklinn.net>

	I currently have a (quite weak) computer that mostly sits idle (shares 
the web connection), Tbird 750; 640Mb RAM; Windows Server 2003 Standard 
Edition.

	Since the computer sits there doing nothing, I could probably put a 
buildbot on it if needed (since the python-dev thread states that many 
windows flavour would be appreciable and that Win2003 may not be 
extremely common), but i'd like to know how often it'll try to build, 
and how long the build itself may take on such a platform.

Morel Xavier

From skip at pobox.com  Thu Jan  5 14:38:21 2006
From: skip at pobox.com (skip at pobox.com)
Date: Thu, 5 Jan 2006 07:38:21 -0600
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BB0965.5080301@masklinn.net>
References: <43BB0965.5080301@masklinn.net>
Message-ID: <17341.8525.39858.703311@montanaro.dyndns.org>

    Morel> but i'd like to know how often it'll try to build, and how long
    Morel> the build itself may take on such a platform.

It should build every time there's a checkin on trunk or the 2.4 branch.  As
for performance, take a look at

    http://www.python.org/dev/buildbot/

to see how long some of the other build bots take and extrapolate from that.

Skip


From barry at python.org  Thu Jan  5 14:38:18 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 05 Jan 2006 08:38:18 -0500
Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of
	r41907-	python/trunk/Makefile.pre.in)
In-Reply-To: <43BC5C08.1010600@v.loewis.de>
References: <20060103143057.4DB8E1E401C@bag.python.org>
	<ee2a432c0601032205s684f18f8o95d540345eaadacf@mail.gmail.com>
	<43BB9562.2070404@acm.org>	<200601050238.51766.anthony@interlink.com.au>
	<1136406410.10342.36.camel@geddy.wooz.org>
	<43BC5C08.1010600@v.loewis.de>
Message-ID: <1136468298.15418.43.camel@geddy.wooz.org>

On Thu, 2006-01-05 at 00:36 +0100, "Martin v. L?wis" wrote:

> The portable way would be to check for svnversion in configure, and then
> only use it if it was found. You could also check for .svn in configure,
> and generate the entire buildno generation.
> 
> OTOH, I also think we should get rid of buildno entirely. Instead,
> svnversion should be compiled into the object file, or, if it is absent,
> $Revision$ should be used; the release process should be updated to
> force a commit to the tag/Modules/buildno.c right after creating the
> tag. sys.build_number should go, and be replaced with sys.svn_info,
> which should also include the branch from which the checkout/export
> was made. $Revision$ should only be trusted if it comes from a
> tag/.
> 
> Should I write a PEP for that?

To be honest I don't think we need a PEP for that.  I saw that you
checked in a bunch of stuff here, and that's great, thanks!

I was working on a patch to add a PY_BUILDNO macro to
Include/patchlevel.h, which would have "$Revision$" as its value.
patchlevel.h seems like the natural place to put this, since any release
manager is going to be modifying this file anyway.  PEP 101 should be
updated so that updating patchlevel.h be the last commit before an svn
export is done (but that PEP needs an overhaul anyway to change the cvs
references into svn commands).

Thoughts?
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060105/f8c0e376/attachment.pgp 

From anthony at interlink.com.au  Thu Jan  5 15:53:45 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 6 Jan 2006 01:53:45 +1100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <17341.8525.39858.703311@montanaro.dyndns.org>
References: <43BB0965.5080301@masklinn.net>
	<17341.8525.39858.703311@montanaro.dyndns.org>
Message-ID: <200601060153.47011.anthony@interlink.com.au>

FWIW, I have an older box running Ubuntu 05.10 that spends most of 
it's days pining for stuff to do (at the moment, it does DHCP and DNS 
for the house. Yes, I know I have too many damn computers here). I 
can set up a buildbot on it easily enough. It's something like a 
600MHz P3 or something. Is it going to be useful to have this in the 
mix? 

Heck, there's a pile of 500MHz P3s sitting here that I could drop a 
random free unix onto if someone wants to nominate something that's
 
a) useful
b) not a total pain in the clacker to install. 


From tim.peters at gmail.com  Thu Jan  5 16:43:56 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 5 Jan 2006 10:43:56 -0500
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BB0965.5080301@masklinn.net>
References: <43BB0965.5080301@masklinn.net>
Message-ID: <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>

[Morel Xavier]
>         I currently have a (quite weak) computer that mostly sits idle (shares
> the web connection), Tbird 750; 640Mb RAM; Windows Server 2003 Standard
> Edition.
>
>         Since the computer sits there doing nothing, I could probably put a
> buildbot on it if needed (since the python-dev thread states that many
> windows flavour would be appreciable and that Win2003 may not be
> extremely common), but i'd like to know how often it'll try to build,
> and how long the build itself may take on such a platform.

A problem for Windows buildbot slaves is that they need an appropriate
compiler.  Does this machine have MS VC 7.1 installed?  If not, it
can't compile the code.  The Windows Python would also like to build
several other packages (like bz2 and Tcl/Tk).

An "update" style of slave does an svn update rather than a full
checkout, and that usually goes very fast after the first time. 
Likewise compiling if binaries are left behind across runs.

For the rest, open a DOS box on this machine, cd to root of a Python
2.4.2 installation, and time

    python Lib\test\regrtest.py -uall

That's about how long it will take on that machine to run all the
tests from the current trunk too.  Really thorough tests take 8x as
long (with and without purging .pyc/.pyo files first, with and without
-O, and under release and debug builds: 2*2*2 = 8).

From fwierzbicki at gmail.com  Thu Jan  5 19:17:50 2006
From: fwierzbicki at gmail.com (Frank Wierzbicki)
Date: Thu, 5 Jan 2006 13:17:50 -0500
Subject: [Python-Dev] Jython and CPython
In-Reply-To: <bb8868b90512131627m62a86d3bl11016bc6c66a46e4@mail.gmail.com>
References: <dnmlem$1a9$1@sea.gmane.org> <439F2A3F.4000703@v.loewis.de>
	<bb8868b90512131627m62a86d3bl11016bc6c66a46e4@mail.gmail.com>
Message-ID: <4dab5f760601051017m4fdaf4f1l905c18b2b9f29b50@mail.gmail.com>

>  > If the portability problem can be solved by checking things into Jython
>  > instead, I think I would prefer that.
I'm definitely interested in bringing ElementTree into Jython at some
point, though I probably won't have time to look into it until after
the next Jython release.  I'm quite sure that we can work something
out so that Jython-specific portability code can reside in Jython
only.  Though whenever it is cleanly do-able, I'd like to be able to
use the python libraries unchanged.  It sounds like this is a case
where it is not that clean to do...

-Frank

From thomas at xs4all.net  Thu Jan  5 19:30:08 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 5 Jan 2006 19:30:08 +0100
Subject: [Python-Dev] file.next() vs. file.readline()
In-Reply-To: <ca471dc20601041010r68d7bcd8q5d9bb9ae0c664a27@mail.gmail.com>
References: <20060104163419.GF18916@xs4all.nl>
	<ca471dc20601041010r68d7bcd8q5d9bb9ae0c664a27@mail.gmail.com>
Message-ID: <20060105183008.GG18916@xs4all.nl>

On Wed, Jan 04, 2006 at 10:10:07AM -0800, Guido van Rossum wrote:

> I'd say go right ahead and submit a change to SF (and then after it's
> reviewed you can check it in yourself :-).

http://www.python.org/sf?id=1397960

The patch comments and source should explain it all. The diff is quite big
(just over 16k) because of the 16k testfile. As for checking in myself, I
think my write-access was (rightly, considering my absense) removed sometime
last year :)

> In Py3K I want to revise the whole I/O stack to be independent from C
> stdio (except on those platforms where that's the only I/O you have.)

When I first read that, I thought "yuuuuck". Then I went and actually read
all of fileobject.c, went "yuuuuck" more times than I cared to count, and
now I wholeheartedly agree ;)

Changing the read* methods to use the file-iteration buffer would be a lot
simpler than I thought, by the way. And it would simplify a lot of the code
(not just because of the code duplication currently necessary.) I'm sure
there'll be people who like having direct control over howmuch gets read
though. Which is fine, I think that should stay possible, but I don't think
those people deserve to get the full force of Tim's optimizations either.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From arigo at tunes.org  Thu Jan  5 20:38:33 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 5 Jan 2006 20:38:33 +0100
Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of r41907-
	python/trunk/Makefile.pre.in)
In-Reply-To: <43BC5C08.1010600@v.loewis.de>
References: <20060103143057.4DB8E1E401C@bag.python.org>
	<ee2a432c0601032205s684f18f8o95d540345eaadacf@mail.gmail.com>
	<43BB9562.2070404@acm.org>
	<200601050238.51766.anthony@interlink.com.au>
	<1136406410.10342.36.camel@geddy.wooz.org>
	<43BC5C08.1010600@v.loewis.de>
Message-ID: <20060105193833.GA11736@code1.codespeak.net>

Hi Martin,

On Thu, Jan 05, 2006 at 12:36:40AM +0100, "Martin v. L?wis" wrote:
> OTOH, I also think we should get rid of buildno entirely. Instead,
> svnversion should be compiled into the object file, or, if it is absent,
> $Revision$ should be used; the release process should be updated to
> force a commit to the tag/Modules/buildno.c right after creating the
> tag. sys.build_number should go, and be replaced with sys.svn_info,
> which should also include the branch from which the checkout/export
> was made. $Revision$ should only be trusted if it comes from a
> tag/.

All this sounds good.

> Should I write a PEP for that?

I agree with Barry that it's overkill to ask for PEPs for too many small
details.


A bientot,

Armin

From arigo at tunes.org  Thu Jan  5 20:45:38 2006
From: arigo at tunes.org (Armin Rigo)
Date: Thu, 5 Jan 2006 20:45:38 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43B50B64.20904@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
	<43B50B64.20904@v.loewis.de>
Message-ID: <20060105194538.GB11736@code1.codespeak.net>

Hi Martin,

On Fri, Dec 30, 2005 at 11:26:44AM +0100, "Martin v. L?wis" wrote:
> > Hum.  It would be much cleaner to introduce a new format character to
> > replace '#' and deprecate '#'...
> 
> That would certainly be clearer. What character would you suggest?
> 
> I see two drawbacks with that approach:
> 1. writing backwards-compatible modules will become harder.
>    Users have to put ifdefs around the ParseTuple calls (or atleast
>    around the format strings)

Ok, I see your point.

In theory we could reuse a macro-based trick in C extensions:

#include <Python.h>
#ifndef Py_SIZE_CHR
typedef int Py_Ssize_t;
#define Py_SIZE_CHR "#"
#endif

And then we can replace -- say -- the format string "is#s#" with

    "is" Py_SIZE_CHR "s" Py_SIZE_CHR

But it's rather cumbersome.

An equally strange alternative would be to start C modules like this:

#define Py_Ssize_t int  /* compatibility with Python <= 2.4 */
#include <Python.h>

This would do the right thing for <= 2.4, using ints everywhere; and the
Python.h version 2.5 would detect the #define and assume it's a
2.5-compatible module, so it would override the #define with the real
thing *and* turn on the ssize_t interpretation of the '#' format
character.

Not that I think that this is a good idea.  Just an idea.

I still don't like the idea of a magic #define that changes the behavior
of '#include <Python.h>', but I admit I don't find any better solution.
I suppose I'll just blame C.


A bientot,

Armin

From jjl at pobox.com  Thu Jan  5 20:52:22 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 5 Jan 2006 19:52:22 +0000 (UTC)
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
Message-ID: <Pine.LNX.4.58.0601051915060.6240@alice>

On Thu, 5 Jan 2006, Tim Peters wrote:
[...]
> A problem for Windows buildbot slaves is that they need an appropriate
> compiler.  Does this machine have MS VC 7.1 installed?  If not, it
> can't compile the code.  The Windows Python would also like to build
> several other packages (like bz2 and Tcl/Tk).

Might a buildbot running this setup of David Munman's (free MS compiler +
NAnt interpreting the MS project file) be useful?

http://groups.google.co.uk/group/comp.lang.python/browse_frm/thread/226584dd47047bb6/1e33ad19197bee20


(I'm not offering a buildbot myself, just pointing out the possibility of
using this tool)

There always the chance it gets something wrong or falls over while trying
to interpret the project file, of course.  That in itself would be
beneficial, though, if there's a committer willing to fix it when it
breaks -- I currently have access to MS compilers, but I remember many
happy :-/ hours as a graduate student trying to compile Fortran and C
extensions on win32 with free compilers.


> An "update" style of slave does an svn update rather than a full
> checkout, and that usually goes very fast after the first time. 
> Likewise compiling if binaries are left behind across runs.
[...]

Much though I like SVN, it seems its working copy management still leaves
a little to be desired:  Even quite recently (fairly sure it was client
version 1.2.*, on Win XP) and with read-only checkouts used only for
builds, I've still seen it end up in an incorrect state.  I suspect 'svn
switch' or 'svn up -r xxxxx' was the culprit, though, so perhaps it's not
a problem if exactly 'svn up' is the only svn command ever executed on the
checkout.  Still, perhaps it's wise to wipe the checkout every so often?


John

From martin at v.loewis.de  Thu Jan  5 21:41:08 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 21:41:08 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <20060105194538.GB11736@code1.codespeak.net>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
	<43B50B64.20904@v.loewis.de>
	<20060105194538.GB11736@code1.codespeak.net>
Message-ID: <43BD8464.4060801@v.loewis.de>

Armin Rigo wrote:
> This would do the right thing for <= 2.4, using ints everywhere; and the
> Python.h version 2.5 would detect the #define and assume it's a
> 2.5-compatible module, so it would override the #define with the real
> thing *and* turn on the ssize_t interpretation of the '#' format
> character.

This would be very similar to the PY_SIZE_T_CLEAN approach, except
that it would also help to detect spelling mistakes.

>From an implementation point of view, the real challenge is to
give PyArg_ParseTuple a different meaning; I do this be #defining
it to PyArg_ParseTupleSsize_t (to preserve binary compatibility
for the original interpretation of ParseTuple). Putting
additional flags arguments in the entire code is also quite
hackish.

> I still don't like the idea of a magic #define that changes the behavior
> of '#include <Python.h>', but I admit I don't find any better solution.
> I suppose I'll just blame C.

More precisely, the printf style of function calling, and varargs
functions. ISO C is pretty type safe, but with varargs functions,
you lose that completely.

I still hope I can write a C parser some day that does
ParseTuple/BuildValue checking.

Regards,
Martin

From martin at v.loewis.de  Thu Jan  5 21:44:32 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 21:44:32 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <200601060153.47011.anthony@interlink.com.au>
References: <43BB0965.5080301@masklinn.net>	<17341.8525.39858.703311@montanaro.dyndns.org>
	<200601060153.47011.anthony@interlink.com.au>
Message-ID: <43BD8530.1000606@v.loewis.de>

Anthony Baxter wrote:
> FWIW, I have an older box running Ubuntu 05.10 that spends most of 
> it's days pining for stuff to do (at the moment, it does DHCP and DNS 
> for the house. Yes, I know I have too many damn computers here). I 
> can set up a buildbot on it easily enough. It's something like a 
> 600MHz P3 or something. Is it going to be useful to have this in the 
> mix? 

With the gentoo installation, I think we have "enough" linux for the
moment. Somebody noticed that the Waterfall view of buildbot quickly
becomes unreadable if there are too many builds.

> Heck, there's a pile of 500MHz P3s sitting here that I could drop a 
> random free unix onto if someone wants to nominate something that's
>  
> a) useful
> b) not a total pain in the clacker to install. 

For a), I think one of the BSDs might be useful. Whether they qualify
for b), I don't know.

Regards,
Martin

From martin at v.loewis.de  Thu Jan  5 21:47:21 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 21:47:21 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <Pine.LNX.4.58.0601051915060.6240@alice>
References: <43BB0965.5080301@masklinn.net>	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
Message-ID: <43BD85D9.5030301@v.loewis.de>

John J Lee wrote:
> Might a buildbot running this setup of David Munman's (free MS compiler +
> NAnt interpreting the MS project file) be useful?

I feel that any contribution here takes quite some time initially, so
somebody making that offer should accept some pain until it really
works self-contained. I would need to know the exact sequence of
commands that are necessary for the build; I assume the autoconf
builder would be useless.

Regards,
Martin

From anthony at interlink.com.au  Thu Jan  5 22:08:08 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 6 Jan 2006 08:08:08 +1100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BD8530.1000606@v.loewis.de>
References: <43BB0965.5080301@masklinn.net>
	<200601060153.47011.anthony@interlink.com.au>
	<43BD8530.1000606@v.loewis.de>
Message-ID: <200601060808.13439.anthony@interlink.com.au>

On Friday 06 January 2006 07:44, Martin v. L?wis wrote:
> With the gentoo installation, I think we have "enough" linux for
> the moment. Somebody noticed that the Waterfall view of buildbot
> quickly becomes unreadable if there are too many builds.

My only concern is that it's gentoo, not just linux. I know that for a 
couple of my other open source projects I usually don't spend too 
long debugging bizarrely broken apparent bugs, because it ends up 
being some strange build flag or some such on the gentoo box in 
question. On the other hand, this box is unlikely to have been built 
with a selection of gcc flags that Neal just selected randomly from 
the gcc manual <wink> so it's probably going to be better than that.

> > Heck, there's a pile of 500MHz P3s sitting here that I could drop
> > a random free unix onto if someone wants to nominate something
> > that's
> >
> > a) useful
> > b) not a total pain in the clacker to install.
>
> For a), I think one of the BSDs might be useful. Whether they
> qualify for b), I don't know.

Anyone else have an opinion on the ease of installation for the 
various BSDs? Last time I tried one (which was several years ago) it 
was Not Very Good. 

Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From bob at redivi.com  Thu Jan  5 22:12:03 2006
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 5 Jan 2006 13:12:03 -0800
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <200601060808.13439.anthony@interlink.com.au>
References: <43BB0965.5080301@masklinn.net>
	<200601060153.47011.anthony@interlink.com.au>
	<43BD8530.1000606@v.loewis.de>
	<200601060808.13439.anthony@interlink.com.au>
Message-ID: <6ECB8452-9DC1-44FE-846F-885677FE1B9F@redivi.com>


On Jan 5, 2006, at 1:08 PM, Anthony Baxter wrote:

> On Friday 06 January 2006 07:44, Martin v. L?wis wrote:
>> With the gentoo installation, I think we have "enough" linux for
>> the moment. Somebody noticed that the Waterfall view of buildbot
>> quickly becomes unreadable if there are too many builds.
>
> My only concern is that it's gentoo, not just linux. I know that for a
> couple of my other open source projects I usually don't spend too
> long debugging bizarrely broken apparent bugs, because it ends up
> being some strange build flag or some such on the gentoo box in
> question. On the other hand, this box is unlikely to have been built
> with a selection of gcc flags that Neal just selected randomly from
> the gcc manual <wink> so it's probably going to be better than that.
>
>>> Heck, there's a pile of 500MHz P3s sitting here that I could drop
>>> a random free unix onto if someone wants to nominate something
>>> that's
>>>
>>> a) useful
>>> b) not a total pain in the clacker to install.
>>
>> For a), I think one of the BSDs might be useful. Whether they
>> qualify for b), I don't know.
>
> Anyone else have an opinion on the ease of installation for the
> various BSDs? Last time I tried one (which was several years ago) it
> was Not Very Good.

FreeBSD and OpenBSD are painless these days, assuming that you're  
comfortable reading text during installation.  No experience with  
NetBSD.

-bob


From jjl at pobox.com  Thu Jan  5 22:58:25 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 5 Jan 2006 21:58:25 +0000 (UTC)
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <Pine.LNX.4.58.0601051915060.6240@alice>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
Message-ID: <Pine.LNX.4.58.0601052157260.6240@alice>

On Thu, 5 Jan 2006, John J Lee wrote:
> Might a buildbot running this setup of David Munman's (free MS compiler +
> NAnt interpreting the MS project file) be useful?

s/Munman/Murmann/


John

From martin at v.loewis.de  Thu Jan  5 23:07:31 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 23:07:31 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <200601060808.13439.anthony@interlink.com.au>
References: <43BB0965.5080301@masklinn.net>
	<200601060153.47011.anthony@interlink.com.au>
	<43BD8530.1000606@v.loewis.de>
	<200601060808.13439.anthony@interlink.com.au>
Message-ID: <43BD98A3.2060005@v.loewis.de>

Anthony Baxter wrote:
> My only concern is that it's gentoo, not just linux. I know that for a 
> couple of my other open source projects I usually don't spend too 
> long debugging bizarrely broken apparent bugs, because it ends up 
> being some strange build flag or some such on the gentoo box in 
> question.

For buildbot, failures to build are good (if they are reproducable);
if it succeeds for gentoo, but fails on some other system on which
it ought to work, then adding this system as a build slave would be
useful.

Regards,
Martin

P.S. That's assuming we get in a state where the tests "usually"
pass, of course.

From alexander.kozlovsky at gmail.com  Thu Jan  5 22:56:01 2006
From: alexander.kozlovsky at gmail.com (Alexander Kozlovsky)
Date: Fri, 6 Jan 2006 00:56:01 +0300
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
Message-ID: <611494241.20060106005601@gmail.com>

Hello!

I have some proposal for Python 3.0 (interesting one, from my point
of view). I'm sorry for my English, it is not very good.


Abstract
========

There are three different peculiarity in Python 2.x
in respect of 'self' method argument:

1. Each method must have explicit 'self' argument in its argument list
2. This argument must be used explicitly for instance's attribute value
   assignment
3. The count of explicit arguments in method definition differs from
   count of explicit parameters when method is called

Example 1 (Python 2.x):
-----------------------

    class Foo:
        def __init__(self, x):   # 1: Explicit 'self' argument
            self.x = x           # 2: 'self' must be used explicitly
        def bar(self, a, b):     # 3: There are three arguments...
            print self.x + a + b

    Foo(10).bar(20, 30)          # ...but only two explicit parameters
                                 #    is presented

This document proposes to change this, as the next example shows:

Example 2 (Python 3.0):
-----------------------

    class Foo:
        def __init__(x):         # 1: Implicit self
            .x = x               # 2: Brief form of: self.x = x
        def bar(a, b):           # 3: Two arguments...
            print .x + a + b

    Foo(10).bar(20, 30)          # ...and exactly two parameters

According  Python Zen, "Explicit is better then implicit", but
"practicality beats purity" and "Readability counts" ;-)

This draft document describes high-level semantic of proposed changes
and doesn't discuss details of C implementation.

    
Rationale
=========

When programmer tries to pick up some new programming language from
different possibilities (e.g. Smalltalk, Python, Perl, ...), he often
bases his choice on secondary and non-essential details. Almost any
language has peculiarities, which can distract potential users
from language evaluation. Examples of such warts may be #@$&% perlish
syntax in Ruby, abundance of parenthesis in Lisp and Schema, and
explicit 'self' argument in Python.

Of course, from technical point of view, such peculiarities are
completely non-issue. Parenthesis is not problem in Lisp if you use
a good text editor, perlisms in Ruby can be avoided, etc. But from
sociological and usability points of view, such warts can cause
remarkable hurt, because they affect first impression about language.

In many superficial language comparisons Python's OO approach dismissed
as after-thought because of the explicit 'self'. Example:

   But the most important aspect, why I am using Ruby instead of
   Python or Perl are the object-orientated features of Ruby, and
   that Ruby was designed object-oriented right from beginning,
   unlike Python and Perl where object-orientation was added on
   later. You can recognize this in e.g. in Python very good,
   because the first parameter (often named self) of every method of
   a class is the object on which the method is called

   (http://www.ntecs.de/old-hp/s-direktnet/rb/ruby.pdf)

   
Of course, this words about Python are not true. Python was
object-oriented from the beginning, and explicit 'self' is intentional
design decision, which is elegant in some aspects. But from
pragmatical point of view, implicit 'self' in Python 3.0 may
lower entry barrier for many people and assist Python's wide adoption.

The proposed change is not backward-compatible with Python 2.x

Detailed proposals
==================

1. 'self' becomes a keyword, and may be used inside function
   definition to denote a special implicit function argument

2. New syntax shortcut is introduced: '.attr' inside a function
   is exact equivalent to 'self.attr'. Full syntax can be used
   as well
   
3. 'class' keyword can be used in two different syntactical
   construction:
   a) If 'class' keyword immediately followed by identifier,
      then it is usual class definition.
   b) In all other cases (inside a function) 'class' keyword
      denotes value of implicit function argument

   >>> # in Python 3.0:
   >>> def sample(a, b):
   ...     class C: pass         # ordinary class definition
   ...     print class.__name__  # 'class' is implicit argument
   ...

4. Each function (not only class methods) accepts two implicit
   arguments, 'class' and 'self'. With ordinary function call,
   this arguments has 'None' value

   >>> def f(a, b):
   ...     return [class, self, a, b]
   ...
   >>> f(1, 2)
   [None, None, 1, 2]

   Implicit 'class' and 'self' attributes don't participates in
   partial function application

5. Each function have two constant attributes, __class__ and __self__,
   both of them have value 'None'

   >>> f.__class__, f.__self__
   (None, None)

6. New builtin function bind(func, self_, [class_]) is introduced.
   The result is a new function with the same name,
   __self__ and __class__ attributes of which are equal to
   self_ and class_, correspondly. If bind function is called
   without class_ argument, then __class__ value in new function
   remains the same as in previous ones.

   >>> f
   <function f[None, None] at 0x01636F30>
   >>> g = bind(f, 'foo', 'bar')
   >>> g
   <function f['foo', 'bar'] at 0x01640770>
   >>> g.__class__, g.__self__
   ('bar', 'foo')

   When function is called, its __class__ and __self__ attributes
   are accessible inside function body as 'class' and 'self':

   >>> g(10, 20)
   ['foo', 'bar', 10, 20]

   Once bounded function can be rebinded easily:

   >>> bind(g, 100, 200)
   <function f[100, 200] at 0x0165CAB0>
   
7. In process of function extraction from class or instance dictionary
   it became bound automatically (the machinery is in
   object.__getattribute__ and type.__getattribute__ methods):

   >>> def func(a, b):
   ...     return [class, self, a, b]
   ...
   >>> def Class:
   ...     func = func
   ...
   >>> instance = Class()
   >>> func(10, 20)
   [None, None, 10, 20]
   >>> Class.func(10, 20)
   [<class '__main__.Class'>, None, 10, 20]
   >>> instance.func(10, 20)
   [<class '__main__.Class'>, <__main__.Class object at 0x016434F0>, 10, 20]
   
   More examples with metaclasses:

   >>> def func(a, b):
   ...     return [class, self, a, b]
   ...
   >>> class Meta(type):
   ...     func = func
   ...
   >>> class Class:
   ...     __metaclass__ = Meta
   ...
   >>> instance = Class()
   >>> func(10, 20)
   [None, None, 10, 20]
   >>> Meta.func(10, 20)
   [<class '__main__.Meta'>, None, 10, 20]
   >>> Class.func(10, 20)
   [<class '__main__.Meta'>, <class '__main__.Class'>, 10, 20]
   >>> instance.func(10, 20)
   Traceback (most recent call last):
     ...
   AttributeError: 'Class' object has no attribute 'func'

9. Methods can be overloaded this way:

   >>> class MyList(list):
   ...     def __getitem__(key):
   ...         print "overloaded method!"
   ...         return super(MyList, self).__getitem__(key)
   ...     def __setitem__(key, value):
   ...         print "overloaded without super"
   ...         bind(list.__setitem__, self)(key, value)
   ...
   
10. It is interesting that with this semantics there are no need in
   @classmethod and @staticmethod decorators:
   - If you want to create class method, simply don't use 'self' argument
   - If you want to create static method, don't use 'class' and 'self'
   argument

   >>> class Example:
   ...     class_var = 100
   ...     def my_class_method():   # no need for @classmethod
   ...         class.class_var += 1
   ...     def my_static_method():  # no need for @staticmethod
   ...         print "hello!"
   ...     def usual_method():
   ...         print .name
   ...



What do you think about this?
   

Best regards,
 Alexander                          mailto:kozlovsky at mail.spbnit.ru


From martin at v.loewis.de  Thu Jan  5 23:18:26 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 23:18:26 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <20060104.003058.48524852.warner@lothar.com>
References: <20060104.003058.48524852.warner@lothar.com>
Message-ID: <43BD9B32.3030700@v.loewis.de>

Brian Warner wrote:
> There was also a patch[1] to add some regexps to svn_buildbot.py to do this
> kind of branch determination. I haven't been able to review it properly yet,
> but it may give you some ideas.

The patch itself is completely broken. It removes random parts of
svn_buildbot.py, so after applying it, svn_buildbot isn't even
syntactically correct anymore.

However, I adjusted the patch, so it now works fine.
(the other issue was that the regex was not entirely correct:
it assumes a trunk/<project> structure, where we have a
<project>/trunk structure).

So at the moment, I'm quite happy with buildbot. My only wish is that
it would do static pages, rather than being a web server. I set it up
as a reverse proxy (so nobody knows what the port number is, but still).

Regards,
Martin

From martin at v.loewis.de  Thu Jan  5 23:21:58 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 05 Jan 2006 23:21:58 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <20060105022855.GB5183@activestate.com>
References: <20060104.003058.48524852.warner@lothar.com>
	<20060105022855.GB5183@activestate.com>
Message-ID: <43BD9C06.3080401@v.loewis.de>

Trent Mick wrote:
> Or for separate logic projects being built with the same builtbot
> master. For example, say Python's buildbot wanted to do regular builds
> and tests of the distutils tree
> (http://svn.python.org/view/distutils/trunk/).

I believe you could always get it arranged the way you like by properly
setting up an "uninteresting changes" filter. You would have several
builders, one change source, and each builder would filter out the
changes it is interested in.

Regards,
Martin

From alexander.kozlovsky at gmail.com  Fri Jan  6 00:34:23 2006
From: alexander.kozlovsky at gmail.com (Alexander Kozlovsky)
Date: Fri, 6 Jan 2006 02:34:23 +0300
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <611494241.20060106005601@gmail.com>
References: <611494241.20060106005601@gmail.com>
Message-ID: <1542680550.20060106023423@gmail.com>

I wrote:
> 5. Each function have two constant attributes, __class__ and __self__,
>    both of them have value 'None'

Of course, this attributes have names 'im_class' and 'im_self',
as before, but can be used with any function.

I have not sleep enough last night :)



Best regards,
Alexander                        mailto:alexander.kozlovsky at gmail.com


From martin at v.loewis.de  Fri Jan  6 01:01:00 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jan 2006 01:01:00 +0100
Subject: [Python-Dev] [Buildbot-devel] Re:  buildbot
In-Reply-To: <6db0ea510601051432g5791c209j57fc6810664a7f25@mail.gmail.com>
References: <20060104.003058.48524852.warner@lothar.com>	
	<20060105022855.GB5183@activestate.com>
	<43BD9C06.3080401@v.loewis.de>
	<6db0ea510601051432g5791c209j57fc6810664a7f25@mail.gmail.com>
Message-ID: <43BDB33C.5060203@v.loewis.de>

Trent Mick wrote:
> I meant that more as a justification for improving the Waterfall
> status receiver to support separate summary pages for separate
> projects and trunks... all with the same buildbot master server.
>    python.org/dev/buildbot/python/...
>    python.org/dev/buildbot/python-release24-maint/...
>    python.org/dev/buildbot/distutils/...

Ah, right. On result presentation, there is definitely room for
improvement.

Regards,
Martin

From martin at v.loewis.de  Fri Jan  6 01:33:53 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jan 2006 01:33:53 +0100
Subject: [Python-Dev] buildno (Was: [Python-checkins] commit of
	r41907-	python/trunk/Makefile.pre.in)
In-Reply-To: <1136468298.15418.43.camel@geddy.wooz.org>
References: <20060103143057.4DB8E1E401C@bag.python.org>	
	<ee2a432c0601032205s684f18f8o95d540345eaadacf@mail.gmail.com>	
	<43BB9562.2070404@acm.org>	<200601050238.51766.anthony@interlink.com.au>	
	<1136406410.10342.36.camel@geddy.wooz.org>
	<43BC5C08.1010600@v.loewis.de>
	<1136468298.15418.43.camel@geddy.wooz.org>
Message-ID: <43BDBAF1.2040506@v.loewis.de>

Barry Warsaw wrote:
> I was working on a patch to add a PY_BUILDNO macro to
> Include/patchlevel.h, which would have "$Revision$" as its value.

As you can see, this is done now. The value appears at the Python
level only for tags, though, because it will be unreliable for the
trunk and for branches.

> patchlevel.h seems like the natural place to put this, since any release
> manager is going to be modifying this file anyway.  PEP 101 should be
> updated so that updating patchlevel.h be the last commit before an svn
> export is done (but that PEP needs an overhaul anyway to change the cvs
> references into svn commands).

It would still be one level behind: patchlevel.h gets N, then 'svn cp'
creates the tag, producing N+1. OTOH, for a tag, the revision number
is nearly irrelevant.

Regards,
Martin

From ncoghlan at gmail.com  Fri Jan  6 01:55:29 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 06 Jan 2006 10:55:29 +1000
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <611494241.20060106005601@gmail.com>
References: <611494241.20060106005601@gmail.com>
Message-ID: <43BDC001.3070602@gmail.com>

Alexander Kozlovsky wrote:
> Hello!
> 
> I have some proposal for Python 3.0 (interesting one, from my point
> of view). I'm sorry for my English, it is not very good.

Your English seems fine. About the only thing I noticed is that you have the 
meaning of 'function arguments' vs 'function parameters' switched from a 
Python point of view - the parameters are defined as part of the function 
definition, while the arguments are provided at call time. This is really a 
minor semantic quibble though - I expect most people wouldn't have any real 
trouble figuring out what you meant.

Even though I still disagree with it, this is one of the better reasoned "no 
explicit self" proposals I've encountered - even if Guido ends up not liking 
it, I believe it should still be recorded as a PEP on python.org.

To sum the proposal up in my own words:
   Eliminate the need for explicit class and self slots in class and instance 
methods by implicitly providing those slots on all functions.

The main concern I have is with the answer to the question "How many 
positional arguments does the function have if I retrieve it from the class, 
rather than from an instance?" (this is the common concern for almost all 
proposals to remove the explicit self and class_ slots).

That is, the rationale for requiring the explicit self is an outgrowth of the 
fact that methods can be retrieved directly from the class:

      class Foo:
          def __init__(self, x):   # 1: Explicit 'self' argument
              self.x = x           # 2: 'self' must be used explicitly
          def bar(self, a, b):     # 3: There are three parameters...
              print self.x + a + b

      f = Foo.bar                  # We retrieve the unbound method
      f(Foo(10), 20, 30)           # And there are three arguments at call time
      f = Foo().bar                # Using an instance binds the first argument
      f(20, 30)                    # Which means there are two arguments left

You can also bypass the type machinery entirely, and retrieve the raw function 
object from the class dictionary:

     f = Foo.__dict__["bar"] # This gives a function. . .
     f(Foo(10), 20, 30)      # which takes three arguments as written

Under the proposal being discussed, things become far less clear:

     class Foo:
         def __init__(x):         # 1: Implicit self
             .x = x               # 2: Brief form of: self.x = x
         def bar(a, b):           # 3: Two arguments...
             print .x + a + b

     f = Foo(10).bar              # We agree this accepts 2 arguments
     f = Foo.bar                  # How many arguments does f accept?
     f = Foo.__dict__["bar"]      # How many arguments does it accept now?

The answer to the first question *has* to be 3, or we lose too much 
functionality - but that's seriously surprising (the method appears to require 
two arguments when its defined, but actually requires 3 due to its being 
retrieved from a class). And it still requires that a distinction be made 
between instance, class and static methods in order to control the signature 
of the retrieved method.

However, that answer creates its own problems - what if we have a 3-argument 
function that does exactly what we want our method to do? We'd need some way 
of signalling to the class that the function should be left alone when being 
retrieved from the class, but have the first argument bound automatically when 
being retrieved from an instance.

This is where the "Explicit is better than implicit" and "Practicality beats 
purity" *both* kick in in favour of explicit self and class_ parameters - the 
signature of the retrieved function is exactly what the source code says it 
is, because there aren't any implicit parameters getting slipped into the 
parameter list when you aren't looking.

As I see it, the real issue is that people are often coming from a Java or C++ 
background where you can't manipulate functions and classes as first-class 
objects - the idea that the instance method signature could be written to 
describe the signature of the unbound method returned by "Foo.bar" rather than 
the bound method returned by "Foo().bar" is an entirely foreign concept.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From barry at python.org  Fri Jan  6 02:59:01 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 05 Jan 2006 20:59:01 -0500
Subject: [Python-Dev] buildno (Was: [Python-checkins] commit
	of	r41907-	python/trunk/Makefile.pre.in)
In-Reply-To: <43BDBAF1.2040506@v.loewis.de>
References: <20060103143057.4DB8E1E401C@bag.python.org>
	<ee2a432c0601032205s684f18f8o95d540345eaadacf@mail.gmail.com>
	<43BB9562.2070404@acm.org>	<200601050238.51766.anthony@interlink.com.au>
	<1136406410.10342.36.camel@geddy.wooz.org>
	<43BC5C08.1010600@v.loewis.de>
	<1136468298.15418.43.camel@geddy.wooz.org>
	<43BDBAF1.2040506@v.loewis.de>
Message-ID: <1136512741.15418.86.camel@geddy.wooz.org>

On Fri, 2006-01-06 at 01:33 +0100, "Martin v. L?wis" wrote:

> As you can see, this is done now. The value appears at the Python
> level only for tags, though, because it will be unreliable for the
> trunk and for branches.

Cool, thanks.  I can chuck my local diffs now. :)

> > patchlevel.h seems like the natural place to put this, since any release
> > manager is going to be modifying this file anyway.  PEP 101 should be
> > updated so that updating patchlevel.h be the last commit before an svn
> > export is done (but that PEP needs an overhaul anyway to change the cvs
> > references into svn commands).
> 
> It would still be one level behind: patchlevel.h gets N, then 'svn cp'
> creates the tag, producing N+1. OTOH, for a tag, the revision number
> is nearly irrelevant.

Unless we tagged and then modified the file in that tag as the very last
thing we do before we create the tarball.  Or is that too evil?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060105/cf8ba7be/attachment.pgp 

From tim.peters at gmail.com  Fri Jan  6 04:10:36 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 5 Jan 2006 22:10:36 -0500
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <Pine.LNX.4.58.0601051915060.6240@alice>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
Message-ID: <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>

[John J Lee]
> Might a buildbot running this setup of David Munman's (free MS compiler +
> NAnt interpreting the MS project file) be useful?
>
> http://groups.google.co.uk/group/comp.lang.python/browse_frm/thread/226584dd47047bb6/1e33ad19197bee20

No comment from me about that (don't know anything about it, and don't
have time to learn).

Someone who wants to see a platform exercised needs to volunteer that
platform themself (and realize that every stinkin' little step has to
be so well defined that the people running the buildbot _master_ can
send the exact steps needed to the slave).

> ...
> Much though I like SVN, it seems its working copy management still leaves
> a little to be desired:  Even quite recently (fairly sure it was client
> version 1.2.*, on Win XP) and with read-only checkouts used only for
> builds, I've still seen it end up in an incorrect state.  I suspect 'svn
> switch' or 'svn up -r xxxxx' was the culprit, though, so perhaps it's not
> a problem if exactly 'svn up' is the only svn command ever executed on the
> checkout.

I doubt anyone slings more svn projects, branches and tags than Zope
Corp, and I've had no problems with svn on WinXP there except when a
project switches from making copies of externals to getting them via
svn:externals instead -- and then everyone has problems, regardless of
platform.  What I _have_ had problems with is PuTTY, and recently
discovered that all my months-long "svn+ssh" problems went away after
backing off to the older PuTTY 0.57 (and come back again immediately
upon switching to 0.58).

> Still, perhaps it's wise to wipe the checkout every so often?

I think it is.  And while I haven't seen this under MS VC7.1 yet, a
few times I caught VC 6.0  failing to recompile after a relevant
header file changed.  Certainly from-scratch checkout + build should
be done before a release.

From trentm at gmail.com  Thu Jan  5 23:32:14 2006
From: trentm at gmail.com (Trent Mick)
Date: Thu, 5 Jan 2006 14:32:14 -0800
Subject: [Python-Dev] [Buildbot-devel] Re:  buildbot
In-Reply-To: <43BD9C06.3080401@v.loewis.de>
References: <20060104.003058.48524852.warner@lothar.com>
	<20060105022855.GB5183@activestate.com> <43BD9C06.3080401@v.loewis.de>
Message-ID: <6db0ea510601051432g5791c209j57fc6810664a7f25@mail.gmail.com>

> > Or for separate logic projects being built with the same builtbot
> > master. For example, say Python's buildbot wanted to do regular builds
> > and tests of the distutils tree
> > (http://svn.python.org/view/distutils/trunk/).
>
> I believe you could always get it arranged the way you like by properly
> setting up an "uninteresting changes" filter. You would have several
> builders, one change source, and each builder would filter out the
> changes it is interested in.

I meant that more as a justification for improving the Waterfall
status receiver to support separate summary pages for separate
projects and trunks... all with the same buildbot master server.
   python.org/dev/buildbot/python/...
   python.org/dev/buildbot/python-release24-maint/...
   python.org/dev/buildbot/distutils/...


Trent

--
Trent Mick
trentm at gmail.com

From kay.schluehr at gmx.net  Fri Jan  6 07:26:36 2006
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Fri, 06 Jan 2006 07:26:36 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <611494241.20060106005601@gmail.com>
References: <611494241.20060106005601@gmail.com>
Message-ID: <dpl2is$2t4$1@sea.gmane.org>

Just one short comment of your ( well written ) rationale and PEP. Ruby 
might be hyped at the moment but at least in the german labour market 
and the biggest german freelancer mediation Ruby is quite inexistent. 
Even when the nerdish blogosphere is setting different priorities SAP 
and Intel made million $ investments in Zend Corp. and PHP but not in 
RoR. The slashdot sociology is not all important.

If its just about rhetorical battles, hype and popularity Pythonistas 
would do a better service for themselves not to change Pythons object 
model but write a J2EE-like spec, standardize it and provide a reference 
implementation ( "THE agile alternative!... ").

Kay


From martin at v.loewis.de  Fri Jan  6 08:27:52 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jan 2006 08:27:52 +0100
Subject: [Python-Dev] buildno (Was: [Python-checkins] commit
	of	r41907-	python/trunk/Makefile.pre.in)
In-Reply-To: <1136512741.15418.86.camel@geddy.wooz.org>
References: <20060103143057.4DB8E1E401C@bag.python.org>	
	<ee2a432c0601032205s684f18f8o95d540345eaadacf@mail.gmail.com>	
	<43BB9562.2070404@acm.org>	<200601050238.51766.anthony@interlink.com.au>	
	<1136406410.10342.36.camel@geddy.wooz.org>
	<43BC5C08.1010600@v.loewis.de>	
	<1136468298.15418.43.camel@geddy.wooz.org>
	<43BDBAF1.2040506@v.loewis.de>
	<1136512741.15418.86.camel@geddy.wooz.org>
Message-ID: <43BE1BF8.3020606@v.loewis.de>

Barry Warsaw wrote:
>>It would still be one level behind: patchlevel.h gets N, then 'svn cp'
>>creates the tag, producing N+1. OTOH, for a tag, the revision number
>>is nearly irrelevant.
> 
> 
> Unless we tagged and then modified the file in that tag as the very last
> thing we do before we create the tarball.  Or is that too evil?

That would work, and I wouldn't see anything wrong with it. I believe it
would also work to modify the working copy, and then svn cp it (instead
of svn committing it) - atleast the svn docs suggest that you can copy
a working copy into a repo URL.

Regards,
Martin

From martin at v.loewis.de  Fri Jan  6 08:39:15 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jan 2006 08:39:15 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
References: <43BB0965.5080301@masklinn.net>	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>	<Pine.LNX.4.58.0601051915060.6240@alice>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
Message-ID: <43BE1EA3.8040405@v.loewis.de>

Tim Peters wrote:
>>Still, perhaps it's wise to wipe the checkout every so often?
> 
> 
> I think it is.  And while I haven't seen this under MS VC7.1 yet, a
> few times I caught VC 6.0  failing to recompile after a relevant
> header file changed.  Certainly from-scratch checkout + build should
> be done before a release.

I would like to do this in buildbot, but I'm not sure how to
(i.e. wipe the build occasionally, but not every time).

For example, I could imagine completely cleaning the build directory
every time the build number % 10 == 0. Still, what the precise
buildbot incantation would be, I don't know.

Regards,
Martin

From anthony at interlink.com.au  Fri Jan  6 09:16:32 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri, 6 Jan 2006 19:16:32 +1100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BE1EA3.8040405@v.loewis.de>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
Message-ID: <200601061916.38944.anthony@interlink.com.au>

On Friday 06 January 2006 18:39, Martin v. L?wis wrote:
> I would like to do this in buildbot, but I'm not sure how to
> (i.e. wipe the build occasionally, but not every time).
>
> For example, I could imagine completely cleaning the build
> directory every time the build number % 10 == 0. Still, what the
> precise buildbot incantation would be, I don't know.

At least with the way Twisted is set up, the buildbot also sits in an 
IRC channel and sends updates there. It can also be controlled from 
there. Is this worth doing? A 'force clean build' command could be 
added...

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From mwh at python.net  Fri Jan  6 12:34:35 2006
From: mwh at python.net (Michael Hudson)
Date: Fri, 06 Jan 2006 11:34:35 +0000
Subject: [Python-Dev] Automated Python testing
In-Reply-To: <43B308E0.20205@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Wed,
	28 Dec 2005 22:51:28 +0100")
References: <20051228194543.1217.425460361.divmod.quotient.8597@ohm>
	<43B308E0.20205@v.loewis.de>
Message-ID: <2mmzi9zj1w.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Currently, my buildbot isn't connected to IRC at all. If I ever
> enable that aspect, I'll use allowForce=False again to disable
> remotely invoking builds.

#python-dev on freenode is ready and waiting should you decide to
activate this :)

Cheers,
mwh

-- 
  well, take it from an old hand: the only reason it would be easier
  to program in C is that you can't easily express complex problems
  in C, so you don't.                   -- Erik Naggum, comp.lang.lisp

From skink at evhr.net  Fri Jan  6 13:10:41 2006
From: skink at evhr.net (Fabien Schwob)
Date: Fri, 06 Jan 2006 13:10:41 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <611494241.20060106005601@gmail.com>
References: <611494241.20060106005601@gmail.com>
Message-ID: <43BE5E41.10301@evhr.net>

 > Example 1 (Python 2.x):
 > -----------------------
 >
 >     class Foo:
 >         def __init__(self, x):   # 1: Explicit 'self' argument
 >             self.x = x           # 2: 'self' must be used explicitly
 >         def bar(self, a, b):     # 3: There are three arguments...
 >             print self.x + a + b
 >
 >     Foo(10).bar(20, 30)          # ...but only two explicit parameters
 >                                  #    is presented
 >
 > This document proposes to change this, as the next example shows:
 >
 > Example 2 (Python 3.0):
 > -----------------------
 >
 >     class Foo:
 >         def __init__(x):         # 1: Implicit self
 >             .x = x               # 2: Brief form of: self.x = x
 >         def bar(a, b):           # 3: Two arguments...
 >             print .x + a + b
 >
 >     Foo(10).bar(20, 30)          # ...and exactly two parameters

In my case, I think that the problem of _self_ is mainly in the method 
definition. It's a little "hard" to understand why you have to use 
myFunction(self, otherArgs) when you create a class method. But the use 
of self in the code of the method is a good thing because it allow you 
to clearly say that you are working on a class property. In my case, I 
would like to have the following syntax in Python 3.0 :

class Foo:
     def __init__(x):
	self.x = x
     def bar(a, b):
	print self.x + a + b

My 0.2? ;)

-- 
Fabien SCHWOB

From ianb at colorstudy.com  Fri Jan  6 18:14:28 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 06 Jan 2006 11:14:28 -0600
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43BDC001.3070602@gmail.com>
References: <611494241.20060106005601@gmail.com> <43BDC001.3070602@gmail.com>
Message-ID: <43BEA574.8010607@colorstudy.com>

Nick Coghlan wrote:
[...]
> Under the proposal being discussed, things become far less clear:
> 
>      class Foo:
>          def __init__(x):         # 1: Implicit self
>              .x = x               # 2: Brief form of: self.x = x
>          def bar(a, b):           # 3: Two arguments...
>              print .x + a + b
> 
>      f = Foo(10).bar              # We agree this accepts 2 arguments
>      f = Foo.bar                  # How many arguments does f accept?
>      f = Foo.__dict__["bar"]      # How many arguments does it accept now?

I think this might return things very similar to what are returned now. 
  The only reasonable alternative I can think of is that Foo.bar would 
return something that simply could not be called until it was bound, so 
this:

   inst = Foo()
   f = Foo.bar
   print f(inst, 1, 2)

would have to be translated to this this:

   inst = Foo()
   f = Foo.bar
   meth = bind(f, inst)
   print meth(1, 2)

It seems clear given the rest of the proposal that this second form will 
work, it's just a question of whether the first form will work at all. 
Personally I find the first form is rather uncommon anyway, and 50% of 
the time I write it it's a bug.  So the more explicit second form would 
be fine with me.  Well... it becomes more complex for decorators, I guess:

   def printargs(func):
       def replacement(*args, **kw):
           print args, kw
           return func(*args, **kw)
       return replacement
   class Foo:
       @printargs
       def null(a): pass

What is printargs going to print?  Will it even work?  I'd guess you'd 
have to do:

   def printargs(func):
       def replacement(*args, **kw):
           print args, kw
           return bind(func, self)(*args, **kw)
       return replacement

But what does that do if you apply printargs to a function that isn't a 
method?  I generally have found it awkard to apply decorators to both 
methods and functions, except when the decorators pass through all the 
arguments unchanged (as printargs does).  But I don't know if this is an 
improvement; I guess it depends on what bind(func, self) does outside of 
a method invocation.  I think self should be undefined in that case. 
Well, I guess you could do:

   def printargs(func):
       def replacement(*args, **kw):
           print args, kw
           try:
               bound = bind(func, self, class)
           except NameError:
               try:
                   bound = bind(func, None, class)
               except NameError:
                   bound = func
           return bound(*args, **kw)
       return replacement

Ugh.

> The answer to the first question *has* to be 3, or we lose too much 
> functionality - but that's seriously surprising (the method appears to require 
> two arguments when its defined, but actually requires 3 due to its being 
> retrieved from a class). And it still requires that a distinction be made 
> between instance, class and static methods in order to control the signature 
> of the retrieved method.
> 
> However, that answer creates its own problems - what if we have a 3-argument 
> function that does exactly what we want our method to do? We'd need some way 
> of signalling to the class that the function should be left alone when being 
> retrieved from the class, but have the first argument bound automatically when 
> being retrieved from an instance.
> 
> This is where the "Explicit is better than implicit" and "Practicality beats 
> purity" *both* kick in in favour of explicit self and class_ parameters - the 
> signature of the retrieved function is exactly what the source code says it 
> is, because there aren't any implicit parameters getting slipped into the 
> parameter list when you aren't looking.
> 
> As I see it, the real issue is that people are often coming from a Java or C++ 
> background where you can't manipulate functions and classes as first-class 
> objects - the idea that the instance method signature could be written to 
> describe the signature of the unbound method returned by "Foo.bar" rather than 
> the bound method returned by "Foo().bar" is an entirely foreign concept.

I disagree; I'm 100% comfortable and have internalized Python's use of 
an explicit self parameter, but it still doesn't feel more explicit or 
practical than the traditional way of writing out methods in other 
languages.  I'm inclined to say that this might be too substantial of a 
change to Python, but I don't think the current way method definitions 
work is particular elegant, and it hasn't really grown on me over all 
these years.  I still frequently spell method signatures incorrectly. 
And I can't explain the current situation to a new user, except to say 
"this is just the way it is", because the explanation only makes sense 
when you are much further into the language.  And don't get me started 
on the error messages when you get the parameter count wrong...


(As an aside directed at the original PEP, I think discussion of leaving 
self out of expressions, e.g., ".x" for "self.x", should be separate 
from the rest of this PEP).


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From trentm at ActiveState.com  Fri Jan  6 18:31:10 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 6 Jan 2006 09:31:10 -0800
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BE1EA3.8040405@v.loewis.de>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
Message-ID: <20060106173110.GD15276@activestate.com>

[Martin v. Loewis wrote]
> I would like to do this in buildbot, but I'm not sure how to
> (i.e. wipe the build occasionally, but not every time).
> 
> For example, I could imagine completely cleaning the build directory
> every time the build number % 10 == 0. Still, what the precise
> buildbot incantation would be, I don't know.

(Still learning my buildbot mojo.) One idea would be to do what
Mozilla's Tinderbox does: they have one set of builds that are
incremental and one set that are full. Actually looking around on
tinderbox.mozilla.org I could only find incremental builds so I'm not
sure what they are doing.

To wipe out the build occassionally you could (presumably) add a
starting step to the Python 'builder' (in the build master.cfg) to 
    rm -rf $builddir
every, say, Sunday night.


Trent

-- 
Trent Mick
trentm at activestate.com

From Martin.vonLoewis at hpi.uni-potsdam.de  Fri Jan  6 19:40:07 2006
From: Martin.vonLoewis at hpi.uni-potsdam.de (=?iso-8859-1?Q?von_L=F6wis_Martin?=)
Date: Fri, 6 Jan 2006 19:40:07 +0100
Subject: [Python-Dev] Compiler warnings for 64-bit portability problems
Message-ID: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de>

I just found that the intel compiler (icc 9.0)
also supports compiler warnings for portability
problems.

For the file

#include <sys/types.h>

int foo(size_t x)
{
        return x;
}

it says (with -Wall)

a.c(3): remark #1418: external definition with no prior declaration
  int foo(size_t x)
      ^

a.c(5): remark #1682: implicit conversion of a 64-bit integral type to a smaller integral type (potential portability problem)
        return x;
               ^

If you just want the latter warning, pass -Wp64.

Regards,
Martin

P.S. In case you want to install it on a Debian AMD64 system, 
here is the procedure:
1. register at 

https://registrationcenter.intel.com/EvalCenter/EvalForm.aspx?ProductID=408

2. download and unpack the l_cc_c_9.0.030 tarfile.

3. don't try alien, it won't work.

4. don't try the Intel install.sh, it won't work
   (atleast, it didn't work for me)

5. Instead, install with 'rpm -i --nodeps *.em64t.rpm'

6. edit /opt/intel/cce/9.0/bin/icc, replacing all
   occurrences of <INSTALLDIR> with /opt/intel/cce/9.0

7. Put the *.lic file that Intel sent you into 
   /opt/intel/cce/9.0/bin



From jimjjewett at gmail.com  Fri Jan  6 20:14:21 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Fri, 6 Jan 2006 14:14:21 -0500
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
Message-ID: <fb6fbf560601061114y38fec96t4c28dfa2cb594f37@mail.gmail.com>

Nick Coghlan wrote:

>    Eliminate the need for explicit class and self
>    slots in class and instance methods by
>    implicitly providing those slots on all functions.

> How many positional arguments does the function
> have if I retrieve it from the class, rather than from
> an instance?

To keep things simple, it should have the same
number of positional arguments no matter how
you retrieve it -- think of self and cls as keyword
arguments that we currently write in an odd manner.

>    class Foo:
>        def __init__(x):         # 1: Implicit self
>            .x = x               # 2: Brief form of: self.x = x
>        def bar(a, b):           # 3: Two arguments...
>            print .x + a + b

>    f = Foo(10).bar              # We agree this accepts 2 arguments
>    f = Foo.bar                  # How many arguments does f accept?
>    f = Foo.__dict__["bar"]      # How many arguments does it accept now?

> The answer to the first question *has* to be 3,
> or we lose too much functionality

No, it should be 2.  If you use it as a staticmethod,
there isn't any problem.  If you really want a method
outside of its context, then pass self (or cls) as a keyword.

    f(4,5,self=Foo(10))

(I also think we could use class as the keyword, since
    class=expression
is different from
    class Name:
but that is a separate question.)

If you like having self at the front -- well, wasn't
the "keywords must be last" restriction likely to
go away even before python 3.0?

> ... a Java or C++ background where
> you can't manipulate functions and classes as
> first-class objects - the idea that the instance
> method signature could be written to describe
> the signature of the unbound method returned
> by "Foo.bar" rather than the bound method
> returned by "Foo().bar" is an entirely foreign
> concept.

I find it sort of strange that the signatures are
different in the first place.  I prefer the one
without boilerplate.

-jJ

From arigo at tunes.org  Fri Jan  6 20:34:48 2006
From: arigo at tunes.org (Armin Rigo)
Date: Fri, 6 Jan 2006 20:34:48 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <611494241.20060106005601@gmail.com>
References: <611494241.20060106005601@gmail.com>
Message-ID: <20060106193448.GA656@code1.codespeak.net>

Hi Alexander,

On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote:
> There are three different peculiarity in Python 2.x
> in respect of 'self' method argument:

Yuk!  This has been discussed again and again already.  *Please* move
this discussion to comp.lang.python.


A bientot,

Armin

From alexander.kozlovsky at gmail.com  Fri Jan  6 21:26:55 2006
From: alexander.kozlovsky at gmail.com (Alexander Kozlovsky)
Date: Fri, 6 Jan 2006 23:26:55 +0300
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
Message-ID: <1325280794.20060106232655@gmail.com>

Hello!

Ian Bicking wrote:
> (As an aside directed at the original PEP, I think discussion of leaving
> self out of expressions, e.g., ".x" for "self.x", should be separate 
> from the rest of this PEP).

Yes, I'm fully agree.


Nick Coghlan wrote:
> The main concern I have is with the answer to the question "How many
> positional arguments does the function have if I retrieve it from the class, 
> rather than from an instance?" (this is the common concern for almost all 
> proposals to remove the explicit self and class_ slots).

In this case, the function always (!!!) has fixed number of
positional arguments equal to number of positional parameters
inside the function definition:

  >>> def func(a, b):
  ...     print class, self, a, b
  ...
  >>> class Foo:
  ...     bar = func
  ...
  >>> instance = Foo()
  >>> func(1, 2)          # there are 2 arguments
  None None 1 2           #   <- class and self both equal to None
  >>> Foo.bar(1, 2)       # 2 arguments again!!
  <class Foo> None 1 2    #   <- self is equal to None
  >>> instance.bar(1, 2)  # 2 arguments always!!!
  <class Foo> <Foo object at 0x016434F0> 1 2

  
Nick Coghlan wrote:
> To sum the proposal up in my own words:
>    Eliminate the need for explicit class and self slots in class
> and instance methods by implicitly providing those slots on all
> functions.

Yes, I think, it is what I mean.

With my proposal, 'self' is no longer the first explicit or implicit
positional argument (!), instead, it is 'pseudo-argument', the value
of which is equal to function's 'im_self' attribute. Any function
contains two special read-only attributes: 'im_class' and 'im_self',
and values of this attributes are accessible inside function body
as values of 'class' and 'self' pseudo-arguments.

Any function has this attributes, for ordinary function their values
are set to 'None'. Example:

  >>> # Python 3.0
  >>> def f(): return class, self
  >>> f()
  (None, None)

There is new built-in function:

    bind(old_function, self_, class_=None) -> new_function
  
The result is the new function with the same code object which the
old function had, but with different im_class and im_self attributes
values.

- If both self_ and class_ arguments are None, then 'im_class' and
  'im_self' attributes of new function is set to equal to None.

- If self_ is not None, then class_ must be None or self_.__class__.
  'im_self' attribute of new function is set to self_, and im_class
  attribute is set to self_.__class__

- If self_ is None but class_ is not None, then 'im_self' attribute
  is set to None, and 'im_class' attribute is set to class_

Consider this expression (extraction of a method from a class):

   x = Foo.bar

This expression is equivalent of:

   x = bind(Foo.__dict__["bar"], None, Foo)

After this, x.im_class is Foo and x.im_self is None
The type.__getattribute__ contains implementation of this
   
Consider next expression (extraction of a method from an instance):

   y = instance.bar

This expression is equivalent of:

   y = bind(instance.__class__.__dict__["bar"], instance)

After this, y.im_class is instance.__class__ and y.im_self is instance.
The object.__getattribute__ is responsible for this

Ian Bicking wrote:
> Well... it becomes more complex for decorators, I guess:
> 
>    def printargs(func):
>        def replacement(*args, **kw):
>            print args, kw
>            return func(*args, **kw)
>        return replacement
>    class Foo:
>        @printargs
>        def null(a): pass
> 
> What is printargs going to print?  Will it even work?  I'd guess you'd 
> have to do:
> 
>    def printargs(func):
>        def replacement(*args, **kw):
>            print args, kw
>            return bind(func, self)(*args, **kw)
>        return replacement

I think, it should be:

    def printargs(func):
        def replacement(*args, **kw):
            print args, kw
            return bind(func, self, class)(*args, **kw)
        return replacement

Yep, the code in decorators will be more complicated than it is today.
I did not get it before...

> I guess it depends on what bind(func, self) does outside of
> a method invocation.  I think self should be undefined in that case.

'self' will be None in that case. Any function has 'self' pseudo-argument,
but in a plain function (not a method) 'self' and 'class' both equal
to None.



-- 
Best regards,
 Alexander                mailto:alexander.kozlovsky at gmail.com


From guido at python.org  Fri Jan  6 22:19:16 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 6 Jan 2006 13:19:16 -0800
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060106193448.GA656@code1.codespeak.net>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
Message-ID: <ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>

On 1/6/06, Armin Rigo <arigo at tunes.org> wrote:
> On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote:
> > There are three different peculiarity in Python 2.x
> > in respect of 'self' method argument:
>
> Yuk!  This has been discussed again and again already.  *Please* move
> this discussion to comp.lang.python.

Yes please. This won't change.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Fri Jan  6 23:02:28 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 06 Jan 2006 23:02:28 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <20060106173110.GD15276@activestate.com>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
	<20060106173110.GD15276@activestate.com>
Message-ID: <43BEE8F4.5080304@v.loewis.de>

Trent Mick wrote:
> (Still learning my buildbot mojo.) One idea would be to do what
> Mozilla's Tinderbox does: they have one set of builds that are
> incremental and one set that are full. Actually looking around on
> tinderbox.mozilla.org I could only find incremental builds so I'm not
> sure what they are doing.

I know how to set up full vs. incremental *builders*; they would
appear in different lanes in the Waterfall, and they would operate
on different build directories. I don't think I want additional
lanes to the Waterfall.

> To wipe out the build occassionally you could (presumably) add a
> starting step to the Python 'builder' (in the build master.cfg) to 
>     rm -rf $builddir
> every, say, Sunday night.

Sure, that would be the idea. How to formulate it?

Regards,
Martin

From martin at v.loewis.de  Sat Jan  7 00:01:32 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jan 2006 00:01:32 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <200601061916.38944.anthony@interlink.com.au>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
	<200601061916.38944.anthony@interlink.com.au>
Message-ID: <43BEF6CC.1020603@v.loewis.de>

Anthony Baxter wrote:
> At least with the way Twisted is set up, the buildbot also sits in an 
> IRC channel and sends updates there. It can also be controlled from 
> there. Is this worth doing? A 'force clean build' command could be 
> added...

The problem with irc-enabling (or web-enabling) such things is that
there is a potential for abuse. Of course, in this case, we could wait
for the abuse to happen.

Regards,
Martin

From martin at v.loewis.de  Sat Jan  7 00:03:46 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jan 2006 00:03:46 +0100
Subject: [Python-Dev] Automated Python testing
In-Reply-To: <2mmzi9zj1w.fsf@starship.python.net>
References: <20051228194543.1217.425460361.divmod.quotient.8597@ohm>	<43B308E0.20205@v.loewis.de>
	<2mmzi9zj1w.fsf@starship.python.net>
Message-ID: <43BEF752.8010506@v.loewis.de>

Michael Hudson wrote:
> #python-dev on freenode is ready and waiting should you decide to
> activate this :)

Ok, I added "him"; "his" nick is py-bb. Commands include "hello",
"status", "version". "force" is disabled.

Regards,
Martin

From trentm at ActiveState.com  Sat Jan  7 01:12:21 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 6 Jan 2006 16:12:21 -0800
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BEE8F4.5080304@v.loewis.de>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
	<20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
Message-ID: <20060107001221.GA13845@activestate.com>

> > To wipe out the build occassionally you could (presumably) add a
> > starting step to the Python 'builder' (in the build master.cfg) to 
> >     rm -rf $builddir
> > every, say, Sunday night.
> 
> Sure, that would be the idea. How to formulate it?

I think I'm part of the way there with the following. I've subclassed
the "SVN" source build step to add support for new source mode:
"update_and_clobber_occassionally". Basically it (hackily) changes the
source type btwn "update", which we usually want, and "clobber", which
we sometimes want.

The current problem with this is that I don't know if the build step
objects have access to a local data store -- so it could, say, count how
many builds have been done to know to clobber every tenth one. The
current code just chooses to clobber for any build on Sunday. Not a very
satisfying solution.

Brian,
Is there a way for build steps to maintain some state?


    ------------ clipped from the build master's master.cfg -----------------
    ...
    class SVNEx(step.SVN):
        """A SVN Source build step that adds the ability for the Source
        "mode" to be a combination of "update" and "clobber". This is useful if
        your general just want to update the source tree from source code control,
        but occassionally want to clobber and start fresh.

        To use this functionality use mode="update_and_clobber_occassionally".
        To control when "occassionally" is now override the "should_clobber"
        method. The default implementation is currently hardcoded to
        "every Sunday". (This is a HACK. Perhaps there should be constructor
        options to clobber every Nth time or to have cron-like arguments -- see
        "Nightly" in scheduler.py. I don't know if "steps" have access to a local
        data store to be able to do this -- e.g. keep a counter.)

        Ideally this would be an option of the base "Source" class in
        buildbot/process/step.py.
        """
        def __init__(self, **kwargs):
            if kwargs.has_key("mode") \
               and kwargs["mode"] == "update_and_clobber_occassionally":
                self.update_and_clobber_occassionally = True
                kwargs["mode"] = "update"
            else:
                self.update_and_clobber_occassionally = False
            step.SVN.__init__(self, **kwargs)

        def should_clobber(self):
            from datetime import date
            is_sunday = date.today().weekday() == 6 # it is Sunday
            from twisted.python import log
            log.msg("should_clobber? %s", (is_sunday and "yes" or "no"))
            return is_sunday

        def startVC(self, *args, **kwargs):
            if self.update_and_clobber_occassionally:
                if self.should_clobber():
                    self.args["mode"] = "clobber"
                else:
                    self.args["mode"] = "update"
            step.SVN.startVC(self, *args, **kwargs)


    python_factory = factory.GNUAutoconf(
        s(SVNEx, svnurl="http://svn.python.org/projects/python/trunk",
          mode="update_and_clobber_occassionally"),
        test=["make", "test"],  # use `make testall`?
    )

    # Then use python_factory as something like so:
    #c['builders'] = [
    #    {'name': "linux-x86",
    #          'slavename': "...",
    #          'builddir': "python",
    #          'factory': python_factory,
    #          },
    #    ...
    #]
    ------------ clipped from the build master's master.cfg -----------------



Cheers,
Trent

-- 
Trent Mick
trentm at activestate.com

From martin at v.loewis.de  Sat Jan  7 01:13:53 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 07 Jan 2006 01:13:53 +0100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <20060107001221.GA13845@activestate.com>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
	<20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
Message-ID: <43BF07C1.3060608@v.loewis.de>

Trent Mick wrote:
> I think I'm part of the way there with the following. I've subclassed
> the "SVN" source build step to add support for new source mode:
> "update_and_clobber_occassionally". Basically it (hackily) changes the
> source type btwn "update", which we usually want, and "clobber", which
> we sometimes want.

Ah, but that would require changes to the slaves, right? I would
prefer a solution that avoids that.

Regards,
Martin

From trentm at ActiveState.com  Sat Jan  7 01:16:51 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Fri, 6 Jan 2006 16:16:51 -0800
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BF07C1.3060608@v.loewis.de>
References: <43BB0965.5080301@masklinn.net>
	<1f7befae0601050743p32e542f4gb899153b2d3ae94c@mail.gmail.com>
	<Pine.LNX.4.58.0601051915060.6240@alice>
	<1f7befae0601051910g189d9689te2e6f9532cc3d5b3@mail.gmail.com>
	<43BE1EA3.8040405@v.loewis.de>
	<20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<43BF07C1.3060608@v.loewis.de>
Message-ID: <20060107001651.GB13845@activestate.com>

[Martin v. Loewis wrote]
> Trent Mick wrote:
> > I think I'm part of the way there with the following. I've subclassed
> > the "SVN" source build step to add support for new source mode:
> > "update_and_clobber_occassionally". Basically it (hackily) changes the
> > source type btwn "update", which we usually want, and "clobber", which
> > we sometimes want.
> 
> Ah, but that would require changes to the slaves, right? I would
> prefer a solution that avoids that.

I don't think so. In my little test setup I didn't have to make any
change to the slave.

Trent

-- 
Trent Mick
trentm at activestate.com

From murman at gmail.com  Sat Jan  7 04:11:34 2006
From: murman at gmail.com (Michael Urman)
Date: Fri, 6 Jan 2006 21:11:34 -0600
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43BD8464.4060801@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
	<43B50B64.20904@v.loewis.de>
	<20060105194538.GB11736@code1.codespeak.net>
	<43BD8464.4060801@v.loewis.de>
Message-ID: <dcbbbb410601061911t4395a487hfc9541bbaba824fb@mail.gmail.com>

[I just noticed that I sent this mail to just Martin when I meant it
for the list. Sorry Martin!]

On 1/5/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> More precisely, the printf style of function calling, and varargs
> functions. ISO C is pretty type safe, but with varargs functions,
> you lose that completely.
>
> I still hope I can write a C parser some day that does
> ParseTuple/BuildValue checking.

I put together a non-parsing checker last month to help me feel more
secure after http://python.org/sf/1365916. It's awful code, but the
simple things are easy to change or extend. Fixing the false positives
and other misinterpretations is harder.

http://www.tortall.net/mu/static/fmtcheck.py?raw - it takes a list of
directories to os.walk for c source files.

Michael
--
Michael Urman  http://www.tortall.net/mu/blog

From nnorwitz at gmail.com  Sat Jan  7 04:18:37 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Fri, 6 Jan 2006 19:18:37 -0800
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <dcbbbb410601061911t4395a487hfc9541bbaba824fb@mail.gmail.com>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
	<43B50B64.20904@v.loewis.de>
	<20060105194538.GB11736@code1.codespeak.net>
	<43BD8464.4060801@v.loewis.de>
	<dcbbbb410601061911t4395a487hfc9541bbaba824fb@mail.gmail.com>
Message-ID: <ee2a432c0601061918j462814a4qf8c995febdeb5622@mail.gmail.com>

On 1/6/06, Michael Urman <murman at gmail.com> wrote:
>
> I put together a non-parsing checker last month to help me feel more
> secure after http://python.org/sf/1365916. It's awful code, but the
> simple things are easy to change or extend. Fixing the false positives
> and other misinterpretations is harder.

Perhaps you could use Synopsis to parse the C code?

  http://synopsis.fresco.org/

I'm hoping to look into this more.  IIRC, it generates an AST from the
C source which you can iterate over to find problems.  It's got Python
APIs.

I was hoping it could provide the basis for a static analysis tool to
help find local memory (ref) leaks among other things.

n

From anthony at interlink.com.au  Sat Jan  7 04:48:56 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat, 7 Jan 2006 14:48:56 +1100
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <43BEF6CC.1020603@v.loewis.de>
References: <43BB0965.5080301@masklinn.net>
	<200601061916.38944.anthony@interlink.com.au>
	<43BEF6CC.1020603@v.loewis.de>
Message-ID: <200601071449.01642.anthony@interlink.com.au>

On Saturday 07 January 2006 10:01, Martin v. L?wis wrote:
> The problem with irc-enabling (or web-enabling) such things is that
> there is a potential for abuse. Of course, in this case, we could
> wait for the abuse to happen.

That would be my vote. Worst comes to worst, we lock it down to a list 
of known users. 

Anthony

-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From kay.schluehr at gmx.net  Sat Jan  7 08:57:48 2006
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sat, 07 Jan 2006 08:57:48 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
Message-ID: <dpns9t$gil$1@sea.gmane.org>

Guido van Rossum wrote:

>>Yuk!  This has been discussed again and again already.  *Please* move
>>this discussion to comp.lang.python.
> 
> 
> Yes please. This won't change.

Then simply reject the PEP and the discussion can be stopped on 
comp.lang.python too. Or why do you think it should be discussed there 
again and again or elsewhere on the web ( e.g. in Bruce Eckels blog on 
Artima )?

Regards


From fredrik at pythonware.com  Sat Jan  7 18:20:14 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 7 Jan 2006 18:20:14 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net><dp2mmh$45b$1@sea.gmane.org>
	<43B56EEE.9090904@egenix.com>
Message-ID: <dpot8g$8a9$1@sea.gmane.org>

M.-A. Lemburg wrote:

> I haven't followed the thread, so many I'm repeating things.
>
> Has anyone considered using e.g. MediaWiki (the wiki used for
> Wikipedia) for Python documentation ?
>
> I'm asking because this wiki has proven to be ideally suited
> for creating complex documentation tasks and offers many features
> which would make Python documentation a lot easier and more
> accessible:
>
> * people wouldn't have to learn LaTeX to commit doc-patches
> * it's easy to monitor and revert changes, discuss changes
> * there's version history available
> * new docs would be instantly available on the web
> * builtin search facility, categories and all the other nifty
>   wiki stuff
> * it's one of the more popular wikis around and due to Wikipedia
>   it's here to stay
> * conversion to XML and DocBook is possible, providing
>   entry points for conversion to other formats (including
>   LaTeX)
> * more following means more tools (over time)
>
> Just a thought.

anyone knows anything about support for semantic markup ?

(afaict, the current moinmoin wiki would solve 80% of the issues involved
here; I'm aiming for those 80% plus 80% of the remaining 20%).

</F>




From facundobatista at gmail.com  Sat Jan  7 19:17:15 2006
From: facundobatista at gmail.com (Facundo Batista)
Date: Sat, 7 Jan 2006 15:17:15 -0300
Subject: [Python-Dev] ConfigParser to save with order
Message-ID: <e04bdf310601071017u72ae5b75h@mail.gmail.com>

ConfigParser saves the data in a not-predefined order. This is because
it keeps, internally, the information in dictionaries.

I opened a patch in SF 1399309 that orders the info to be saved in the
file through the ConfigParser write() method.

This patch does not let the user to specify the order, but allows to
predict the order in which it'll be saved.

There's another patch,  the 1371075, that allows to specify the order
through an "ordered dict".

What do we want ConfigParser to do?

Regards,

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From nnorwitz at gmail.com  Sat Jan  7 21:45:27 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 7 Jan 2006 12:45:27 -0800
Subject: [Python-Dev] Checking in a broken test was: Re: [Python-checkins]
	r41940 - python/trunk/Lib/test/test_compiler.py
Message-ID: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>

[moving to python-dev]

> On 1/7/06, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
> > Well, it is not the test that's broken... it's compiler.

[In reference to:
http://mail.python.org/pipermail/python-checkins/2006-January/048715.html]

In the past, we haven't checked in tests which are known to be broken.
 There are several good reasons for this.  I would prefer you, 1) also
fix the code so the test doesn't fail, 2) revert the change (there's
still a bug report open, right?), or 3) generalize tests for known
bugs.

I strongly prefer #1, but have been thinking about adding #3.  There
are many open bug reports that fall into two broad categories: 
incorrect behaviour and crashers.  I've been thinking about adding two
tests which incorporate these bugs as a way of consolidating where the
known problems are.  Also, it's great when we have test cases that can
be moved to the proper place once the fix has been checked in.

I'm proposing something like add two files to Lib/test:
outstanding_bugs.py and outstanding_crashes.py.  Both would be normal
test files with info about the bug report and the code that causes
problems.

This test in test_compiler should be moved to outstanding_bugs.py.

And for a different discussion:

On 1/7/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> Maybe.  Guido's statement (maybe short of a pronouncement)
> was that keyword-only arguments were OK in principle, and
> that *args could follow keywords.  It wasn't true yet because
> no one had put in the work, but it would be an acceptable
> change.
>
> I interpret this to mean that
>
>     def f(a=1, b): pass
>
> should not necessarily raise an error, but I would like to see what
> it does to
>
>     def f(a=1, b):
>         print a,b
>     f(b=7)
>
> before saying that it is OK.

From pedronis at strakt.com  Sat Jan  7 22:26:01 2006
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sat, 07 Jan 2006 22:26:01 +0100
Subject: [Python-Dev] some interesting readings
Message-ID: <43C031E9.6070704@strakt.com>

because I was reminded of them recently, because they may be useful 
landmarks in the prospective of future discussions, because expanding
one's understanding of the problem/solution space of language design
is quite a good thing if one is interested in such things...

1)
Gilad Bracha.  Pluggable Type Systems . OOPSLA04 Workshop on Revival of 
Dynamic  Languages  ( 
http://pico.vub.ac.be/%7Ewdmeuter/RDL04/papers/Bracha.pdf )

As a talk:
Pluggable Types, originally given at Aarhus University in March 2003, 
and repeated since at Berne and elsewhere.
( http://bracha.org/pluggable-types.pdf )

2)
http://homepages.cwi.nl/~ralf/OOHaskell/
state of the art experiment on trying to reconcile object orientation, 
type inference and as much as possible expressiveness

PS: I think 1 is much more relevant than 2 for Python as we know it.

happy reading.






From tim.peters at gmail.com  Sun Jan  8 00:24:06 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 7 Jan 2006 18:24:06 -0500
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
Message-ID: <1f7befae0601071524x55fc0652pbfebc5212d1bc6f9@mail.gmail.com>

[Neal Norwitz]
> ...
> In the past, we haven't checked in tests which are known to be broken.

It's an absolute rule that you never check in a change (whether a test
or anything else) that causes ``regretst.py -uall`` to fail.  Even if
it passes on your development box, but fails on someone else's box,
it's fair game to revert that change.

Since this particular change caused -uall to fail on all boxes, I had
no qualms about reverting rev 41940.

Reinhold, if you're working on a fix for the problem too, feel free to
check that test in on a branch instead (short-lived branches are easy
& fast under SVN).  Branches aren't required to pass any tests.  The
test on the branch can be merged into the trunk later, after a fix has
also been checked in on that branch.

...

> There are many open bug reports that fall into two broad categories:
> incorrect behaviour and crashers.  I've been thinking about adding two
> tests which incorporate these bugs as a way of consolidating where the
> known problems are.  Also, it's great when we have test cases that can
> be moved to the proper place once the fix has been checked in.
>
> I'm proposing something like add two files to Lib/test:
> outstanding_bugs.py and outstanding_crashes.py.  Both would be normal
> test files with info about the bug report and the code that causes
> problems.
>
> This test in test_compiler should be moved to outstanding_bugs.py.

That would be fine provided that the failing tests aren't normally run
via -uall.

From guido at python.org  Sun Jan  8 01:43:49 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Jan 2006 16:43:49 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
Message-ID: <ca471dc20601071643n2d243706s64ad3fc5fc1519c4@mail.gmail.com>

On 1/7/06, Neal Norwitz <nnorwitz at gmail.com> wrote:
> I'm proposing something like add two files to Lib/test:
> outstanding_bugs.py and outstanding_crashes.py.  Both would be normal
> test files with info about the bug report and the code that causes
> problems.

I like this approach. regrtest.py won't runthem, but you can run them
manually to help in finding the bug (if the cause isn't known yet) or
verifying it's been fixed (if it is).

> And for a different discussion:
>
> On 1/7/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> > Maybe.  Guido's statement (maybe short of a pronouncement)
> > was that keyword-only arguments were OK in principle, and
> > that *args could follow keywords.  It wasn't true yet because
> > no one had put in the work, but it would be an acceptable
> > change.
> >
> > I interpret this to mean that
> >
> >     def f(a=1, b): pass
> >
> > should not necessarily raise an error, but I would like to see what
> > it does to
> >
> >     def f(a=1, b):
> >         print a,b
> >     f(b=7)
> >
> > before saying that it is OK.

I suspect you've got that wrong. I don't recall saying that you should
be able to put positional arguments after keyword arguments in a call
-- the meaning would be ambiguous to humans even if it can be given a
clear meaning by the compiler. Similarly, in a def it would also be
unclear to the human reader, and I'm not sure what the point would be.
(In the above example, the real question is what f(7) would mean --
does it set b to 7 or a?)

However, I *do* recall saying something about the relative ordering of
*args and **kwds. In particular, I think we should be allowed to write

 f(1, 2, 3, *args, x=1, y=2, **kwds)   # also without **kwds

and perhaps also

  f(1, 2, 3, *args, 4, 5, 6)

(though I'm less interested in supporting the latter).

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From nnorwitz at gmail.com  Sun Jan  8 01:58:30 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sat, 7 Jan 2006 16:58:30 -0800
Subject: [Python-Dev] Compiler warnings for 64-bit portability problems
In-Reply-To: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de>
References: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de>
Message-ID: <ee2a432c0601071658x1bdf3aeo7b143fd59ae34dfc@mail.gmail.com>

On 1/6/06, von L?wis Martin <Martin.vonLoewis at hpi.uni-potsdam.de> wrote:
> I just found that the intel compiler (icc 9.0)
> also supports compiler warnings for portability
> problems.

Cool.  Thanks for the info.  It would be nice if Intel would provide
Python developers with a permanent icc license for Python.  Can anyone
help with that?

> 4. don't try the Intel install.sh, it won't work
>    (atleast, it didn't work for me)

I don't follow directions very well (I'm also a doubter), so I had to
try it myself. :-)
It worked for me on gentoo amd64 non-root install.

I needed to set LD_LIBRARY_PATH to ~/intel/cce/9.0/lib, but otherwise
it was pretty painless.

This worked:  CC=icc ./configure --without-cxx
But I think this should work too: CC=icc CXX=icc ./configure

I've fixed a bunch of problems on HEAD.  I think I will move to ssize
branch since that will be a much bigger help.  I have some outstanding
modifications, but here are the results of make: 
http://python.org/neal/icc-warnings.txt

Summary (type, warning count, description):

warning.#170:    2 pointer points outside of underlying object
(obmalloc.c PT(0))
warning.#188:   14 enumerated type mixed with another type
warning.#810: 1478 conversion from "long" to "int" may lose significant bits
warning.#1418: 137 external definition with no prior declaration
warning.#1419:  53 external declaration in primary source file
warning.#1684: 285 conversion from pointer to same-sized integral type
(potential portability problem) -- all due to offsetof()

remark.#111:  121 statement is unreachable
remark.#177:    1 func "marshal_write_mod" was declared but never referenced
remark.#181:    9 arg is incompatible w/corresponding format string conversion
remark.#279:  862 controlling expression is constant
remark.#593:   17 variable set, but not used
remark.#869: 1317 parameter not used
remark.#981:  130 operands are evaluated in unspecified order
remark.#1469:  10 "cc" clobber ignored (wtf? use of htons() causes this msg)
remark.#1572: 111 floating-point equality/inequality comparisons are unreliable
remark.#1599:  30 declaration hides parameter

Note there are a lot of warning even when we mask off the higher bits
(e.g., x = y & 0xff still generats a warning if x is a char).

n

From guido at python.org  Sun Jan  8 02:04:39 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Jan 2006 17:04:39 -0800
Subject: [Python-Dev] Compiler warnings for 64-bit portability problems
In-Reply-To: <ee2a432c0601071658x1bdf3aeo7b143fd59ae34dfc@mail.gmail.com>
References: <5AC0F51A926CD442BC08D55DA7DAB1D40129F4AB@3mxma1r.hpi.uni-potsdam.de>
	<ee2a432c0601071658x1bdf3aeo7b143fd59ae34dfc@mail.gmail.com>
Message-ID: <ca471dc20601071704t57df3984h66b4849da04ffc27@mail.gmail.com>

On 1/7/06, Neal Norwitz <nnorwitz at gmail.com> wrote:
> Cool.  Thanks for the info.  It would be nice if Intel would provide
> Python developers with a permanent icc license for Python.  Can anyone
> help with that?

I'll try. A dutch friend from long ago (CWI) is now working for
Intel's compiler group and has tried to interest me in providing
feedback about their compiler (and also their alternative to
valgrind). I'll contact him.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Jan  8 02:10:16 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Jan 2006 17:10:16 -0800
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
Message-ID: <ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>

I think it's moot unless you also preserve comments. Ideally would be
something that prserved everything (ordering, blank lines, comments
etc.) from how it was read in. Modifying a value should keep its
position. Adding a value should add it to the end of the section it's
in (unless there are cases where ordering is important to the
semantics -- are there?).

--Guido

On 1/7/06, Facundo Batista <facundobatista at gmail.com> wrote:
> ConfigParser saves the data in a not-predefined order. This is because
> it keeps, internally, the information in dictionaries.
>
> I opened a patch in SF 1399309 that orders the info to be saved in the
> file through the ConfigParser write() method.
>
> This patch does not let the user to specify the order, but allows to
> predict the order in which it'll be saved.
>
> There's another patch,  the 1371075, that allows to specify the order
> through an "ordered dict".
>
> What do we want ConfigParser to do?
>
> Regards,
>
> .    Facundo
>
> Blog: http://www.taniquetil.com.ar/plog/
> PyAr: http://www.python.org/ar/
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Jan  8 02:12:06 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 7 Jan 2006 17:12:06 -0800
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <dpns9t$gil$1@sea.gmane.org>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
Message-ID: <ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>

On 1/6/06, Kay Schluehr <kay.schluehr at gmx.net> wrote:
> Then simply reject the PEP and the discussion can be stopped on
> comp.lang.python too.

Only in the most severe cases does it make sense to create a PEP
specifically to be rejected.

> Or why do you think it should be discussed there
> again and again or elsewhere on the web ( e.g. in Bruce Eckels blog on
> Artima )?

Perhaps because people don't understand it?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ncoghlan at gmail.com  Sun Jan  8 05:01:20 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 08 Jan 2006 14:01:20 +1000
Subject: [Python-Dev] some interesting readings
In-Reply-To: <43C031E9.6070704@strakt.com>
References: <43C031E9.6070704@strakt.com>
Message-ID: <43C08E90.1070308@gmail.com>

Samuele Pedroni wrote:
> because I was reminded of them recently, because they may be useful 
> landmarks in the prospective of future discussions, because expanding
> one's understanding of the problem/solution space of language design
> is quite a good thing if one is interested in such things...
> 
> 1)
> Gilad Bracha.  Pluggable Type Systems . OOPSLA04 Workshop on Revival of 
> Dynamic  Languages  ( 
> http://pico.vub.ac.be/%7Ewdmeuter/RDL04/papers/Bracha.pdf )
> 
> As a talk:
> Pluggable Types, originally given at Aarhus University in March 2003, 
> and repeated since at Berne and elsewhere.
> ( http://bracha.org/pluggable-types.pdf )
> 
> 2)
> http://homepages.cwi.nl/~ralf/OOHaskell/
> state of the art experiment on trying to reconcile object orientation, 
> type inference and as much as possible expressiveness
> 
> PS: I think 1 is much more relevant than 2 for Python as we know it.

I'd have to agree with that - I didn't actually make it all the way through 
the second one, but an awful of lot of what I did read seemed to taken up with 
clever workarounds designed to trick the Haskell type inferencer into letting 
the authors of the paper do some fairly basic things (like having a 
heterogeneous collection of subtypes).

There are some fascinating ideas in the first paper, though. It actually had 
me wondering about the possibilities of PyPy's object spaces, but I don't 
really know enough about those to determine whether or not such a connection 
is actually meaningful.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From pje at telecommunity.com  Sun Jan  8 05:27:14 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 07 Jan 2006 23:27:14 -0500
Subject: [Python-Dev] some interesting readings
In-Reply-To: <43C08E90.1070308@gmail.com>
References: <43C031E9.6070704@strakt.com>
 <43C031E9.6070704@strakt.com>
Message-ID: <5.1.1.6.0.20060107232224.02266cb8@mail.telecommunity.com>

At 02:01 PM 1/8/2006 +1000, Nick Coghlan wrote:
>Samuele Pedroni wrote:
> > 2)
> > http://homepages.cwi.nl/~ralf/OOHaskell/
> > state of the art experiment on trying to reconcile object orientation,
> > type inference and as much as possible expressiveness
> >
> > PS: I think 1 is much more relevant than 2 for Python as we know it.
>
>I'd have to agree with that - I didn't actually make it all the way through
>the second one, but an awful of lot of what I did read seemed to taken up 
>with
>clever workarounds designed to trick the Haskell type inferencer into letting
>the authors of the paper do some fairly basic things (like having a
>heterogeneous collection of subtypes).

Yeah, I thought the second one was a really strong argument for *not* 
trying to transpose "traditional" OO code directly into Haskell, but rather 
focusing on polymorphic functions instead.  In Python terms, functions like 
len() and copy() rather than focusing on building methods like __len__() 
and __copy__().  Since Haskell allows you to provide multiple definitions 
for a function based on matching types or values, there's really no point 
to trying to make actual methods.  It seemed to me a lot like the whole 
"implicit self" argument; i.e., imposing a foreign paradigm on the language 
for familiarity's sake.


From kay.schluehr at gmx.net  Sun Jan  8 10:56:34 2006
From: kay.schluehr at gmx.net (Kay Schluehr)
Date: Sun, 08 Jan 2006 10:56:34 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
Message-ID: <dpqnkj$qph$1@sea.gmane.org>

Guido van Rossum wrote:
> On 1/6/06, Kay Schluehr <kay.schluehr at gmx.net> wrote:
> 
>>Then simply reject the PEP and the discussion can be stopped on
>>comp.lang.python too.
> 
> 
> Only in the most severe cases does it make sense to create a PEP
> specifically to be rejected.
>>Or why do you think it should be discussed there
>>again and again or elsewhere on the web ( e.g. in Bruce Eckels blog on
>>Artima )?
> 
> 
> Perhaps because people don't understand it?

They don't accept it even more than they don't understand it. This
leads to repeated guesses that the object model might perhaps change in 
Py3K with respect to explicit self [1]. It's not all just an educational 
issue. If this is not a severe case where an autoritarian decision 
should be made and also be made visible at a central and recoverable 
place I don't know what?

Regards
Kay

[1] See e.g. Bruce Eckels postings on this page:

http://www.artima.com/forums/flat.jsp?forum=106&thread=141312&start=75&msRange=15


From g.brandl-nospam at gmx.net  Sun Jan  8 11:21:03 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sun, 08 Jan 2006 11:21:03 +0100
Subject: [Python-Dev] Birkenfeld's gone
Message-ID: <dpqp2g$u8f$1@sea.gmane.org>

Hello,

today, when two Python developers here had approached me about the PSF, I
realized that it is time to correct a mistake which I had made over three years
ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time).

I then assumed a different name, partly to anonymize myself as others had advised.
When I came across Python, I instantly liked it, and since I had interest in
Open Source development in general, I wanted to try and contribute to the development.
And, as a matter of course, I used my different name for that. When I realized that
I liked the community and the development of Python very much, I decided to
"unveil" myself, but I could never overcome myself -- till now.

I'm really sorry, and I hope you could at least partly understand what caused
me to act so irrational.

regards,
Georg

PS: I'm quite prepared now for the PSU to visit my place and interrog


From martin at v.loewis.de  Sun Jan  8 12:03:55 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 12:03:55 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43BEA574.8010607@colorstudy.com>
References: <611494241.20060106005601@gmail.com> <43BDC001.3070602@gmail.com>
	<43BEA574.8010607@colorstudy.com>
Message-ID: <43C0F19B.2000005@v.loewis.de>

Ian Bicking wrote:
> would have to be translated to this this:
> 
>    inst = Foo()
>    f = Foo.bar
>    meth = bind(f, inst)
>    print meth(1, 2)

+1 for an explicit "bind unbound method operation", although I
would spell it as

    inst = Foo()
    f = Foo.bar
    meth = f.bind(inst)
    print meth(1, 2)

Regards,
Martin

From munna_tank at yahoo.co.in  Sun Jan  8 13:00:24 2006
From: munna_tank at yahoo.co.in (RASHMI TANK)
Date: Sun, 8 Jan 2006 12:00:24 +0000 (GMT)
Subject: [Python-Dev] Python-Dev Digest, Vol 29, Issue 111
In-Reply-To: <mailman.13049.1135875794.18700.python-dev@python.org>
Message-ID: <20060108120024.77871.qmail@web8703.mail.in.yahoo.com>


  Respected sir,
   
  iam a un knowledged and un experienced person.
  i have an email to you but no reply. How you find me?
  i had joined icash business through icashline.com 
  but i failed. you always giving emails to me where as idoesnt have any 
  know ledge of web and free hosting and linking web pages and ee business
  and mass mailing softwere.
  please reply first and then give other supports.
  thanking you.
   
  please stop mails from you . i will inform to you then you send.
  ok. due to i am unable to view my computor for some period. may be3 months from now.
  thanking you.


Send instant messages to your online friends http://in.messenger.yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060108/ee0fecf6/attachment.html 

From thomas at xs4all.net  Sun Jan  8 13:25:28 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 8 Jan 2006 13:25:28 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
Message-ID: <20060108122527.GH18916@xs4all.nl>

On Sat, Jan 07, 2006 at 05:12:06PM -0800, Guido van Rossum wrote:
> On 1/6/06, Kay Schluehr <kay.schluehr at gmx.net> wrote:
> > Then simply reject the PEP and the discussion can be stopped on
> > comp.lang.python too.

> Only in the most severe cases does it make sense to create a PEP
> specifically to be rejected.

Yet if it isn't recorded, people will keep bringing it up. How about a
'rejected ideas' PEP for ideas that are right out no matter how people
argue? A single PEP, with oneliners to describe ideas, one or two lines to
explain 'why not', references to the python-list or python-dev discussions,
if any, and a sign in big friendly letters saying "if you really must
discuss these subjects, do it on python-list, not python-dev". Some of the
stuff that could be in there:

 - implicit rather than explicit self: invalidates too many tricks
 - omitting () on functioncalls: like implicit self
 - changing all statements into expressions (e.g. logix): python isn't
   (going to be) a functional language
 - methods for tuples: tuples are records, not collections; use lists instead
 - sigils to indicate 'self.' (e.g. @foo): loses readability, wins too little
 - '?' and other non-alphanumerical characters in identifiers: like sigils
 - strict private/public accessing: 'fake' protection; use closures instead

etc. No need to come up with them all, I'm sure all the prize ideas will pop
back up eventually ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From martin at v.loewis.de  Sun Jan  8 13:35:16 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 13:35:16 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108122527.GH18916@xs4all.nl>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl>
Message-ID: <43C10704.80404@v.loewis.de>

Thomas Wouters wrote:
> Yet if it isn't recorded, people will keep bringing it up. How about a
> 'rejected ideas' PEP for ideas that are right out no matter how people
> argue?

Recorded it is, in the mailing list archive.

However, a central place might be better, preferably with referrals to
pronouncements or other discussion. A Wiki comes to mind...

Regards,
Martin

From fredrik at pythonware.com  Sun Jan  8 15:09:14 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 8 Jan 2006 15:09:14 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org><4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>
	<43B4A238.2030101@python.org><dp2qt7$dib$1@sea.gmane.org>
	<1135943776.8971.77.camel@warna.dub.corp.google.com>
Message-ID: <dpr6eb$v1$1@sea.gmane.org>

Donovan Baarda wrote:

> > No, it's a fundamental goal: to support light-weight generation of rendered
> > markup directly from source code, to enable simple tools (CGI scripts, etc)
> > to be able to render reference documentation.
>
> Python is run-time interpreted, but I don't think we need its
> documentation to be.

Are you sure ?

>>> help(help)

>From the pythondoc/lib goal list:

    Support tighter (and 2-way) integration between the library reference
    and help tools (pydoc, help, inspect, etc) and documentation generators
    (epydoc, pythondoc, etc).

> "trivial" is a relative term.

When I say "trivial" in this context, I mean that any programmer should
be capable of writing a Python tool that does some kind of basic processing
of the source format from scratch.  The easier it is to do this, the more
likely it is that someone will come up with a new way to use the information
("Web 2.0", you know...)

> As a some-times developer not really interested in writing
> documentation, I must admit I really like ReST. It looks like plain
> text, so I can write and use it as plain text, but it also can magically
> transform to other formats and look pretty good.

Lots of "looks like plain text if you squint" formats can produce output
that looks pretty good.

The problem is semantics; I haven't seen any format that looks like plain
text but provides the amount of machine-readable semantic information
that we have today.  And I want more semantics, not less.

> I hate writing XML/HTML, and my only experience with LaTex was brief and
> painful too. If I was forced to choose, I'd probably pick LaTex even
> though I don't know it as well as XML/HTML (I'd rather face an unknown
> evil than that particular devil I know).

Familiarity breeds contempt ?  We're not talking about well-formed XML
here, nor 1996-era hand-rolled web sites.  If you look at typical Java-
Doc or PythonDoc markup, or the translated samples, you'll notice that
the source code contains very few HTML tags.

(I seriously doubt that your problems with XML/HTML is the use of < in-
stead of {...)

> I'm not going to write docs for stuff I didn't write myself. I'm not
> going to submit patches to Javadoc, LaTex or XML/HTML, but I might post
> vague "change this to..." bug reports.

> Writing good documentation is hard (which is why I prefer to leave it to
> others if I can), but fixing almost good documentation doesn't have to
> be /.../

Since I'm focussed on the library reference, I'm a lot more interested in
the fixers than the essay writers.  Give people a way to find things to fix,
and a way to fix it, and they will.

> > It's the workflow that's the real problem here, but you cannot fix the workflow
> > without fixing the markup.
>
> I must say I disagree with this statement... changing the markup will
> not change the workflow at all, just shave 3-6 mins off the compile-test
> step. I don't do this enough to really know if that's worth it.

Do you think wikipedia would be what it is today if the edit/save process
included a 3-6 minute "regenerate site" pass ?

Heck, I've discovered dozens of bogosities in the current library reference
while working on the translation tools, and I have the sources and full SVN
access, but I find that the overhead discourages me from doing anything
about them ("maybe later").  I simply don't have time to fix anything but
very minor typos within a standard 15-minute "let's improve python" time
slot, and I sure don't think I'm unique in this respect.

</F>




From rwgk at yahoo.com  Sun Jan  8 15:28:57 2006
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Sun, 8 Jan 2006 06:28:57 -0800 (PST)
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <611494241.20060106005601@gmail.com>
Message-ID: <20060108142857.89587.qmail@web31506.mail.mud.yahoo.com>

--- Alexander Kozlovsky <alexander.kozlovsky at gmail.com> wrote:
> What do you think about this?

I (who writes Python code for a living) love it! See also:
http://cci.lbl.gov/~rwgk/python/adopt_init_args_2005_07_02.html

***Please*** make Python more selfish. Note that this is also an obvious avenue
for significant performance increases. If self is implicit you don't have to do
the dictionary lookup for "self" all the time as is the case now.

Good luck with your battle!

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From fredrik at pythonware.com  Sun Jan  8 15:25:01 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 8 Jan 2006 15:25:01 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
References: <611494241.20060106005601@gmail.com><20060106193448.GA656@code1.codespeak.net><ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com><dpns9t$gil$1@sea.gmane.org><ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl>
Message-ID: <dpr7bu$3in$1@sea.gmane.org>

Thomas Wouters wrote:

> > Only in the most severe cases does it make sense to create a PEP
> > specifically to be rejected.
>
> Yet if it isn't recorded, people will keep bringing it up. How about a
> 'rejected ideas' PEP for ideas that are right out no matter how people
> argue? A single PEP, with oneliners to describe ideas, one or two lines to
> explain 'why not', references to the python-list or python-dev discussions,
> if any, and a sign in big friendly letters saying "if you really must
> discuss these subjects, do it on python-list, not python-dev". Some of the
> stuff that could be in there:
>
>  - implicit rather than explicit self: invalidates too many tricks
>  - omitting () on functioncalls: like implicit self
>  - changing all statements into expressions (e.g. logix): python isn't
>    (going to be) a functional language
>  - methods for tuples: tuples are records, not collections; use lists instead
>  - sigils to indicate 'self.' (e.g. @foo): loses readability, wins too little
>  - '?' and other non-alphanumerical characters in identifiers: like sigils
>  - strict private/public accessing: 'fake' protection; use closures instead
>
> etc. No need to come up with them all, I'm sure all the prize ideas will pop
> back up eventually ;)

the FAQ touches many of these.  maybe a couple of clarifications to the
relevant FAQ texts (explaining things in terms of design tradeoffs, rather
than absolute truths) would be good enough ?

</F>




From g.brandl-nospam at gmx.net  Sun Jan  8 15:36:42 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sun, 08 Jan 2006 15:36:42 +0100
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins] r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
Message-ID: <dpr81r$5ie$1@sea.gmane.org>

Neal Norwitz wrote:
> [moving to python-dev]
> 
>> On 1/7/06, Reinhold Birkenfeld <reinhold-birkenfeld-nospam at wolke7.net> wrote:
>> > Well, it is not the test that's broken... it's compiler.
> 
> [In reference to:
> http://mail.python.org/pipermail/python-checkins/2006-January/048715.html]
> 
> In the past, we haven't checked in tests which are known to be broken.

Okay. I apologize. I originally intended to fix it, but I found that I don't have
the time to search the bug.

>  There are several good reasons for this.  I would prefer you, 1) also
> fix the code so the test doesn't fail, 2) revert the change (there's
> still a bug report open, right?), or 3) generalize tests for known
> bugs.
> 
> I strongly prefer #1, but have been thinking about adding #3.  There
> are many open bug reports that fall into two broad categories: 
> incorrect behaviour and crashers.  I've been thinking about adding two
> tests which incorporate these bugs as a way of consolidating where the
> known problems are.  Also, it's great when we have test cases that can
> be moved to the proper place once the fix has been checked in.
> 
> I'm proposing something like add two files to Lib/test:
> outstanding_bugs.py and outstanding_crashes.py.  Both would be normal
> test files with info about the bug report and the code that causes
> problems.
> 
> This test in test_compiler should be moved to outstanding_bugs.py.

I added outstanding_bugs.py and this initial test.

regards,
Georg


From fredrik at pythonware.com  Sun Jan  8 15:42:29 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 8 Jan 2006 15:42:29 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
References: <611494241.20060106005601@gmail.com>
	<20060108142857.89587.qmail@web31506.mail.mud.yahoo.com>
Message-ID: <dpr8cn$6h9$1@sea.gmane.org>

Ralf W. Grosse-Kunstleve wrote:

> ***Please*** make Python more selfish. Note that this is also an obvious avenue
> for significant performance increases. If self is implicit you don't have to do
> the dictionary lookup for "self" all the time as is the case now.

what dictionary lookup ?

</F>




From skip at pobox.com  Sun Jan  8 15:49:10 2006
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 8 Jan 2006 08:49:10 -0600
Subject: [Python-Dev] Birkenfeld's gone
In-Reply-To: <dpqp2g$u8f$1@sea.gmane.org>
References: <dpqp2g$u8f$1@sea.gmane.org>
Message-ID: <17345.9830.144828.412332@montanaro.dyndns.org>


    Georg> I then assumed a different name, partly to anonymize myself as
    Georg> others had advised....  When I realized that I liked the
    Georg> community and the development of Python very much, I decided to
    Georg> "unveil" myself, but I could never overcome myself -- till now.

Hmmm...  How do we  know Reinhold Birkenfeld and  Georg Brandl are  the same
person?  In fact, how do we know  GB didn't "overcome"  RB in some nefarious
way and that RB isn't swimming with the  fishes in the  Rhein or buried deep
in the Black  Forest?  Different first  name, different last name, different
ISP.  From here it appears  your hair color may have  changed as well.  Only
the "-nospam" in your email addresses is the same...

I'm sure the PSU will be able to

From fredrik at pythonware.com  Sun Jan  8 15:57:06 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 8 Jan 2006 15:57:06 +0100
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
Message-ID: <dpr983$95r$1@sea.gmane.org>

Neal Norwitz wrote:

> In the past, we haven't checked in tests which are known to be broken.
>  There are several good reasons for this.  I would prefer you, 1) also
> fix the code so the test doesn't fail, 2) revert the change (there's
> still a bug report open, right?), or 3) generalize tests for known
> bugs.
>
> I strongly prefer #1, but have been thinking about adding #3.  There
> are many open bug reports that fall into two broad categories:
> incorrect behaviour and crashers.  I've been thinking about adding two
> tests which incorporate these bugs as a way of consolidating where the
> known problems are.  Also, it's great when we have test cases that can
> be moved to the proper place once the fix has been checked in.

many test frameworks support "expected failures" for this purpose.
how hard would it be to add a

    unittest.FailingTestCase

class that runs a TestCase, catches any errors in it, and signals an
error ("test foo passed unexpectedly") if it runs cleanly ?

</F>




From rwgk at yahoo.com  Sun Jan  8 16:35:53 2006
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Sun, 8 Jan 2006 07:35:53 -0800 (PST)
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <dpr8cn$6h9$1@sea.gmane.org>
Message-ID: <20060108153553.78685.qmail@web31510.mail.mud.yahoo.com>

--- Fredrik Lundh <fredrik at pythonware.com> wrote:

> Ralf W. Grosse-Kunstleve wrote:
> 
> > ***Please*** make Python more selfish. Note that this is also an obvious
> avenue
> > for significant performance increases. If self is implicit you don't have
> to do
> > the dictionary lookup for "self" all the time as is the case now.
> 
> what dictionary lookup ?

IIUC, "self" is first looked up in the local dictionary.

Please try the code below to see the performance impact. As an alternative to a
copy/paste exercise try this:

wget http://cci.lbl.gov/~rwgk/python/self_lookup.py
python self_lookup.py

The output with Python 2.4.1/Fedora3/Opteron is:

loop_function: 13.10
   loop_class: 17.11

Cheers,
        Ralf


import time

def loop_function(x, n):
  result = 0
  for i in xrange(n):
    result += x
  return result

class loop_class:

  def __init__(self, x, n):
    self.x = x
    self.n = n

  def __call__(self):
    result = 0
    for i in xrange(self.n):
      result += self.x
    return result

def run(x=3.1415, n=10**8):
  t0 = time.time()
  loop_function(x=x, n=n)
  print "loop_function: %.2f" % (time.time() - t0)
  t0 = time.time()
  loop_class(x=x, n=n)()
  print "   loop_class: %.2f" % (time.time() - t0)

if (__name__ == "__main__"):
  run()



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From fredrik at pythonware.com  Sun Jan  8 16:49:36 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 8 Jan 2006 16:49:36 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
References: <dpr8cn$6h9$1@sea.gmane.org>
	<20060108153553.78685.qmail@web31510.mail.mud.yahoo.com>
Message-ID: <dprcai$j8v$1@sea.gmane.org>

Ralf W. Grosse-Kunstleve wrote:

> > what dictionary lookup ?
>
> IIUC, "self" is first looked up in the local dictionary.

no, self is a local variable.  self.x means that "x" is looked up in the in-
stance dictionary, though.

> Please try the code below to see the performance impact.

oh, please.  do you seriously think that if you don't have to type "self"
yourself, Python will suddenly be able to turn all instance variables into
local function variables without any performance overhead ?

</F>




From rwgk at yahoo.com  Sun Jan  8 16:54:34 2006
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Sun, 8 Jan 2006 07:54:34 -0800 (PST)
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
Message-ID: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com>

--- Guido van Rossum <guido at python.org> wrote:

> On 1/6/06, Armin Rigo <arigo at tunes.org> wrote:
> > On Fri, Jan 06, 2006 at 12:56:01AM +0300, Alexander Kozlovsky wrote:
> > > There are three different peculiarity in Python 2.x
> > > in respect of 'self' method argument:
> >
> > Yuk!  This has been discussed again and again already.  *Please* move
> > this discussion to comp.lang.python.
> 
> Yes please.

This is a death penalty, even without your explicit rejection. No professional
person has the time to sift through that volume of noise.

> This won't change.

The outside world is trying to tell you something; and they are not all dumb as
you suggest in your other posting.

I suggest creating python-pep at python.org.

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From thomas at xs4all.net  Sun Jan  8 16:58:16 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 8 Jan 2006 16:58:16 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108153553.78685.qmail@web31510.mail.mud.yahoo.com>
References: <dpr8cn$6h9$1@sea.gmane.org>
	<20060108153553.78685.qmail@web31510.mail.mud.yahoo.com>
Message-ID: <20060108155816.GI18916@xs4all.nl>

On Sun, Jan 08, 2006 at 07:35:53AM -0800, Ralf W. Grosse-Kunstleve wrote:

> IIUC, "self" is first looked up in the local dictionary.

No. Local variables are stored in a tuple (more or less,) and indexed by,
er, index. Loading a local variable onto the stack is a fairly fast
operation.

> Please try the code below to see the performance impact. As an alternative to a
> copy/paste exercise try this:
> 
> wget http://cci.lbl.gov/~rwgk/python/self_lookup.py
> python self_lookup.py

> The output with Python 2.4.1/Fedora3/Opteron is:
> 
> loop_function: 13.10
>    loop_class: 17.11

The main difference isn't the lookup of 'self', it's the attribute retrieval
of 'x' from 'self'.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From rwgk at yahoo.com  Sun Jan  8 16:58:29 2006
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Sun, 8 Jan 2006 07:58:29 -0800 (PST)
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <dprcai$j8v$1@sea.gmane.org>
Message-ID: <20060108155829.69611.qmail@web31513.mail.mud.yahoo.com>

--- Fredrik Lundh <fredrik at pythonware.com> wrote:
> > Please try the code below to see the performance impact.
> 
> oh, please.  do you seriously think that if you don't have to type "self"
> yourself, Python will suddenly be able to turn all instance variables into
> local function variables without any performance overhead ?

Yes, exactly, since you can make the local namespace vs. instance attribute
distinction in C.

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From pedronis at strakt.com  Sun Jan  8 17:13:27 2006
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sun, 08 Jan 2006 17:13:27 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108155829.69611.qmail@web31513.mail.mud.yahoo.com>
References: <20060108155829.69611.qmail@web31513.mail.mud.yahoo.com>
Message-ID: <43C13A27.5030702@strakt.com>

Ralf W. Grosse-Kunstleve wrote:
> --- Fredrik Lundh <fredrik at pythonware.com> wrote:
> 
>>>Please try the code below to see the performance impact.
>>
>>oh, please.  do you seriously think that if you don't have to type "self"
>>yourself, Python will suddenly be able to turn all instance variables into
>>local function variables without any performance overhead ?
> 
> 
> Yes, exactly, since you can make the local namespace vs. instance attribute
> distinction in C.
> 
> Cheers,
>         Ralf
> 
> 

that indeed proves why this discussion for whatever value it has, 
belongs to comp.lang.python. This is not a place to clarify people
how Python currently works, or to have discussions made of guesses and 
half-formed speculations.

From rwgk at yahoo.com  Sun Jan  8 17:09:22 2006
From: rwgk at yahoo.com (Ralf W. Grosse-Kunstleve)
Date: Sun, 8 Jan 2006 08:09:22 -0800 (PST)
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108155816.GI18916@xs4all.nl>
Message-ID: <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com>

--- Thomas Wouters <thomas at xs4all.net> wrote:

> The main difference isn't the lookup of 'self', it's the attribute retrieval
> of 'x' from 'self'.

I see. Thanks!
If you put 'self' into a special category (with corresponding C code), couldn't
you use the same "indexing" tricks as for local variables and make 'self.x'
just as fast as 'x'?

Cheers,
        Ralf



		
__________________________________________ 
Yahoo! DSL ? Something to write home about. 
Just $16.99/mo. or less. 
dsl.yahoo.com 


From amk at amk.ca  Sun Jan  8 18:21:16 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Sun, 8 Jan 2006 12:21:16 -0500
Subject: [Python-Dev] PyCon TX 2006: Early-bird registration ends Jan. 15!
Message-ID: <20060108172116.GB8928@rogue.amk.ca>

Early bird registration for PyCon TX 2006 ends on January 15th, so
there's only a week left. To register, please go to:

        http://us.pycon.org/TX2006/Registration

You can still register after Jan. 15th, but the cost will go up by
US$65 (US$25 for students).

This year PyCon will feature a day of tutorials before the three days
of regular presentations.  Course outlines for all the tutorials have
been posted; see

        http://wiki.python.org/moin/PyCon2006/Tutorials

All of the PyCon tutorials are still open for new registrations, but
space is limited.

Don't forget to book your hotel room, too.  PyCon TX 2006 is being
held at a Dallas/Addison hotel, and we have negotiated a special low
rate of $79 plus applicable taxes:

        http://us.pycon.org/Addison/Hotels

We hope to see you in Texas!

Regards,


A.M. Kuchling
amk at amk.ca
Chair, PyCon 2006
http://us.pycon.org


From thomas at xs4all.net  Sun Jan  8 17:26:59 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 8 Jan 2006 17:26:59 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com>
References: <20060108155816.GI18916@xs4all.nl>
	<20060108160922.88708.qmail@web31507.mail.mud.yahoo.com>
Message-ID: <20060108162659.GJ18916@xs4all.nl>

On Sun, Jan 08, 2006 at 08:09:22AM -0800, Ralf W. Grosse-Kunstleve wrote:
> --- Thomas Wouters <thomas at xs4all.net> wrote:
> > The main difference isn't the lookup of 'self', it's the attribute retrieval
> > of 'x' from 'self'.

> I see. Thanks!
> If you put 'self' into a special category (with corresponding C code), couldn't
> you use the same "indexing" tricks as for local variables and make 'self.x'
> just as fast as 'x'?

No need to make 'self' magical for that. You do, however, need to make the
*object* magical for that. It works for local namespaces because the Python
compiler knows exactly which names are local and which aren't. There are two
cases where the compiler can't know for sure (using 'exec' without 'in' and
using 'from module import *') and in those cases, Python doesn't optimize
local variable access all the way. This all won't work for objects and their
attributes because too much is possible at runtime for the current compiler
to handle.

The compiler could, of course, check everything in the whole program,
keeping track of what could possibly adjust which attributes of what object
where. That would be 'type inferencing' and has been (and is being, I guess)
attempted by various people in various projets. However, if your compiler is
so smart, it won't have _any_ problems taking an exlicit self into account :)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From pje at telecommunity.com  Sun Jan  8 17:27:16 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 08 Jan 2006 11:27:16 -0500
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108160922.88708.qmail@web31507.mail.mud.yahoo.com>
References: <20060108155816.GI18916@xs4all.nl>
Message-ID: <5.1.1.6.0.20060108111702.01b12308@mail.telecommunity.com>

At 08:09 AM 1/8/2006 -0800, Ralf W. Grosse-Kunstleve wrote:
>--- Thomas Wouters <thomas at xs4all.net> wrote:
>
> > The main difference isn't the lookup of 'self', it's the attribute 
> retrieval
> > of 'x' from 'self'.
>
>I see. Thanks!
>If you put 'self' into a special category (with corresponding C code), 
>couldn't
>you use the same "indexing" tricks as for local variables and make 'self.x'
>just as fast as 'x'?

It's not nearly that simple.  See e.g.:

http://mail.python.org/pipermail/python-dev/2002-February/019854.html

Note, however, that such a speedup is entirely independent of the 
syntax.  Trying to link the syntax with the performance is completely bogus.


From pje at telecommunity.com  Sun Jan  8 17:31:00 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun, 08 Jan 2006 11:31:00 -0500
Subject: [Python-Dev] Rejected ideas PEP (was re: Draft proposal: Implicit
 self in Python 3.0)
In-Reply-To: <20060108122527.GH18916@xs4all.nl>
References: <ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060108112847.043ff7e8@mail.telecommunity.com>

At 01:25 PM 1/8/2006 +0100, Thomas Wouters wrote:
>On Sat, Jan 07, 2006 at 05:12:06PM -0800, Guido van Rossum wrote:
> > On 1/6/06, Kay Schluehr <kay.schluehr at gmx.net> wrote:
> > > Then simply reject the PEP and the discussion can be stopped on
> > > comp.lang.python too.
>
> > Only in the most severe cases does it make sense to create a PEP
> > specifically to be rejected.
>
>Yet if it isn't recorded, people will keep bringing it up. How about a
>'rejected ideas' PEP for ideas that are right out no matter how people
>argue? A single PEP, with oneliners to describe ideas, one or two lines to
>explain 'why not', references to the python-list or python-dev discussions,
>if any, and a sign in big friendly letters saying "if you really must
>discuss these subjects, do it on python-list, not python-dev". Some of the
>stuff that could be in there:
>
>  - implicit rather than explicit self: invalidates too many tricks
>  - omitting () on functioncalls: like implicit self
>  - changing all statements into expressions (e.g. logix): python isn't
>    (going to be) a functional language
>  - methods for tuples: tuples are records, not collections; use lists instead
>  - sigils to indicate 'self.' (e.g. @foo): loses readability, wins too little
>  - '?' and other non-alphanumerical characters in identifiers: like sigils
>  - strict private/public accessing: 'fake' protection; use closures instead
>
>etc. No need to come up with them all, I'm sure all the prize ideas will pop
>back up eventually ;)

+1, along with a modification to the PEP procedures to require that people 
check the rejected ideas PEP before submitting a new proposal.  The 
"rejected ideas" PEP should presumably be a low-numbered process PEP.


From thomas at xs4all.net  Sun Jan  8 17:37:12 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 8 Jan 2006 17:37:12 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C10704.80404@v.loewis.de>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de>
Message-ID: <20060108163712.GK18916@xs4all.nl>

On Sun, Jan 08, 2006 at 01:35:16PM +0100, "Martin v. L?wis" wrote:
> Thomas Wouters wrote:
> > Yet if it isn't recorded, people will keep bringing it up. How about a
> > 'rejected ideas' PEP for ideas that are right out no matter how people
> > argue?
> Recorded it is, in the mailing list archive.

> However, a central place might be better, preferably with referrals to
> pronouncements or other discussion. A Wiki comes to mind...

My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ,
the minds of many, many people, not just Python developers) but that it
isn't easily findable and it isn't easily accessible in a single location. I
thought PEP's where supposed to be that, and if I have a particular idea for
new syntax or new semantics, PEPs would be the place I'd look, not the FAQ
or a Wiki. I guess a Wiki'd work, but then I wonder, why aren't all PEPs in
a Wiki instead? At least until they are actually accepted, after which they
get finalized and formalized and implemented and what not. (I'm not a fan of
Wiki's, but that may be because I've seen too many chaotic, unstructured,
messy, unmaintained ones.)

And I would like to point out how hard it is to google or grep for ideas
like this. For instance, the 'x, y, *rest = someseq' syntax. I wouldn't know
what to call it (or what others would call it, and you can't google for the
syntax (since the variables can be named anything). Grepping through
downloaded mailinglist archives might find you some discussion, but probably
without any indication whether anything is final or current. Grepping
through the PEPs might find you something, if it's PEP'ed, but it's a lot
easier to browse the list of (non-accepted/implemented) PEPs. You may not
know the name of the construct you're looking for, but someone else's name
for it would likely ring a bell. And a PEP titled 'Rejected Ideas' would
certainly be the first place to look ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From tim.peters at gmail.com  Sun Jan  8 17:42:09 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 8 Jan 2006 11:42:09 -0500
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <43C0DF8C.7050104@v.loewis.de>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
Message-ID: <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>

[neal.norwitz]
>>               PyErr_Format(PyExc_ValueError,
>> -                          "%s() requires a code object with %d free vars,"
>> -                          " not %d",
>> +                          "%s() requires a code object with %ld free vars,"
>> +                          " not %ld",
>>                            PyString_AsString(op->func_name),
>>                            nclosure, nfree);

[Martin v. L?wis]
> I'm not sure whether this is the right fix. This says nclosure and nfree
> are longs; this is true on most 32-bit systems (where ssize_t is int
> and int and long have the same width), and on some 64-bit systems (where
> ssize_t and long are the same). It does not work for Win64 (where
> ssize_t is larger than long).

Ya, that doesn't work.  We'll have similar problems with obmalloc
debug output if obmalloc is changed to escape 32-bitness on Win64 --
and anywhere else we need to format one of these things.

> I'm tempted to use %zd in these places, but fear that it isn't portable
> enough (it *is* part of C99).

MS VC 7.1 doesn't support the z length qualifier, and I expect that
kills it right there.  So we need a macro, say Py_SIZE_T_WIDTH, in
pyport.h:

/* Length qualifier for use in formats when printing size_t or
Py_ssize_t values. */
#ifndef Py_SIZE_T_WIDTH
#   define Py_SIZE_T_WIDTH "z"
#endif

That handles C99-ish platforms by defalt.  Other platforms (like
Windows) would need to supply their own expansion in their pyconfig.h.
 Neal's format would become the god-awful
but platform-neutral:

          "%s() requires a code object with %" Py_SIZE_T_WIDTH "d free vars,"
          " not %" Py_SIZE_T_WIDTH "d"

I suspect we're going to have other problems when someone gets around
to passing a size_t-ish value to PyString_Format() or PyErr_Format(). 
Maybe we could teach those about the "z" qualifier on all platforms.

From tim.peters at gmail.com  Sun Jan  8 18:16:13 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 8 Jan 2006 12:16:13 -0500
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108163712.GK18916@xs4all.nl>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de>
	<20060108163712.GK18916@xs4all.nl>
Message-ID: <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>

[Thomas Wouters]
> My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ,
> the minds of many, many people, not just Python developers) but that it
> isn't easily findable and it isn't easily accessible in a single location. I
> thought PEP's where supposed to be that, and if I have a particular idea for
> new syntax or new semantics, PEPs would be the place I'd look, not the FAQ
> or a Wiki.

Luckily, in his benevolent infinite wisdom, I expect Guido reserved
PEP number 13 for exactly this purpose:  for a meta-PEP to record the
unlucky PEP ideas that are so unlikely to get accepted that it's not
worth anyone's time to write an actual PEP for them.  I like the
title:

    Don't Bother:  PEPs Rejected Before Being Written

No, I'm not kidding.  At least I don't think I am.

> ...
> And I would like to point out how hard it is to google or grep for ideas
> like this. For instance, the 'x, y, *rest = someseq' syntax. I wouldn't know
> what to call it (or what others would call it, and you can't google for the
> syntax (since the variables can be named anything).

If PEP 13 grows large enough, won't be easy to find there either.  But
it will be easy to remember _where_ to look.

> ...
> And a PEP titled 'Rejected Ideas' would certainly be the first place to look ;)

Too bland ;-)

From martin at v.loewis.de  Sun Jan  8 18:19:19 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 18:19:19 +0100
Subject: [Python-Dev] Checking in a broken test was:
 Re:	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dpr983$95r$1@sea.gmane.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dpr983$95r$1@sea.gmane.org>
Message-ID: <43C14997.30309@v.loewis.de>

Fredrik Lundh wrote:
> many test frameworks support "expected failures" for this purpose.
> how hard would it be to add a
> 
>     unittest.FailingTestCase
> 
> class that runs a TestCase, catches any errors in it, and signals an
> error ("test foo passed unexpectedly") if it runs cleanly ?

I don't know how hard it would be, but I would also consider this
appropriate. Of course, this should work on a case-by-case basis:
if there are multiple test methods in a class, unexpected passes
of each method should be reported.

Regards,
Martin

From martin at v.loewis.de  Sun Jan  8 18:22:56 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 18:22:56 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com>
References: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com>
Message-ID: <43C14A70.1050206@v.loewis.de>

Ralf W. Grosse-Kunstleve wrote:
> The outside world is trying to tell you something; and they are not all dumb as
> you suggest in your other posting.

But they might suffer from misunderstandings, such as your
misunderstanding in how local variables work and whether
'self' is looked up in a dictionary.

So it's being dumb - just being uninformed.

Regards,
Martin

From martin at v.loewis.de  Sun Jan  8 18:31:35 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 18:31:35 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108163712.GK18916@xs4all.nl>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>
	<43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl>
Message-ID: <43C14C77.8010308@v.loewis.de>

Thomas Wouters wrote:
> My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ,
> the minds of many, many people, not just Python developers) but that it
> isn't easily findable and it isn't easily accessible in a single location.

Why would a single Wiki page not be accessible in a single location? Why
is the FAQ not accessible in a single location?

> I
> thought PEP's where supposed to be that, and if I have a particular idea for
> new syntax or new semantics, PEPs would be the place I'd look, not the FAQ
> or a Wiki.

Right. However, I doubt you would look in the "rejected ideas" PEP.

> I guess a Wiki'd work, but then I wonder, why aren't all PEPs in
> a Wiki instead?

There might be several reasons - one  is that a PEP is supposed to get
posted to mailing lists and newsgroups, something that is not readily
possible with Wiki pages.

> And I would like to point out how hard it is to google or grep for ideas
> like this. For instance, the 'x, y, *rest = someseq' syntax.

I agree that collecting them in a single place is a good idea. Whether
this is a PEP, a Wiki page, or the FAQ is nearly irrelevant, except:
- it is strange to call it a "Python Enhancement Proposal"
- in either the FAQ or the PEP, it will stay in its initial form
  forever, since nobody but the original author will edit it
  (that could be true of a Wiki page as well, but on a Wiki page,
  people *know* they are meant to edit it if they want to say
  something)

Regards,
Martin

From martin at v.loewis.de  Sun Jan  8 18:32:41 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 18:32:41 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C14A70.1050206@v.loewis.de>
References: <20060108155434.24567.qmail@web31511.mail.mud.yahoo.com>
	<43C14A70.1050206@v.loewis.de>
Message-ID: <43C14CB9.8050906@v.loewis.de>

Martin v. L?wis wrote:
> But they might suffer from misunderstandings, such as your
> misunderstanding in how local variables work and whether
> 'self' is looked up in a dictionary.
> 
> So it's being dumb - just being uninformed.

Sorry: "*not* being dumb" is what I wanted to say.

Regards,
Martin

From tim.peters at gmail.com  Sun Jan  8 18:42:09 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 8 Jan 2006 12:42:09 -0500
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
Message-ID: <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com>

[Tim]
> ...
> I suspect we're going to have other problems when someone gets around
> to passing a size_t-ish value to PyString_Format() or PyErr_Format().
> Maybe we could teach those about the "z" qualifier on all platforms.

Yup.  Because the supporting PyString_FromFormatV() doesn't know
anything about z qualifiers now, if you _tried_ to use %zd then even
on a C99-platform PyString_FromFormatV() wouldn't do any conversions
from that point on:  it would just copy the format string from the
first "z" onward into the output buffer, and ignore the trailing
inputs.  The good news is that we could teach it about "z" (but would
still need a macro or some such to implement "z" correctly across
platforms).

From fdrake at acm.org  Sun Jan  8 18:34:59 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun, 8 Jan 2006 12:34:59 -0500
Subject: [Python-Dev]
 =?iso-8859-1?q?Checking_in_a_broken_test_was=3A_Re?=
 =?iso-8859-1?q?=3A=09=5BPython-checkins=5Dr41940_-_python/trunk/Lib/test/?=
 =?iso-8859-1?q?test=5Fcompiler=2Epy?=
In-Reply-To: <43C14997.30309@v.loewis.de>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dpr983$95r$1@sea.gmane.org> <43C14997.30309@v.loewis.de>
Message-ID: <200601081234.59951.fdrake@acm.org>

On Sunday 08 January 2006 12:19, Martin v. L?wis wrote:
 > I don't know how hard it would be, but I would also consider this
 > appropriate. Of course, this should work on a case-by-case basis:
 > if there are multiple test methods in a class, unexpected passes
 > of each method should be reported.

I like the way trial (from twisted) supports this.  The test method is written 
normally, in whatever class makes sense.  Then the test is marked with an 
attribute to say it isn't expected to pass yet.  When the code is fixed and 
the test passes, you get that information in trial's output, and can unmark 
the test.  This avoids having to refactor test classes just to update the 
status of a test.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From martin at v.loewis.de  Sun Jan  8 18:52:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 18:52:39 +0100
Subject: [Python-Dev] [Python-checkins] r41972
	-	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
Message-ID: <43C15167.7000702@v.loewis.de>

Tim Peters wrote:
> That handles C99-ish platforms by defalt.  Other platforms (like
> Windows) would need to supply their own expansion in their pyconfig.h.
>  Neal's format would become the god-awful
> but platform-neutral:
> 
>           "%s() requires a code object with %" Py_SIZE_T_WIDTH "d free vars,"
>           " not %" Py_SIZE_T_WIDTH "d"

That's too ugly, IMO. So I could accept to write

        "%s() requires a code object with %ld free vars,"
        " not %ld", PyString_AsString(op->func_name),
 	(long)nclosure, (long)nfree);

instead. Alternatively, if there is a real chance that it overflows
LONG_MAX (which there isn't, AFAIK - we don't support that many free
vars), we could also write

        "%s() requires a code object with %ld free vars,"
        " not %ld", PyString_AsString(op->func_name),
 	Py_long_ceil(nclosure), Py_long_ceil(nfree));

with

#define Py_long_ceil(n) ((n) > LONG_MAX ? LONG_MAX : \
    (n) < LONG_MIN ? LONG_MIN : (n))

On most platforms, the compiler should notice that the conditions
are both known false, and collaps this to (n).

If this ever overflows, somebody will have to remember that
this is really +/- infinity.

Regards,
Martin

P.S. Proposals for a different macro name are thankfully accepted.

From martin at v.loewis.de  Sun Jan  8 18:59:35 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 18:59:35 +0100
Subject: [Python-Dev] [Python-checkins] r41972
	-	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>	<43C0DF8C.7050104@v.loewis.de>	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com>
Message-ID: <43C15307.2060600@v.loewis.de>

Tim Peters wrote:
> Yup.  Because the supporting PyString_FromFormatV() doesn't know
> anything about z qualifiers now, if you _tried_ to use %zd then even
> on a C99-platform PyString_FromFormatV() wouldn't do any conversions
> from that point on:  it would just copy the format string from the
> first "z" onward into the output buffer, and ignore the trailing
> inputs.  The good news is that we could teach it about "z" (but would
> still need a macro or some such to implement "z" correctly across
> platforms).

On VC7.1, we could use 'L', right? On other platforms, we could check
whether sizeof(size_t) is sizeof(long), and use 'l', else we could
refuse compilation (I doubt this would rule out any relevant system).

Regards,
Martin

From martin at v.loewis.de  Sun Jan  8 19:01:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 08 Jan 2006 19:01:20 +0100
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <200601081234.59951.fdrake@acm.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>
	<43C14997.30309@v.loewis.de> <200601081234.59951.fdrake@acm.org>
Message-ID: <43C15370.2070700@v.loewis.de>

Fred L. Drake, Jr. wrote:
> I like the way trial (from twisted) supports this.  The test method is written 
> normally, in whatever class makes sense.  Then the test is marked with an 
> attribute to say it isn't expected to pass yet.  When the code is fixed and 
> the test passes, you get that information in trial's output, and can unmark 
> the test.  This avoids having to refactor test classes just to update the 
> status of a test.

So how is the mark added? I would suggest

   @xfail
   def test_foo(self):
     self.assertEquals(0,1)

Regards,
Martin

From foom at fuhm.net  Sun Jan  8 19:09:27 2006
From: foom at fuhm.net (James Y Knight)
Date: Sun, 8 Jan 2006 13:09:27 -0500
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <43C15370.2070700@v.loewis.de>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>
	<43C14997.30309@v.loewis.de> <200601081234.59951.fdrake@acm.org>
	<43C15370.2070700@v.loewis.de>
Message-ID: <B0689571-E980-44FF-B745-4A1620B1AF0C@fuhm.net>


On Jan 8, 2006, at 1:01 PM, Martin v. L?wis wrote:

> Fred L. Drake, Jr. wrote:
>
>> I like the way trial (from twisted) supports this.  The test  
>> method is written
>> normally, in whatever class makes sense.  Then the test is marked  
>> with an
>> attribute to say it isn't expected to pass yet.  When the code is  
>> fixed and
>> the test passes, you get that information in trial's output, and  
>> can unmark
>> the test.  This avoids having to refactor test classes just to  
>> update the
>> status of a test.
>>
>
> So how is the mark added? I would suggest
>
>    @xfail
>    def test_foo(self):
>      self.assertEquals(0,1)

   def test_foo(self):
     self.assertEquals(0,1)
   test_foo.todo = "Why this test fails description message."

James

From g.brandl-nospam at gmx.net  Sun Jan  8 20:04:45 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sun, 08 Jan 2006 20:04:45 +0100
Subject: [Python-Dev] test_curses
Message-ID: <dprnoe$p98$1@sea.gmane.org>

The call to curses.setupterm() leaves my terminal in a bad state.

The reset program outputs:
Erase set to delete.
Kill set to control-U (^U).
Interrupt set to control-C (^C).

Doesn't the setupterm() have to be paired with something like shutdownterm()?

regards,
Georg


From thomas at xs4all.net  Sun Jan  8 20:30:18 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 8 Jan 2006 20:30:18 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C14C77.8010308@v.loewis.de>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de>
	<20060108163712.GK18916@xs4all.nl> <43C14C77.8010308@v.loewis.de>
Message-ID: <20060108193018.GL18916@xs4all.nl>

On Sun, Jan 08, 2006 at 06:31:35PM +0100, "Martin v. L?wis" wrote:
> Thomas Wouters wrote:
> > My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ,
> > the minds of many, many people, not just Python developers) but that it
> > isn't easily findable and it isn't easily accessible in a single location.

> Why would a single Wiki page not be accessible in a single location? Why
> is the FAQ not accessible in a single location?

Unless either of those would replace PEPs (or include all information from
all PEPs), they would be a second location.

> > I thought PEP's where supposed to be that, and if I have a particular
> > idea for new syntax or new semantics, PEPs would be the place I'd look,
> > not the FAQ or a Wiki.

> Right. However, I doubt you would look in the "rejected ideas" PEP.

You may have the luxury of that doubt, but I don't :) I'm sorry if my
comments sounded hypothetical; they weren't. I did wonder about (among other
things) 'x, y, *rest = l', and because I didn't know the name, I did browse
the PEP list. A 'Rejected Ideas' PEP would've caught my eye even if I didn't
have a hunch Guido would reject 'x, y, *r = l'.

> I agree that collecting them in a single place is a good idea. Whether
> this is a PEP, a Wiki page, or the FAQ is nearly irrelevant, except:
> - it is strange to call it a "Python Enhancement Proposal"

They are proposals to enhance Python, though. That's exactly why I argue
they should be in a PEP, not in some other location: it's all neatly bundled
together. It's just not a PEP per single proposal.

> - in either the FAQ or the PEP, it will stay in its initial form
>   forever, since nobody but the original author will edit it
>   (that could be true of a Wiki page as well, but on a Wiki page,
>   people *know* they are meant to edit it if they want to say
>   something)

I don't think the rejected-ideas collection should be publically editable
either. It's not a discussion forum, it's not meant to get people to
persuade Guido to change his mind, it's a list of "these things just aren't
going to happen, deal with it". A Meta-PEP like Tim suggested seems most
appropriate to me, even if it isn't purely Meta; practicality beats purity
and all that.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From g.brandl-nospam at gmx.net  Sun Jan  8 20:39:08 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sun, 08 Jan 2006 20:39:08 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
Message-ID: <dprpos$us8$1@sea.gmane.org>

Hi,

 >>> import locale
 >>> locale.setlocale(locale.LC_NUMERIC, "")
'de_DE at euro'
 >>> "%f" % 1.0
'1.000000'
 >>> u"%f" % 1.0
u'1,000000'
 >>>


Is this intended? This breaks test_format on my box when test_builtin (method
test_float_with_comma) is executed first.

regards,
Georg


From ianb at colorstudy.com  Sun Jan  8 21:43:17 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 08 Jan 2006 14:43:17 -0600
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>
	<43C10704.80404@v.loewis.de>	<20060108163712.GK18916@xs4all.nl>
	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
Message-ID: <43C17965.5070408@colorstudy.com>

Tim Peters wrote:
> [Thomas Wouters]
> 
>>My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ,
>>the minds of many, many people, not just Python developers) but that it
>>isn't easily findable and it isn't easily accessible in a single location. I
>>thought PEP's where supposed to be that, and if I have a particular idea for
>>new syntax or new semantics, PEPs would be the place I'd look, not the FAQ
>>or a Wiki.
> 
> 
> Luckily, in his benevolent infinite wisdom, I expect Guido reserved
> PEP number 13 for exactly this purpose:  for a meta-PEP to record the
> unlucky PEP ideas that are so unlikely to get accepted that it's not
> worth anyone's time to write an actual PEP for them.  I like the
> title:
> 
>     Don't Bother:  PEPs Rejected Before Being Written
> 
> No, I'm not kidding.  At least I don't think I am.

+1.  I think it's rather patronizing to send things back to python-list 
when you know people are wasting their time discussing them because they 
will never be accepted.  People on python-list have useful things to do 
too, they don't just read to waste their time.

I would prefer PEP form over a wiki page, as I'd rather this be truly 
authoritative, and only Guido can really completely reject an idea. 
Justifying the rejections can be done by anyone though; maybe each idea 
could link to a wiki page on the topic.

I think it's also important to be clear on what's being rejected.  Often 
these rejected ideas address a real issue, and if you think about the 
issue from another angle you might be able to solve that without falling 
into the trap that the oft-rejected proposal fell into.  But it's easy 
to confuse that the issue or use case is being explicitly ignored, 
rather than the particulars.  For instance, Thomas said "changing all 
statements into expressions (e.g. logix): python isn't (going to be) a 
functional language" -- and that's what shouldn't be in the PEP.  All 
statements aren't going to be expressions; the editorialization that 
Python isn't going to be a functional language is both rather 
inaccurate, misses the real reason for statements, and needlessly 
alienates people who like functional programming (and they have been 
*needlessly* alienated by discussions about lambda and filter).

So... maybe Guido or python-dev should write/vet the justifications too. 
  When you are putting up walls and specifically discouraging community 
participation on certain issues, it should be done in a sensitive way.


-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From pedronis at strakt.com  Sun Jan  8 22:19:17 2006
From: pedronis at strakt.com (Samuele Pedroni)
Date: Sun, 08 Jan 2006 22:19:17 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C17965.5070408@colorstudy.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>	<43C10704.80404@v.loewis.de>	<20060108163712.GK18916@xs4all.nl>	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
	<43C17965.5070408@colorstudy.com>
Message-ID: <43C181D5.8040702@strakt.com>

Ian Bicking wrote:
> Tim Peters wrote:
> 
>>[Thomas Wouters]
>>
>>
>>>My point isn't that it isn't archived somewhere (mailinglists, wiki, FAQ,
>>>the minds of many, many people, not just Python developers) but that it
>>>isn't easily findable and it isn't easily accessible in a single location. I
>>>thought PEP's where supposed to be that, and if I have a particular idea for
>>>new syntax or new semantics, PEPs would be the place I'd look, not the FAQ
>>>or a Wiki.
>>
>>
>>Luckily, in his benevolent infinite wisdom, I expect Guido reserved
>>PEP number 13 for exactly this purpose:  for a meta-PEP to record the
>>unlucky PEP ideas that are so unlikely to get accepted that it's not
>>worth anyone's time to write an actual PEP for them.  I like the
>>title:
>>
>>    Don't Bother:  PEPs Rejected Before Being Written
>>
>>No, I'm not kidding.  At least I don't think I am.
> 
> 
> +1.  I think it's rather patronizing to send things back to python-list 
> when you know people are wasting their time discussing them because they 
> will never be accepted.

Well, the problem is that most of these proposers think that their 
proposal deserve attention, is valuable no matter what, but python-dev
is not the place to defuse/clarify that in a gentle way for the 
proposer. Of course some people do exactly that here out of kindness 
etc., I personally don't think it's a good approach.
python-list is a better place to get clarifications about the way Python
is, will stay.

Half-baked, not well thought out ideas, as huge threads in the past
going nowhere have proven, are not healthy to python-dev s/n ratio.

Even with such a valuable PEP in place, I expect some people to need 
clarification discussions and python-list will still be the right place 
to have them.


>  People on python-list have useful things to do 
> too, they don't just read to waste their time.
> 
> I would prefer PEP form over a wiki page, as I'd rather this be truly 
> authoritative, and only Guido can really completely reject an idea. 
> Justifying the rejections can be done by anyone though; maybe each idea 
> could link to a wiki page on the topic.
> 
> I think it's also important to be clear on what's being rejected.  Often 
> these rejected ideas address a real issue, and if you think about the 
> issue from another angle you might be able to solve that without falling 
> into the trap that the oft-rejected proposal fell into.  But it's easy 
> to confuse that the issue or use case is being explicitly ignored, 
> rather than the particulars.  For instance, Thomas said "changing all 
> statements into expressions (e.g. logix): python isn't (going to be) a 
> functional language" -- and that's what shouldn't be in the PEP.  All 
> statements aren't going to be expressions; the editorialization that 
> Python isn't going to be a functional language is both rather 
> inaccurate, misses the real reason for statements, and needlessly 
> alienates people who like functional programming (and they have been 
> *needlessly* alienated by discussions about lambda and filter).
> 
> So... maybe Guido or python-dev should write/vet the justifications too. 
>   When you are putting up walls and specifically discouraging community 
> participation on certain issues, it should be done in a sensitive way.
> 
> 


From fredrik at pythonware.com  Sun Jan  8 22:02:19 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 8 Jan 2006 22:02:19 +0100
Subject: [Python-Dev] [Doc-SIG] that library reference, again
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>	<43B4A238.2030101@python.org>	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net><dp2mmh$45b$1@sea.gmane.org><43B56EEE.9090904@egenix.com>
	<dpot8g$8a9$1@sea.gmane.org>
Message-ID: <dprukt$goj$1@sea.gmane.org>

> anyone knows anything about support for semantic markup ?

http://meta.wikimedia.org/wiki/Semantic_MediaWiki

not sure a full-blown RDF-in-wiki-syntax is really that optimal,
though ;-)

</F>




From nnorwitz at gmail.com  Sun Jan  8 22:47:12 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 8 Jan 2006 13:47:12 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dpr983$95r$1@sea.gmane.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dpr983$95r$1@sea.gmane.org>
Message-ID: <ee2a432c0601081347y42b8d86p6773f1887aa6ccf9@mail.gmail.com>

On 1/8/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
>
> many test frameworks support "expected failures" for this purpose.
> how hard would it be to add a
>
>     unittest.FailingTestCase
>
> class that runs a TestCase, catches any errors in it, and signals an
> error ("test foo passed unexpectedly") if it runs cleanly ?

http://python.org/sf/1399935

From thomas at xs4all.net  Sun Jan  8 22:47:47 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sun, 8 Jan 2006 22:47:47 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C17965.5070408@colorstudy.com>
References: <611494241.20060106005601@gmail.com>
	<20060106193448.GA656@code1.codespeak.net>
	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>
	<dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de>
	<20060108163712.GK18916@xs4all.nl>
	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
	<43C17965.5070408@colorstudy.com>
Message-ID: <20060108214747.GM18916@xs4all.nl>

On Sun, Jan 08, 2006 at 02:43:17PM -0600, Ian Bicking wrote:

> [T]he editorialization that Python isn't going to be a functional language
> is both rather inaccurate, misses the real reason for statements, and
> needlessly alienates people who like functional programming

> So... maybe Guido or python-dev should write/vet the justifications too. 

Oh, none of my examples were anything but that. Particularly the
justification. These were just quick ramblings from my side; all actual
pronouncements should be made channeling or being Guido, and the editorial
review to ease sensitive minds is a group effort.

(On the topic you quoted, though, I do feel that if people are using Python
while they are waiting for Python to turn into a functional language,
they're waiting for the wrong thing. Or at least, I hope so, or *I'm*
waiting for the wrong thing. So I'm not sure what's wrong about the
editorialization, but perhaps I'm just too insensitive towards functional
programming disciples. And I'm usually quite in tune with sensitive minds.
But we can discuss that off-list ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From ianb at colorstudy.com  Sun Jan  8 23:59:57 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 08 Jan 2006 16:59:57 -0600
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060108214747.GM18916@xs4all.nl>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>
	<43C10704.80404@v.loewis.de>	<20060108163712.GK18916@xs4all.nl>	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>	<43C17965.5070408@colorstudy.com>
	<20060108214747.GM18916@xs4all.nl>
Message-ID: <43C1996D.8060902@colorstudy.com>

Thomas Wouters wrote:
>>[T]he editorialization that Python isn't going to be a functional language
>>is both rather inaccurate, misses the real reason for statements, and
>>needlessly alienates people who like functional programming
> 
> 
>>So... maybe Guido or python-dev should write/vet the justifications too. 
> 
> 
> Oh, none of my examples were anything but that. Particularly the
> justification. These were just quick ramblings from my side; all actual
> pronouncements should be made channeling or being Guido, and the editorial
> review to ease sensitive minds is a group effort.

I was also just using your example as an example ;)  As another similar 
example, around the discussions of the deprecation of map and filter I 
see a lot of misunderstandings.  People confused the deprecation of two 
Lisp functions with Guido wanting to discourage functional program in 
general.  The whole thing is based on several misconceptions, but the 
misunderstanding has become very widespread.

A big part of what happened (I think) is that people who defended the 
decision added editorialization that became confused with the original 
intentions.  And that consisted of things like "functional programming 
is hard/confusing, so it's not appropriate in Python".

I think putting these kinds of decisions on a wiki would generally lead 
to these kinds of confusions.  When rejecting an idea altogether I think 
it should be clear exactly what is being rejected, and not imply that 
anything extra is being rejected.  People read a lot

I think there is a valid perception that the Python community is not 
very open to many ideas about changing the language.  I think that's 
okay -- it's this way *because* of the participation and discussion that 
has occurred in the past, because these are old ideas that have been 
rejected and we are trying to move on.  But it leaves people in an 
awkward situation, because how can you really know what is open for 
discussion?  I don't think people on python-list really know this 
either, so asking there won't clarify that.  I doubt there's general 
consensus on python-dev about what is open for change.

I just don't want people to feel discouraged when they try to contribute 
to the Python community and a PEP 13 could help direct people towards 
areas where their contributions are more likely to be useful.  Also I 
think it is unfair to use python-list to clarify things that python-dev 
is not willing to clarify itself.


-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From martin at v.loewis.de  Mon Jan  9 00:31:46 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 00:31:46 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C1996D.8060902@colorstudy.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>	<43C10704.80404@v.loewis.de>	<20060108163712.GK18916@xs4all.nl>	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>	<43C17965.5070408@colorstudy.com>	<20060108214747.GM18916@xs4all.nl>
	<43C1996D.8060902@colorstudy.com>
Message-ID: <43C1A0E2.6030605@v.loewis.de>

Ian Bicking wrote:
> I just don't want people to feel discouraged when they try to contribute 
> to the Python community and a PEP 13 could help direct people towards 
> areas where their contributions are more likely to be useful.  Also I 
> think it is unfair to use python-list to clarify things that python-dev 
> is not willing to clarify itself.

But now: who is going to write it? "Guido should write it" clearly won't
work. And no, I'm explicitly not volunteering either.

Regards,
Martin

From martin at v.loewis.de  Mon Jan  9 00:33:27 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 00:33:27 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <dprpos$us8$1@sea.gmane.org>
References: <dprpos$us8$1@sea.gmane.org>
Message-ID: <43C1A147.60409@v.loewis.de>

Georg Brandl wrote:
>  >>> import locale
>  >>> locale.setlocale(locale.LC_NUMERIC, "")
> 'de_DE at euro'
>  >>> "%f" % 1.0
> '1.000000'
>  >>> u"%f" % 1.0
> u'1,000000'
>  >>>
> 
> 
> Is this intended? This breaks test_format on my box when test_builtin (method
> test_float_with_comma) is executed first.

No. %-formatting should always use the C locale. One should use
locale.format to get locale-aware formatting.

Regards,
Martin

From pedronis at strakt.com  Mon Jan  9 00:51:42 2006
From: pedronis at strakt.com (Samuele Pedroni)
Date: Mon, 09 Jan 2006 00:51:42 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C1996D.8060902@colorstudy.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>	<43C10704.80404@v.loewis.de>	<20060108163712.GK18916@xs4all.nl>	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>	<43C17965.5070408@colorstudy.com>	<20060108214747.GM18916@xs4all.nl>
	<43C1996D.8060902@colorstudy.com>
Message-ID: <43C1A58E.1050200@strakt.com>

Ian Bicking wrote:
> I just don't want people to feel discouraged when they try to contribute 
> to the Python community and a PEP 13 could help direct people towards 
> areas where their contributions are more likely to be useful. 

but people have a lot of options, probably more effective, ranging
from writing great applications in Python, great libraries ... plus 
implementation work before they are left with the hard option that is 
language design to contribute to the community.

> Also I 
> think it is unfair to use python-list to clarify things that python-dev 
> is not willing to clarify itself.
> 

notice that the intersection is not empty yet.

Also PEP 1 contains

"""
PEP authors are responsible for collecting community feedback on a PEP 
before submitting it for review. A PEP that has not been discussed on 
python-list at python.org and/or python-dev at python.org will not be 
accepted. However, wherever possible, long open-ended discussions on 
public mailing lists should be avoided. Strategies to keep the 
discussions efficient include: setting up a separate SIG mailing list 
for the topic, having the PEP author accept private comments in the 
early design phases, setting up a wiki page, etc. PEP authors should use 
their discretion here.
"""

in the past it often happened that water testing, honing
for a PEP happend on python-list before any long discussion went on
on python-dev. Basically I think that throwing half-cooked ideas at 
python-dev, especially for people with no track-record of language 
design contributions to Python, is just a recipe for frustration.


From tim.peters at gmail.com  Mon Jan  9 00:47:12 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 8 Jan 2006 18:47:12 -0500
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <43C15167.7000702@v.loewis.de>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<43C15167.7000702@v.loewis.de>
Message-ID: <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>

[Tim]
>> That handles C99-ish platforms by defalt.  Other platforms (like
>> Windows) would need to supply their own expansion in their pyconfig.h.
>>  Neal's format would become the god-awful
>> but platform-neutral:
>>
>>           "%s() requires a code object with %" Py_SIZE_T_WIDTH "d free vars,"
>>           " not %" Py_SIZE_T_WIDTH "d"

[Martin]
> That's too ugly, IMO. So I could accept to write
>
>         "%s() requires a code object with %ld free vars,"
>         " not %ld", PyString_AsString(op->func_name),
>         (long)nclosure, (long)nfree);
>
> instead. Alternatively, if there is a real chance that it overflows
> LONG_MAX (which there isn't, AFAIK - we don't support that many free
> vars), we could also write
>
>         "%s() requires a code object with %ld free vars,"
>         " not %ld", PyString_AsString(op->func_name),
>         Py_long_ceil(nclosure), Py_long_ceil(nfree));
>
> with
>
> #define Py_long_ceil(n) ((n) > LONG_MAX ? LONG_MAX : \
>     (n) < LONG_MIN ? LONG_MIN : (n))
>
> On most platforms, the compiler should notice that the conditions
> are both known false, and collaps this to (n).
>
> If this ever overflows, somebody will have to remember that
> this is really +/- infinity.

I think you're trying to solve a different problem.  I'm not trying to
special-case the snot out of Neil's specific line of code, along the
lines of "well, yes, the types here were changed from int to
Py_ssize_t, but that was basically bizarre since in fact plain old int
was big enough to hold all plausible values to begin with, so let's
find a way to pass Py_ssize_t arguments in cases where they're really
ints or longs".  If that's what's going on here, it's a symptom of
changing type declarations with little thought just to shut up
compiler warnings.  If `int` or `long` are in fact big enough to hold
all plausible values for these variables, then that's what they should
be declared as.

The problem I am trying to solve is formatting values that are of
types size_t or Py_ssize_t for _good_ reason:  in effect, they really
need to be 64 bits on 64-bit boxes, and "unsigned long"/"long" aren't
good enough for that on all 64-bit platforms. In these cases, using
%ld is wrong, and truncating bits is also wrong.  These cases need a
platform-correct format code.  The pain of separating out the width
modifier could be reduced by defining a few type-oriented macros
instead; e.g.,

#define PyFORMAT_ssize_t "%zd"  /* on C99 boxes */

If it made sense to use Py_ssize_t to begin with in that code, then:

         "%s() requires a code object with " PyFORMAT_ssize_t " free vars,"
         " not " PyFORMAT_ssize_t, PyString_AsString(op->func_name),
         nclosure, free);

That's no more or less painful than using C99's huge pile of PRId8,
PRId16, PRId32 (etc, etc) macros, defined there for similar purposes.

From tim.peters at gmail.com  Mon Jan  9 01:00:54 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 8 Jan 2006 19:00:54 -0500
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <43C15307.2060600@v.loewis.de>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com>
	<43C15307.2060600@v.loewis.de>
Message-ID: <1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com>

[Martin v. L?wis]
> On VC7.1, we could use 'L', right?

We could use the "I" (capital letter eye) length modifier under VC7.1.
 That's good for both size_t and ptrdiff_t formats under VC7.1, where
ptrdiff_t under VC7.1 is really the same concept as Py_ssize_t.  On
32-bit boxes, "I" means 4 bytes, and on 64-bit boxes "I" means 8
bytes.

> On other platforms, we could check whether sizeof(size_t) is sizeof(long), and
> use 'l', else we could refuse compilation (I doubt this would rule out any relevant
> system).

Assuming that's a lowercase letter el (I really can't see any
difference between upper-eye and lower-el in gmail's font), I think
that would be fine.

From thomas at xs4all.net  Mon Jan  9 01:51:23 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 9 Jan 2006 01:51:23 +0100
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C1A0E2.6030605@v.loewis.de>
References: <dpns9t$gil$1@sea.gmane.org>
	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>
	<20060108122527.GH18916@xs4all.nl> <43C10704.80404@v.loewis.de>
	<20060108163712.GK18916@xs4all.nl>
	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
	<43C17965.5070408@colorstudy.com>
	<20060108214747.GM18916@xs4all.nl>
	<43C1996D.8060902@colorstudy.com> <43C1A0E2.6030605@v.loewis.de>
Message-ID: <20060109005123.GN18916@xs4all.nl>

On Mon, Jan 09, 2006 at 12:31:46AM +0100, "Martin v. L?wis" wrote:
> Ian Bicking wrote:
> > I just don't want people to feel discouraged when they try to contribute 
> > to the Python community and a PEP 13 could help direct people towards 
> > areas where their contributions are more likely to be useful.  Also I 
> > think it is unfair to use python-list to clarify things that python-dev 
> > is not willing to clarify itself.

> But now: who is going to write it? "Guido should write it" clearly won't
> work. And no, I'm explicitly not volunteering either.

Well, the PEP will be mostly boilerplate anyway (unless there's a sudden
influx of old ideas) so I'm sure I can whip something up before next
weekend. I'll probably keep the actual list of rejected items to implicit
self (which started the whole discussion), upgrading the Python parser
beyond a simple LL(1), and maybe implicit ()'s for functioncalls, if I feel
inspired. The exact wording, and additional rejects, can be discussed at
length by those interested.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From tim.peters at gmail.com  Mon Jan  9 01:59:43 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 8 Jan 2006 19:59:43 -0500
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <20060109005123.GN18916@xs4all.nl>
References: <dpns9t$gil$1@sea.gmane.org> <20060108122527.GH18916@xs4all.nl>
	<43C10704.80404@v.loewis.de> <20060108163712.GK18916@xs4all.nl>
	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
	<43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl>
	<43C1996D.8060902@colorstudy.com> <43C1A0E2.6030605@v.loewis.de>
	<20060109005123.GN18916@xs4all.nl>
Message-ID: <1f7befae0601081659h3567dfcbs1283d3c8ef82dc94@mail.gmail.com>

[Martin]
>> But now: who is going to write it? "Guido should write it" clearly won't
>> work. And no, I'm explicitly not volunteering either.

[Thomas]
> Well, the PEP will be mostly boilerplate anyway (unless there's a sudden
> influx of old ideas) so I'm sure I can whip something up before next
> weekend. I'll probably keep the actual list of rejected items to implicit
> self (which started the whole discussion), upgrading the Python parser
> beyond a simple LL(1), and maybe implicit ()'s for functioncalls, if I feel
> inspired. The exact wording, and additional rejects, can be discussed at
> length by those interested.

I'll help with this.  That means I want to and intend to, and will
discover that I can't actually make any time for it.  With two of us
putting it off indefinitely, it should get done in half the time ;-).

From Scott.Daniels at Acm.Org  Mon Jan  9 02:10:59 2006
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sun, 08 Jan 2006 17:10:59 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <B0689571-E980-44FF-B745-4A1620B1AF0C@fuhm.net>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<43C14997.30309@v.loewis.de>
	<200601081234.59951.fdrake@acm.org>	<43C15370.2070700@v.loewis.de>
	<B0689571-E980-44FF-B745-4A1620B1AF0C@fuhm.net>
Message-ID: <dpsd6i$pnd$1@sea.gmane.org>

James Y Knight wrote:
> On Jan 8, 2006, at 1:01 PM, Martin v. L?wis wrote:
> 
>> Fred L. Drake, Jr. wrote:
>>
>>> I like the way trial (from twisted) supports this.  The test  
>>> method is written
>>> normally, in whatever class makes sense.  Then the test is marked  
>>> with an
>>> attribute to say it isn't expected to pass yet.  When the code is  
>>> fixed and
>>> the test passes, you get that information in trial's output, and  
>>> can unmark
>>> the test.  This avoids having to refactor test classes just to  
>>> update the
>>> status of a test.
>>>
>> So how is the mark added? I would suggest
>>
>>    @xfail
>>    def test_foo(self):
>>      self.assertEquals(0,1)
> 
>    def test_foo(self):
>      self.assertEquals(0,1)
>    test_foo.todo = "Why this test fails description message."

Looks pretty good.  Here's some code to critique:
     import unittest

     class BrokenTest(unittest.TestCase.failureException):
         def __repr__(self):
              return '%s: %s: %s works now' % (
                      (self.__class__.__name__,) + self.args)

     def known_broken_XXX(reason):
         def wrapper(test_method):
             def replacement(*args, **kwargs):
                 try:
                     test_method(*args, **kwargs)
                 except unittest.TestCase.failureException:
                     pass
                 else:
                     raise BrokenTest(test_method.__name__, reason)
             return replacement
         wrapper.todo = reason
         return wrapper

So your use looks like:

     class SomeTests(unittest.TestCase):
         ...
         @known_broken_XXX('Someday the naive expect this to work')
         def test_distinct_ones(self):
             self.failIfEqual(1, 1.0)



-- Scott David Daniels
Scott.Daniels at Acm.Org


From brett at python.org  Mon Jan  9 02:24:26 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 8 Jan 2006 17:24:26 -0800
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <1f7befae0601081659h3567dfcbs1283d3c8ef82dc94@mail.gmail.com>
References: <dpns9t$gil$1@sea.gmane.org> <43C10704.80404@v.loewis.de>
	<20060108163712.GK18916@xs4all.nl>
	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>
	<43C17965.5070408@colorstudy.com> <20060108214747.GM18916@xs4all.nl>
	<43C1996D.8060902@colorstudy.com> <43C1A0E2.6030605@v.loewis.de>
	<20060109005123.GN18916@xs4all.nl>
	<1f7befae0601081659h3567dfcbs1283d3c8ef82dc94@mail.gmail.com>
Message-ID: <bbaeab100601081724t2dd2b058r54ba11303f6d512e@mail.gmail.com>

On 1/8/06, Tim Peters <tim.peters at gmail.com> wrote:
> [Martin]
> >> But now: who is going to write it? "Guido should write it" clearly won't
> >> work. And no, I'm explicitly not volunteering either.
>
> [Thomas]
> > Well, the PEP will be mostly boilerplate anyway (unless there's a sudden
> > influx of old ideas) so I'm sure I can whip something up before next
> > weekend. I'll probably keep the actual list of rejected items to implicit
> > self (which started the whole discussion), upgrading the Python parser
> > beyond a simple LL(1), and maybe implicit ()'s for functioncalls, if I feel
> > inspired. The exact wording, and additional rejects, can be discussed at
> > length by those interested.
>
> I'll help with this.  That means I want to and intend to, and will
> discover that I can't actually make any time for it.  With two of us
> putting it off indefinitely, it should get done in half the time ;-).

Make it a third.  I am willing to help out as needed since I already
keep an eye out for Py3K stuff.  Watching for stuff we don't want to
discuss again is not that much harder.  =)

-Brett

From warner at lothar.com  Mon Jan  9 02:38:10 2006
From: warner at lothar.com (Brian Warner)
Date: Sun, 08 Jan 2006 17:38:10 -0800 (PST)
Subject: [Python-Dev] buildbot
In-Reply-To: <43BD9B32.3030700@v.loewis.de>
References: <20060104.003058.48524852.warner@lothar.com>
	<43BD9B32.3030700@v.loewis.de>
Message-ID: <20060108.173810.91561560.warner@lothar.com>

> From: "Martin v. L?wis" <martin at v.loewis.de>
> 
> So at the moment, I'm quite happy with buildbot. My only wish is that
> it would do static pages, rather than being a web server. I set it up
> as a reverse proxy (so nobody knows what the port number is, but still).

As a bit of history, the buildbot's predecessor used static pages.. it just
dumped everything into HTML files any time something changed, and then I
pointed apache at the directory containing these files. By the time I started
the current Buildbot project, I had become familiar with Twisted, and elected
to use its built-in web server to provide status information. This makes it
much easier to implement things like the "Force Build" button, and allows the
same status-retrieving API to be used by all the different status-delivery
modules: web, IRC, email, PB. On the other hand, it makes it slightly harder
to merge the buildbot's web space into that of another web server (unless
that other web server also happens to be Twisted: the buildbot can run as a
twisted.web.distrib sub-server, which is how the Twisted buildbot presents
status).

The reverse proxy is a great way to improve the integration of the buildbot's
web pages into the rest of your site's. I'm hoping to improve the web-status
plugin to make it easier to change around the page layout from the buildbot
side, but even with that you'll want the reverse-proxy to remove the
port-number from the URL.

I should mention a feature that you might find useful: you can add some query
arguments to the top-level Waterfall page's URL to restrict the set of
Builders that are displayed. If you append "?show=full-2.3", you'll wind up
with a waterfall display that only contains the "full-2.3" Builder's column.
Appending "?show=full-2.3&show=full-2.4" will show two columns, etc. You can
also create multiple Waterfall instances, each with a different set of
"categories", and use that to manage the complexity of displaying status for
lots of Builders. (with the current release these multiple Waterfalls must
all listen on different TCP ports, unfortunately, but you can always hide
this with the reverse-proxy's URL mapping).

So you could have one page that shows *all* the builders, and another set of
pages that only show specific ones (perhaps one primary builder for each
architecture). If the reverse-proxy or some sort of Apache URL-rewriting
allows you to add query parameters to the target URLs, you might be able to
hide these filtered views behind simple-looking status URLs.

It's not the greatest solution, but it might let you add more Builders
without making the display unworkably complex.

cheers,
 -Brian


From ianb at colorstudy.com  Mon Jan  9 02:41:02 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Sun, 08 Jan 2006 19:41:02 -0600
Subject: [Python-Dev] Draft proposal: Implicit self in Python 3.0
In-Reply-To: <43C1A58E.1050200@strakt.com>
References: <611494241.20060106005601@gmail.com>	<20060106193448.GA656@code1.codespeak.net>	<ca471dc20601061319i40aca09em78f3b983c629e9a5@mail.gmail.com>	<dpns9t$gil$1@sea.gmane.org>	<ca471dc20601071712v184245d2tcada68543c5e2f2b@mail.gmail.com>	<20060108122527.GH18916@xs4all.nl>	<43C10704.80404@v.loewis.de>	<20060108163712.GK18916@xs4all.nl>	<1f7befae0601080916n2dc1388et14578ddc33c5563d@mail.gmail.com>	<43C17965.5070408@colorstudy.com>	<20060108214747.GM18916@xs4all.nl>	<43C1996D.8060902@colorstudy.com>
	<43C1A58E.1050200@strakt.com>
Message-ID: <43C1BF2E.1040506@colorstudy.com>

Samuele Pedroni wrote:
> Ian Bicking wrote:
> 
>>I just don't want people to feel discouraged when they try to contribute 
>>to the Python community and a PEP 13 could help direct people towards 
>>areas where their contributions are more likely to be useful. 
> 
> 
> but people have a lot of options, probably more effective, ranging
> from writing great applications in Python, great libraries ... plus 
> implementation work before they are left with the hard option that is 
> language design to contribute to the community.

I completely agree.

>>Also I 
>>think it is unfair to use python-list to clarify things that python-dev 
>>is not willing to clarify itself.
>>
> 
> 
> notice that the intersection is not empty yet.
> 
> Also PEP 1 contains
> 
> """
> PEP authors are responsible for collecting community feedback on a PEP 
> before submitting it for review. A PEP that has not been discussed on 
> python-list at python.org and/or python-dev at python.org will not be 
> accepted. However, wherever possible, long open-ended discussions on 
> public mailing lists should be avoided. Strategies to keep the 
> discussions efficient include: setting up a separate SIG mailing list 
> for the topic, having the PEP author accept private comments in the 
> early design phases, setting up a wiki page, etc. PEP authors should use 
> their discretion here.
> """
> 
> in the past it often happened that water testing, honing
> for a PEP happend on python-list before any long discussion went on
> on python-dev. Basically I think that throwing half-cooked ideas at 
> python-dev, especially for people with no track-record of language 
> design contributions to Python, is just a recipe for frustration.

Sure; lots of PEPs can go to python-list, and it's fine if people ask 
that a PEP that goes to python-dev be discussed on python-list.  But 
only PEPs that have some chance of being accepted.  In fact, these 
things come up on python-list far more often than python-dev, so a PEP 
13 could also help clean those conversations out of that list too.  Or 
at least if people want to discuss those things, they'll be aware that 
the discussion is only academic.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From martin at v.loewis.de  Mon Jan  9 08:23:37 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 08:23:37 +0100
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>	
	<43C0DF8C.7050104@v.loewis.de>	
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>	
	<43C15167.7000702@v.loewis.de>
	<1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>
Message-ID: <43C20F79.4020908@v.loewis.de>

Tim Peters wrote:

> That's no more or less painful than using C99's huge pile of PRId8,
> PRId16, PRId32 (etc, etc) macros, defined there for similar purposes.

Right - and I consider them just as painful.

If you believe that this is really what we should be doing, then, well,
let's do it.

Regards,
Martin


From nnorwitz at gmail.com  Mon Jan  9 08:41:29 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 8 Jan 2006 23:41:29 -0800
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <43C1A147.60409@v.loewis.de>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
Message-ID: <ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>

On 1/8/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Georg Brandl wrote:
> >  >>> "%f" % 1.0
> > '1.000000'
> >  >>> u"%f" % 1.0
> > u'1,000000'
> >
> > Is this intended? This breaks test_format on my box when test_builtin (method
> > test_float_with_comma) is executed first.
>
> No. %-formatting should always use the C locale. One should use
> locale.format to get locale-aware formatting.

I know very little about locale's.  /f assigned me a bug
http://python.org/sf/1391872 which suggests I run all the tests in a
different locale than C.  I think this is a good idea, but when I set
LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without
me calling locale.setlocale().  Shouldn't we maintain the value from
the env't?

The problem I found is in Python/pythonrun.c.  At least I think that's
the problem.  Around line 230-240 we call setlocale(NULL), then call
setlocale("").  Then later on we restore the locale by calling
setlocale() with the return result from setlocale(NULL).

The man page on my box (gentoo) says that passing NULL returns the
current locale() while passing "" sets the locale from the env't
variable.  Should we just capture the results of setlocale("")?

If my understanding is wrong, how can I start python with the correct
locale?  It seems like that I'm confused since this problem hasn't
seemed to come up before.

n

From martin at v.loewis.de  Mon Jan  9 08:48:47 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 08:48:47 +0100
Subject: [Python-Dev] [Python-checkins] r41972
	-	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>	<43C0DF8C.7050104@v.loewis.de>	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>	<1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com>	<43C15307.2060600@v.loewis.de>
	<1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com>
Message-ID: <43C2155F.8010305@v.loewis.de>

Tim Peters wrote:
> We could use the "I" (capital letter eye) length modifier under VC7.1.
>  That's good for both size_t and ptrdiff_t formats under VC7.1, where
> ptrdiff_t under VC7.1 is really the same concept as Py_ssize_t.

ptrdiff_t has the advantage of being available on all platforms,
being part of C89 (IIRC). Should we use ptrdiff_t instead of
Py_ssize_t? Formally, ptrdiff_t could be different from size_t
(in width); reportedly, there are 8086 compilers which had
a 16-bit size_t (maximum size of a segment), but a 32-bit
ptrdiff_t (allowing for cross-segment differences, something
that apparently became undefined in C99).

Regards,
Martin

From fredrik at pythonware.com  Mon Jan  9 08:51:02 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 9 Jan 2006 08:51:02 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
Message-ID: <dpt4l8$f8i$1@sea.gmane.org>

Neal Norwitz wrote:

> > No. %-formatting should always use the C locale. One should use
> > locale.format to get locale-aware formatting.
>
> I know very little about locale's.  /f assigned me a bug
> http://python.org/sf/1391872 which suggests I run all the tests in a
> different locale than C.  I think this is a good idea, but when I set
> LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without
> me calling locale.setlocale().  Shouldn't we maintain the value from
> the env't?

the locale is a global program setting, and should only be "activated"
when it's actually needed.

my rationale for running the tests with a non-US locale was to increase
the chance of catching bugs where the external locale setting affects
Python's behaviour.

maybe it would be a good idea to add a "use setlocale" flag to the test-
runner ?

</F>




From martin at v.loewis.de  Mon Jan  9 08:57:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 08:57:13 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
Message-ID: <43C21759.2090301@v.loewis.de>

Neal Norwitz wrote:
> I know very little about locale's.  /f assigned me a bug
> http://python.org/sf/1391872 which suggests I run all the tests in a
> different locale than C.  I think this is a good idea, but when I set
> LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without
> me calling locale.setlocale().  Shouldn't we maintain the value from
> the env't?

I feel I'm lacking some link here: why do you think we should do that?

> The problem I found is in Python/pythonrun.c.  At least I think that's
> the problem.  Around line 230-240 we call setlocale(NULL), then call
> setlocale("").  Then later on we restore the locale by calling
> setlocale() with the return result from setlocale(NULL).

Right.

> The man page on my box (gentoo) says that passing NULL returns the
> current locale() while passing "" sets the locale from the env't
> variable.  Should we just capture the results of setlocale("")?

Again: why?

> If my understanding is wrong, how can I start python with the correct
> locale?

What does "start with the correct locale" mean? What is the "correct
locale"? Python should work in any locale, and locale.format
should always give the locale-specific formatting, whereas "%f" % value
should always give the result from the "C" locale (regardless of
whether the application has called setlocale or not).

Regards,
Martin


From martin at v.loewis.de  Mon Jan  9 08:58:30 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 08:58:30 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <dpt4l8$f8i$1@sea.gmane.org>
References: <dprpos$us8$1@sea.gmane.org>
	<43C1A147.60409@v.loewis.de>	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
	<dpt4l8$f8i$1@sea.gmane.org>
Message-ID: <43C217A6.1080906@v.loewis.de>

Fredrik Lundh wrote:
> maybe it would be a good idea to add a "use setlocale" flag to the test-
> runner ?

Sure!

Martin

From nnorwitz at gmail.com  Mon Jan  9 09:24:18 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 9 Jan 2006 00:24:18 -0800
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <43C21759.2090301@v.loewis.de>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
	<43C21759.2090301@v.loewis.de>
Message-ID: <ee2a432c0601090024lb725406xe3bd4d2250401442@mail.gmail.com>

On 1/8/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Neal Norwitz wrote:
> > I know very little about locale's.  /f assigned me a bug
> > http://python.org/sf/1391872 which suggests I run all the tests in a
> > different locale than C.  I think this is a good idea, but when I set
> > LANG or LC_ALL or LC_CTYPE in the env't the locale isn't set without
> > me calling locale.setlocale().  Shouldn't we maintain the value from
> > the env't?
>
> I feel I'm lacking some link here: why do you think we should do that?

neal at janus ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python
>>> import locale
>>> locale.setlocale(locale.LC_ALL)
'C'
>>> locale.setlocale(locale.LC_ALL, 'de_DE')
'de_DE'

I would have expected the first call to setlocale() to return de_DE. 
/f solution of adding a flag to the test runner should work for the
bug report I mentioned.  Part of the reason why I asked was because
I'm confused by the current behaviour.  Is the first call to
setlocale() supposed to return C?  If so, always?  I was looking for
some way for the initial call to setlocale() to return de_DE or
whatever other locale I set.

> What does "start with the correct locale" mean? What is the "correct
> locale"? Python should work in any locale,

Hopefully my explanation above answered all those questions.

> and locale.format
> should always give the locale-specific formatting, whereas "%f" % value
> should always give the result from the "C" locale (regardless of
> whether the application has called setlocale or not).

Right.  I'm working on a patch for this problem which is independent
of what I was asking.  I really should have started a new thread, but
I wasn't completely sure when I asked if there was a relationship. 
Also, in Georg's snippet, he didn't show that he was setting the
locale, only what it was set to.

n

From fredrik at pythonware.com  Mon Jan  9 09:54:09 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 9 Jan 2006 09:54:09 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
References: <dprpos$us8$1@sea.gmane.org>
	<43C1A147.60409@v.loewis.de><ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com><43C21759.2090301@v.loewis.de>
	<ee2a432c0601090024lb725406xe3bd4d2250401442@mail.gmail.com>
Message-ID: <dpt8bj$pbj$1@sea.gmane.org>

Neal Norwitz wrote:

> > I feel I'm lacking some link here: why do you think we should do that?
>
> neal at janus ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python
> >>> import locale
> >>> locale.setlocale(locale.LC_ALL)
> 'C'
> >>> locale.setlocale(locale.LC_ALL, 'de_DE')
> 'de_DE'
>
> I would have expected the first call to setlocale() to return de_DE.

the locale is a program-specific setting (handled by the C runtime library),
and is set to "C" by default.

if you want to use a specific locale, you have to call setlocale to indicate
that you really want a non-default locale.  the "" argument is just a con-
venience; it tells the C runtime library to set the locale based on OS-level
language settings (e.g. LANG on Unix, control panel settings on Windows,
etc).

LANG in itself isn't the locale; it's just a way to tell a locale-aware program
what the preferred locale is.

</F>




From nnorwitz at gmail.com  Mon Jan  9 10:12:31 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 9 Jan 2006 01:12:31 -0800
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <43C1A147.60409@v.loewis.de>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
Message-ID: <ee2a432c0601090112g13181a71v36ae5e555c86a03f@mail.gmail.com>

On 1/8/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Georg Brandl wrote:
> >  >>> import locale
> >  >>> locale.setlocale(locale.LC_NUMERIC, "")
> > 'de_DE at euro'
> >  >>> u"%f" % 1.0
> > u'1,000000'
> >
> > Is this intended? This breaks test_format on my box when test_builtin (method
> > test_float_with_comma) is executed first.
>
> No. %-formatting should always use the C locale. One should use
> locale.format to get locale-aware formatting.

http://python.org/sf/1400181 fixes this problem I hope.

n

From mwh at python.net  Mon Jan  9 10:19:46 2006
From: mwh at python.net (Michael Hudson)
Date: Mon, 09 Jan 2006 09:19:46 +0000
Subject: [Python-Dev] test_curses
In-Reply-To: <dprnoe$p98$1@sea.gmane.org> (Georg Brandl's message of "Sun,
	08 Jan 2006 20:04:45 +0100")
References: <dprnoe$p98$1@sea.gmane.org>
Message-ID: <2m1wzhsqq5.fsf@starship.python.net>

Georg Brandl <g.brandl-nospam at gmx.net> writes:

> The call to curses.setupterm() leaves my terminal in a bad state.

Hmm.

> The reset program outputs:
> Erase set to delete.
> Kill set to control-U (^U).
> Interrupt set to control-C (^C).

It always says that :) (unless you've messed with stty, I guess)

> Doesn't the setupterm() have to be paired with something like shutdownterm()?

Not as far as my memory of curses goes.  From the man page:

       The setupterm routine reads in the terminfo database,
       initializing the terminfo structures, but does not set up the
       output virtualization structures used by curses.

What platform are you on?

Cheers,
mwh

-- 
  #ifndef P_tmpdir
  printf( "Go buy a better computer" );
  exit( ETHESKYISFALLINGANDIWANTMYMAMA );
                         -- Dimitri Maziuk on writing secure code, asr

From benji at benjiyork.com  Mon Jan  9 14:02:42 2006
From: benji at benjiyork.com (Benji York)
Date: Mon, 09 Jan 2006 08:02:42 -0500
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <dpt4l8$f8i$1@sea.gmane.org>
References: <dprpos$us8$1@sea.gmane.org>
	<43C1A147.60409@v.loewis.de>	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
	<dpt4l8$f8i$1@sea.gmane.org>
Message-ID: <43C25EF2.4050401@benjiyork.com>

Fredrik Lundh wrote:
> my rationale for running the tests with a non-US locale was to increase
> the chance of catching bugs where the external locale setting affects
> Python's behaviour.
> 
> maybe it would be a good idea to add a "use setlocale" flag to the test-
> runner ?

Sounds like a good use for buildbot.  After running the tests "normally" 
  they could be re-run with a different locale.
--
Benji York

From facundobatista at gmail.com  Mon Jan  9 18:08:15 2006
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 9 Jan 2006 14:08:15 -0300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
Message-ID: <e04bdf310601090908m4633eeb8t@mail.gmail.com>

2006/1/7, Guido van Rossum <guido at python.org>:

> I think it's moot unless you also preserve comments. Ideally would be
> something that prserved everything (ordering, blank lines, comments
> etc.) from how it was read in. Modifying a value should keep its
> position. Adding a value should add it to the end of the section it's
> in (unless there are cases where ordering is important to the
> semantics -- are there?).

Not in my case.

We can rewrite ConfigParser to store everything and write it back in
the exact order it took it, with new values at the end of each section
(or where the user inserted it), but it'll took a big rewrite of the
module: right now it's using dictionaries and should use lists to
achieve that behaviour.

What I wanted to add to the module was predicatibility: a very needed
feature when you're writing test cases (and that's where I got bite).

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From fdrake at acm.org  Mon Jan  9 18:17:50 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon, 9 Jan 2006 12:17:50 -0500
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <e04bdf310601090908m4633eeb8t@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
Message-ID: <200601091217.51071.fdrake@acm.org>

On Monday 09 January 2006 12:08, Facundo Batista wrote:
 > What I wanted to add to the module was predicatibility: a very needed
 > feature when you're writing test cases (and that's where I got bite).

In that case, would sorting the keys within each section be sufficient when 
writing it back out?

I had a class to do the really-careful editing bit like Guido described once, 
but lost it in a disk crash several years ago.  I really wish I'd gotten that 
into revision control somewhere, but it's too late now.  I've not needed that 
badly enough to re-write it.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From tim.peters at gmail.com  Mon Jan  9 18:48:41 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 9 Jan 2006 12:48:41 -0500
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <43C20F79.4020908@v.loewis.de>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<43C15167.7000702@v.loewis.de>
	<1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>
	<43C20F79.4020908@v.loewis.de>
Message-ID: <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com>

...

[Tim]
>> That's no more or less painful than using C99's huge pile of PRId8,
>> PRId16, PRId32 (etc, etc) macros, defined there for similar purposes.

[Martin]
> Right - and I consider them just as painful.
>
> If you believe that this is really what we should be doing, then, well,
> let's do it.

I think it's clear that in a 64-bit world, we absolutely need a way to
format 64-bit integers.  I'm dubious that Neil's specific line of code
needed Py_ssize_t to begin with, but increasing amounts of other code
will.

Maybe this wasn't clear:  I think Neil's code _should_ be able to put
"%zd" in the format string, not using format macros at all.  IOW, I
believe the supporting routines for PyErr_Format() and
PyString_Format() should be taught to recognize the C99-standard "z"
qualifier and implement its semantics in a platform-correct way. 
That's where the irritating macros would be used, _inside_ the
implementations of Py{Err,String}_Format().  Code passing a Py_ssize_t
to one of those would just use "%zd" directly in its format string,
and on all platforms.

It's only code calling printf/fprintf/sprintf directly with a
Py_ssize_t or size_t output value that would need to use the format
macros, and I don't think there's a lot of that -- looks like most
relevant messages are constructed by Py{Err,String}_Format().

From tim.peters at gmail.com  Mon Jan  9 19:14:10 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 9 Jan 2006 13:14:10 -0500
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <43C2155F.8010305@v.loewis.de>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<1f7befae0601080942o3a7dc194ub790b616bb79c2fe@mail.gmail.com>
	<43C15307.2060600@v.loewis.de>
	<1f7befae0601081600g42ea19bdtfac2368cdd51ba51@mail.gmail.com>
	<43C2155F.8010305@v.loewis.de>
Message-ID: <1f7befae0601091014ic0cdebcv2d4ad0983c995aa8@mail.gmail.com>

[Tim]
>> We could use the "I" (capital letter eye) length modifier under VC7.1.
>>  That's good for both size_t and ptrdiff_t formats under VC7.1, where
>> ptrdiff_t under VC7.1 is really the same concept as Py_ssize_t.

[Martin]
> ptrdiff_t has the advantage of being available on all platforms,
> being part of C89 (IIRC). Should we use ptrdiff_t instead of
> Py_ssize_t? Formally, ptrdiff_t could be different from size_t
> (in width); reportedly, there are 8086 compilers which had
> a 16-bit size_t (maximum size of a segment), but a 32-bit
> ptrdiff_t (allowing for cross-segment differences, something
> that apparently became undefined in C99).

I grew up on 60- and 64-bit boxes, but all I've worked on for the last
decade is 32-bit Pentium chips.  If there's a Python platform where
sizeof(size_t) != sizeof(ptrdiff_t), I don't know about it, but I'm
not sure I have a reason to _expect_ to be aware of such things
anymore.

Regardless, I like Py_ssize_t:  it clearly says "same width as size_t,
but signed".  "Difference between two pointers" is obscurely related
to that at best.

From nnorwitz at gmail.com  Mon Jan  9 19:40:44 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 9 Jan 2006 10:40:44 -0800
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<43C15167.7000702@v.loewis.de>
	<1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>
	<43C20F79.4020908@v.loewis.de>
	<1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com>
Message-ID: <ee2a432c0601091040h7c87d9dcj2aaa3174176c6cf2@mail.gmail.com>

On 1/9/06, Tim Peters <tim.peters at gmail.com> wrote:
> ...
>
> [Tim]
> >> That's no more or less painful than using C99's huge pile of PRId8,
> >> PRId16, PRId32 (etc, etc) macros, defined there for similar purposes.
>
> [Martin]
> > Right - and I consider them just as painful.
> >
> > If you believe that this is really what we should be doing, then, well,
> > let's do it.
>
> I think it's clear that in a 64-bit world, we absolutely need a way to
> format 64-bit integers.  I'm dubious that Neil's specific line of code
> needed Py_ssize_t to begin with, but increasing amounts of other code
> will.

I wasn't always sure which choice to make.  I generally tried to leave
the warnings when I wasn't sure.  Like you and Martin I agree this is
a problem, though I don't have any good ideas how to fix.  They all
seem kinda painful.

I often chose to use Py_ssize_t rather than int if it was capturing
the result of a sequence length.  But often this is larger than what
will be allowed (like in this case).  The problem then comes that we
have cases everywhere if we leave things as ints.

Stuff like this:
  len = (int) PyTuple_Size(co_consts);

How do we know that is safe or not?

Also, maybe it would be nice to have defined maximum sizes enforced by
the compiler for lots of fields in code objects, etc.  Like 32k for #
local variables, parameters, free variables, etc.  But when we add
these together in the stack for extras, we would still need to use at
least 32 bits.  And there's still the possibility of overflow when we
add these pieces together.

n

From g.brandl-nospam at gmx.net  Mon Jan  9 19:55:52 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Mon, 09 Jan 2006 19:55:52 +0100
Subject: [Python-Dev] test_curses
In-Reply-To: <2m1wzhsqq5.fsf@starship.python.net>
References: <dprnoe$p98$1@sea.gmane.org> <2m1wzhsqq5.fsf@starship.python.net>
Message-ID: <dpubjo$buj$1@sea.gmane.org>

Michael Hudson wrote:
> Georg Brandl <g.brandl-nospam at gmx.net> writes:
> 
>> The call to curses.setupterm() leaves my terminal in a bad state.
> 
> Hmm.
> 
>> The reset program outputs:
>> Erase set to delete.
>> Kill set to control-U (^U).
>> Interrupt set to control-C (^C).
> 
> It always says that :) (unless you've messed with stty, I guess)

Well, when I do a reset without meddling with the terminal, it says nothing,
at least on my box.

And, there's more: Ctrl+D doesn't work, Ctrl+C doesn't work.

I just looked, my .profile contains "stty sane cr0 pass8 dec".

>> Doesn't the setupterm() have to be paired with something like shutdownterm()?
> 
> Not as far as my memory of curses goes.  From the man page:
> 
>        The setupterm routine reads in the terminfo database,
>        initializing the terminfo structures, but does not set up the
>        output virtualization structures used by curses.
> 
> What platform are you on?

Linux 2.6, ncurses 5.5, TERM=xterm.

Georg


From g.brandl-nospam at gmx.net  Mon Jan  9 20:17:31 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Mon, 09 Jan 2006 20:17:31 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <43C1A147.60409@v.loewis.de>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
Message-ID: <dpucsc$h1n$1@sea.gmane.org>

Martin v. L?wis wrote:
> Georg Brandl wrote:
>>  >>> import locale
>>  >>> locale.setlocale(locale.LC_NUMERIC, "")
>> 'de_DE at euro'
>>  >>> "%f" % 1.0
>> '1.000000'
>>  >>> u"%f" % 1.0
>> u'1,000000'
>>  >>>
>> 
>> 
>> Is this intended? This breaks test_format on my box when test_builtin (method
>> test_float_with_comma) is executed first.
> 
> No. %-formatting should always use the C locale. One should use
> locale.format to get locale-aware formatting.

While we're at it: test_builtin does

             import locale
             orig_locale = locale.setlocale(locale.LC_NUMERIC, '')
             locale.setlocale(locale.LC_NUMERIC, 'fr_FR')

and later

         finally:
             locale.setlocale(locale.LC_NUMERIC, orig_locale)


Shouldn't the first setlocale have no second argument?

regards,
Georg


From shane at hathawaymix.org  Mon Jan  9 21:48:36 2006
From: shane at hathawaymix.org (Shane Hathaway)
Date: Mon, 09 Jan 2006 13:48:36 -0700
Subject: [Python-Dev] Logging enhancements
Message-ID: <43C2CC24.4090405@hathawaymix.org>

I'd like to create a patch for the logging package, but before I do, I 
need to know whether someone else is working on the package and whether 
my patch is likely to be accepted.  Is there another group I should talk to?

Specifically, I want to change the following:

- The logging.config.fileConfig function has bare excepts that hide 
configuration errors.  I'd like to remove the bare excepts and break up 
fileConfig for clarity.

- I'd like to make it possible to configure the format of exception 
tracebacks.  Some logs want compressed tracebacks like Medusa; others 
want extra information like Zope.

- There is no support for external log rotation.  There is plenty of 
support for Python rotating logs directly, but permission settings often 
make that impossible.  Long-running Python processes need to accept a 
signal which notifies them when an external tool rotates the logs.

- The logging documentation is lacking somewhat.  For example, it fails 
to mention that the logging handlers are in the 'handlers' module, which 
I only learned by searching the source code.

Shane

From martin at v.loewis.de  Mon Jan  9 22:56:12 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 22:56:12 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <ee2a432c0601090024lb725406xe3bd4d2250401442@mail.gmail.com>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>	
	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>	
	<43C21759.2090301@v.loewis.de>
	<ee2a432c0601090024lb725406xe3bd4d2250401442@mail.gmail.com>
Message-ID: <43C2DBFC.1010201@v.loewis.de>

Neal Norwitz wrote:
> neal at janus ~/build/python/svn/clean-ish $ LC_ALL=de_DE ./python
> 
>>>>import locale
>>>>locale.setlocale(locale.LC_ALL)
> 
> 'C'
> 
>>>>locale.setlocale(locale.LC_ALL, 'de_DE')
> 
> 'de_DE'
> 
> I would have expected the first call to setlocale() to return de_DE. 

No, it shouldn't. We are providing C semantics here, which is that
no locale functionality is activated unless the application explicitly
asks for it.

> Is the first call to
> setlocale() supposed to return C?  If so, always?

The return value is implementation-defined, but yes, it is supposed
to return the "C" locale (which is synonymous to the "POSIX" locale
on a POSIX system).

> I was looking for
> some way for the initial call to setlocale() to return de_DE or
> whatever other locale I set.

That shouldn't happen.

Regards,
Martin

From martin at v.loewis.de  Mon Jan  9 23:11:03 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 23:11:03 +0100
Subject: [Python-Dev] [Python-checkins] r41972 -
	python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <ee2a432c0601091040h7c87d9dcj2aaa3174176c6cf2@mail.gmail.com>
References: <20060108060915.A69E01E4002@bag.python.org>	
	<43C0DF8C.7050104@v.loewis.de>	
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>	
	<43C15167.7000702@v.loewis.de>	
	<1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>	
	<43C20F79.4020908@v.loewis.de>	
	<1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com>
	<ee2a432c0601091040h7c87d9dcj2aaa3174176c6cf2@mail.gmail.com>
Message-ID: <43C2DF77.1060409@v.loewis.de>

Neal Norwitz wrote:
> I often chose to use Py_ssize_t rather than int if it was capturing
> the result of a sequence length.  But often this is larger than what
> will be allowed (like in this case).  The problem then comes that we
> have cases everywhere if we leave things as ints.

This is also the strategy I have been following. The alternative would
be to add error handling at the places where truncation occurs (unless
you can *really* determine statically that there never ever will be
truncation). This is too painful, so it is IMO better to carry the wide
values throughout, until the place where the range check is carried out.

> Stuff like this:
>   len = (int) PyTuple_Size(co_consts);
> 
> How do we know that is safe or not?

Well, you know that LOAD_CONST only supports 2**31 constants, so
truncation to int is currently safe (I hope that the compiler detects
cases where somebody tries to create more than 2**16 constants).

Still, this is only the *current* state. Going further, it might be
that some of the restrictions are lifted someday, so we would have
to review all casts then to see whether they are still safe.

Therefore, I think having fewer casts is better.

Regards,
Martin

From martin at v.loewis.de  Mon Jan  9 23:19:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 09 Jan 2006 23:19:39 +0100
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <dpucsc$h1n$1@sea.gmane.org>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
	<dpucsc$h1n$1@sea.gmane.org>
Message-ID: <43C2E17B.1050003@v.loewis.de>

Georg Brandl wrote:
>              import locale
>              orig_locale = locale.setlocale(locale.LC_NUMERIC, '')
>              locale.setlocale(locale.LC_NUMERIC, 'fr_FR')
> 
> and later
> 
>          finally:
>              locale.setlocale(locale.LC_NUMERIC, orig_locale)
> 
> 
> Shouldn't the first setlocale have no second argument?

Oops, right - it certainly should.

Martin

From nnorwitz at gmail.com  Tue Jan 10 00:06:04 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 9 Jan 2006 15:06:04 -0800
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <43C2CC24.4090405@hathawaymix.org>
References: <43C2CC24.4090405@hathawaymix.org>
Message-ID: <ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>

On 1/9/06, Shane Hathaway <shane at hathawaymix.org> wrote:
> I'd like to create a patch for the logging package, but before I do, I
> need to know whether someone else is working on the package and whether
> my patch is likely to be accepted.  Is there another group I should talk to?

Vinay (copied) maintains it separately.  He's the best person to talk
to AFAIK.  He maintains compatability with older pythons, possibly
1.5, I don't recall.  Check PEP 291.

> Specifically, I want to change the following:

There are several bugs (any maybe patches) on SF related to logging. 
AFAIK, they are all assigned to Vinay.  It would be great if you could
fix those as well.

n
--
> - The logging.config.fileConfig function has bare excepts that hide
> configuration errors.  I'd like to remove the bare excepts and break up
> fileConfig for clarity.
>
> - I'd like to make it possible to configure the format of exception
> tracebacks.  Some logs want compressed tracebacks like Medusa; others
> want extra information like Zope.
>
> - There is no support for external log rotation.  There is plenty of
> support for Python rotating logs directly, but permission settings often
> make that impossible.  Long-running Python processes need to accept a
> signal which notifies them when an external tool rotates the logs.
>
> - The logging documentation is lacking somewhat.  For example, it fails
> to mention that the logging handlers are in the 'handlers' module, which
> I only learned by searching the source code.
>
> Shane

From facundobatista at gmail.com  Tue Jan 10 00:11:43 2006
From: facundobatista at gmail.com (Facundo Batista)
Date: Mon, 9 Jan 2006 20:11:43 -0300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <200601091217.51071.fdrake@acm.org>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
	<200601091217.51071.fdrake@acm.org>
Message-ID: <e04bdf310601091511wb10749ds@mail.gmail.com>

2006/1/9, Fred L. Drake, Jr. <fdrake at acm.org>:

> On Monday 09 January 2006 12:08, Facundo Batista wrote:
>  > What I wanted to add to the module was predicatibility: a very needed
>  > feature when you're writing test cases (and that's where I got bite).
>
> In that case, would sorting the keys within each section be sufficient when
> writing it back out?

Yes, because giving a set of sections and values inside them, you know
in advance how they'll finish in the file.


> I had a class to do the really-careful editing bit like Guido described once,
> but lost it in a disk crash several years ago.  I really wish I'd gotten that
> into revision control somewhere, but it's too late now.  I've not needed that
> badly enough to re-write it.

Somebody (me, for example) can write it if we all decide that that's
the desired behaviour. We just need to take a side.

Me, I just need the predicatibility that sort() gives.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From nnorwitz at gmail.com  Tue Jan 10 00:14:48 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 9 Jan 2006 15:14:48 -0800
Subject: [Python-Dev] locale and LC_NUMERIC
In-Reply-To: <43C2DBFC.1010201@v.loewis.de>
References: <dprpos$us8$1@sea.gmane.org> <43C1A147.60409@v.loewis.de>
	<ee2a432c0601082341x23755e80n664a6c742dab657a@mail.gmail.com>
	<43C21759.2090301@v.loewis.de>
	<ee2a432c0601090024lb725406xe3bd4d2250401442@mail.gmail.com>
	<43C2DBFC.1010201@v.loewis.de>
Message-ID: <ee2a432c0601091514v17f361ebra9319b4c67fb1234@mail.gmail.com>

On 1/9/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> >
> > I would have expected the first call to setlocale() to return de_DE.
>
> No, it shouldn't. We are providing C semantics here, which is that
> no locale functionality is activated unless the application explicitly
> asks for it.

Thanks (to /f too).  That was the part that I was missing.  It makes sense now.

I made the patch to do this.  It's called before each class' test run
in regrtest.py.  I would prefer to set the locale before each test
method run, but I need to muck with unittest for that IIRC.

There were 2 failures.  One was in test_email which needs the C locale
for generating the proper timestamp.  The other was (IIRC) in test_re,
2 methods failed.  So we are hopefully in decent shape.

n

From shane at hathawaymix.org  Tue Jan 10 00:29:38 2006
From: shane at hathawaymix.org (Shane Hathaway)
Date: Mon, 09 Jan 2006 16:29:38 -0700
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
References: <43C2CC24.4090405@hathawaymix.org>
	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
Message-ID: <43C2F1E2.8030102@hathawaymix.org>

Neal Norwitz wrote:
> On 1/9/06, Shane Hathaway <shane at hathawaymix.org> wrote:
> 
>>I'd like to create a patch for the logging package, but before I do, I
>>need to know whether someone else is working on the package and whether
>>my patch is likely to be accepted.  Is there another group I should talk to?
> 
> 
> Vinay (copied) maintains it separately.  He's the best person to talk
> to AFAIK.  He maintains compatability with older pythons, possibly
> 1.5, I don't recall.  Check PEP 291.
> 
> 
>>Specifically, I want to change the following:
> 
> 
> There are several bugs (any maybe patches) on SF related to logging. 
> AFAIK, they are all assigned to Vinay.  It would be great if you could
> fix those as well.

What is the time frame for Python 2.5?  If I finish soon and the changes 
are not disruptive, do I have enough time for my changes to make it into 
2.5?

Shane

From aahz at pythoncraft.com  Tue Jan 10 00:42:32 2006
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 9 Jan 2006 15:42:32 -0800
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <43C2F1E2.8030102@hathawaymix.org>
References: <43C2CC24.4090405@hathawaymix.org>
	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
	<43C2F1E2.8030102@hathawaymix.org>
Message-ID: <20060109234231.GA7090@panix.com>

On Mon, Jan 09, 2006, Shane Hathaway wrote:
>
> What is the time frame for Python 2.5?  If I finish soon and the changes 
> are not disruptive, do I have enough time for my changes to make it into 
> 2.5?

There is still no official timeframe for 2.5.  Alpha is hoped for
March/April; final release for June/July.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From tim.peters at gmail.com  Tue Jan 10 00:45:15 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 9 Jan 2006 18:45:15 -0500
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <43C2F1E2.8030102@hathawaymix.org>
References: <43C2CC24.4090405@hathawaymix.org>
	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
	<43C2F1E2.8030102@hathawaymix.org>
Message-ID: <1f7befae0601091545u13e7c3c7ha852ff439d2f20e5@mail.gmail.com>

[Shane Hathaway]
> What is the time frame for Python 2.5?  If I finish soon and the changes
> are not disruptive, do I have enough time for my changes to make it into
> 2.5?

Yup!  2.5 hasn't even had an alpha release yet, and I don't expect
2.5a1 before March:

    http://mail.python.org/pipermail/python-dev/2005-August/055342.html

From shane at hathawaymix.org  Tue Jan 10 00:50:17 2006
From: shane at hathawaymix.org (Shane Hathaway)
Date: Mon, 09 Jan 2006 16:50:17 -0700
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <1f7befae0601091545u13e7c3c7ha852ff439d2f20e5@mail.gmail.com>
References: <43C2CC24.4090405@hathawaymix.org>	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>	<43C2F1E2.8030102@hathawaymix.org>
	<1f7befae0601091545u13e7c3c7ha852ff439d2f20e5@mail.gmail.com>
Message-ID: <43C2F6B9.1060108@hathawaymix.org>

Tim Peters wrote:
> [Shane Hathaway]
> 
>>What is the time frame for Python 2.5?  If I finish soon and the changes
>>are not disruptive, do I have enough time for my changes to make it into
>>2.5?
> 
> 
> Yup!  2.5 hasn't even had an alpha release yet, and I don't expect
> 2.5a1 before March:
> 
>     http://mail.python.org/pipermail/python-dev/2005-August/055342.html

That's good news.  Thanks.

Shane

From vinay_sajip at red-dove.com  Tue Jan 10 01:01:31 2006
From: vinay_sajip at red-dove.com (Vinay Sajip)
Date: Tue, 10 Jan 2006 00:01:31 -0000
Subject: [Python-Dev] Logging enhancements
References: <43C2CC24.4090405@hathawaymix.org>
	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
Message-ID: <001001c61579$05fb01c0$0200a8c0@alpha>

Hi Shane,

On 1/9/06, Shane Hathaway <shane at hathawaymix.org> wrote:
> I'd like to create a patch for the logging package, but before I do, I
> need to know whether someone else is working on the package and whether
> my patch is likely to be accepted.  Is there another group I should talk
to?

I maintain the logging package, and I will happily review patches and apply
them where I think they're fine. I have accepted many patches to the
package, some via SF and some sent privately, but also rejected a few. For
example, the TimedRotatingFileHandler was submitted by someone.

Patches need to be compatible with Python 1.5.2.

> - The logging.config.fileConfig function has bare excepts that hide
> configuration errors.  I'd like to remove the bare excepts and break up
> fileConfig for clarity.

I've no problem with this in principle, though there have been cases where
ConfigParser behaviour is not what you would expect (e.g. a non-existent
file leads to a "missing section" exception). I'd be grateful for more
specifics as to where you have encountered problems.

> - I'd like to make it possible to configure the format of exception
> tracebacks.  Some logs want compressed tracebacks like Medusa; others
> want extra information like Zope.

Again, no problem in principle, but it's probably an unusual use case so any
change should not require extra work for users without specialised
requirements.

> - There is no support for external log rotation.  There is plenty of
> support for Python rotating logs directly, but permission settings often
> make that impossible.  Long-running Python processes need to accept a
> signal which notifies them when an external tool rotates the logs.

I'd be interested in your approach to solve this.

> - The logging documentation is lacking somewhat.  For example, it fails
> to mention that the logging handlers are in the 'handlers' module, which
> I only learned by searching the source code.

Not true for this specific case - for example,
http://docs.python.org/lib/module-logging.html has a list of 12 handlers,
followed by a statement to the effect that all but StreamHandler and
FileHandler are defined in the handlers module.

In general, patches are gratefully received! If you don't want to invest too
much time up front, send me a mail indicating your general approach and
we'll discuss.

Best regards,

Vinay Sajip


From shane at hathawaymix.org  Tue Jan 10 01:29:48 2006
From: shane at hathawaymix.org (Shane Hathaway)
Date: Mon, 09 Jan 2006 17:29:48 -0700
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <001001c61579$05fb01c0$0200a8c0@alpha>
References: <43C2CC24.4090405@hathawaymix.org>	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
	<001001c61579$05fb01c0$0200a8c0@alpha>
Message-ID: <43C2FFFC.9000202@hathawaymix.org>

Vinay Sajip wrote:
> On 1/9/06, Shane Hathaway <shane at hathawaymix.org> wrote:
>>- The logging.config.fileConfig function has bare excepts that hide
>>configuration errors.  I'd like to remove the bare excepts and break up
>>fileConfig for clarity.
> 
> I've no problem with this in principle, though there have been cases where
> ConfigParser behaviour is not what you would expect (e.g. a non-existent
> file leads to a "missing section" exception). I'd be grateful for more
> specifics as to where you have encountered problems.

I wrote a config file with an invalid handler section.  Because of the 
bare except, not only was my error not reported anywhere, but the daemon 
started with no logging at all.  It took me a while to realize what had 
happened.  The logging package needs to report erroneous configurations.

>>- I'd like to make it possible to configure the format of exception
>>tracebacks.  Some logs want compressed tracebacks like Medusa; others
>>want extra information like Zope.
> 
> Again, no problem in principle, but it's probably an unusual use case so any
> change should not require extra work for users without specialised
> requirements.

Agreed.

>>- There is no support for external log rotation.  There is plenty of
>>support for Python rotating logs directly, but permission settings often
>>make that impossible.  Long-running Python processes need to accept a
>>signal which notifies them when an external tool rotates the logs.
> 
> I'd be interested in your approach to solve this.

To be compatible with an external tool like logrotate, all a daemon 
needs to do is close and reopen log files at about the same time log 
rotation happens.  To handle this use case, I suggest the logging module 
needs a function named reopen(), which daemons can call upon receipt of 
a signal.  The reopen function will find all handlers with open files 
and ask them to reopen.

>>- The logging documentation is lacking somewhat.  For example, it fails
>>to mention that the logging handlers are in the 'handlers' module, which
>>I only learned by searching the source code.
> 
> 
> Not true for this specific case - for example,
> http://docs.python.org/lib/module-logging.html has a list of 12 handlers,
> followed by a statement to the effect that all but StreamHandler and
> FileHandler are defined in the handlers module.

Oops.  I guess I skipped over that because I was using the documentation 
as a reference rather than as a tutorial.  I suggest the documentation 
needs to become a better reference.

> In general, patches are gratefully received! If you don't want to invest too
> much time up front, send me a mail indicating your general approach and
> we'll discuss.

I'll get started soon.  I just wanted to be sure I'm not duplicating 
someone else's efforts.

Shane

From barry at python.org  Tue Jan 10 01:37:19 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 09 Jan 2006 19:37:19 -0500
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <001001c61579$05fb01c0$0200a8c0@alpha>
References: <43C2CC24.4090405@hathawaymix.org>
	<ee2a432c0601091506o204aaf5dyf320cd2b2895ab4@mail.gmail.com>
	<001001c61579$05fb01c0$0200a8c0@alpha>
Message-ID: <1136853439.1035.110.camel@geddy.wooz.org>

On Tue, 2006-01-10 at 00:01 +0000, Vinay Sajip wrote:

> I maintain the logging package, and I will happily review patches and apply
> them where I think they're fine. I have accepted many patches to the
> package, some via SF and some sent privately, but also rejected a few. For
> example, the TimedRotatingFileHandler was submitted by someone.
> 
> Patches need to be compatible with Python 1.5.2.

I've been thinking about this compatibility thing too, especially w.r.t.
the email package.  For example, I still want to maintain email 2.5
which has b/c to Python 2.1.  But email 3.0 need only be compatible with
Python 2.3.  Given that we now have everything in Subversion, I think it
makes it much easier to maintain multiple versions of separately
distributed packages, modulo the time it takes to do so.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060109/e481130b/attachment.pgp 

From martin at v.loewis.de  Tue Jan 10 09:15:56 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 09:15:56 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <20060108.173810.91561560.warner@lothar.com>
References: <20060104.003058.48524852.warner@lothar.com>	<43BD9B32.3030700@v.loewis.de>
	<20060108.173810.91561560.warner@lothar.com>
Message-ID: <43C36D3C.7050202@v.loewis.de>

Brian Warner wrote:
> This makes it much easier to implement things like the "Force Build" 
> button, and allows the same status-retrieving API to be used by all 
> the different status-delivery modules: web, IRC, email, PB.

Certainly: I can easily believe that this is easier from a development
point of view.

I can also accept that this is the only way if you want to trigger
things through the Web (such as the Force Build interface). However,
this interface was the first thing I turned off, because there is
too much potential for abuse. The slaves are (in our case) donated
resources, sometimes in machines that also serve other purposes.
The donors need to know what precisely will run on their machines,
and when precisely that will happen, so "Force build" is disabled.

The reason I want static pages is for security concerns. It is not
easy whether buildbot can be trusted to have no security flaws,
which might allow people to start new processes on the master,
or (perhaps worse) on any of the slaves.

I know I could limit the Twisted webserver to localhost using
firewalling/iptables (and I will need to if there is no other
option); just having it generate static pages would have been
more convenient.

BTW, the same security concern applies to the IRC listener
and the slave listeners: how do I know that nobody can take
over my infrastructure using carefully-crafted messages?

I could study the entire protocol stack to convince myself,
but I would prefer if there was an easy answer, like:
this is the set of operations callable through twisted,
and these are the preconditions for calling them.
For the IRC, this is easy to see (just inspect all
command_ procedures), but what about the slaves port?

> It's not the greatest solution, but it might let you add more 
> Builders without making the display unworkably complex.

Thanks! so far, we only have 8 lanes, which is still managable. If size
increases, I come back to these ideas.

Regards,
Martin

From mhammond at skippinet.com.au  Tue Jan 10 09:10:06 2006
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Tue, 10 Jan 2006 19:10:06 +1100
Subject: [Python-Dev] Logging enhancements
In-Reply-To: <43C2FFFC.9000202@hathawaymix.org>
Message-ID: <DAELJHBGPBHPJKEBGGLNMEELJLAD.mhammond@skippinet.com.au>

Shane:
> Vinay Sajip wrote:

> > I'd be interested in your approach to solve this.
>
> To be compatible with an external tool like logrotate, all a daemon
> needs to do is close and reopen log files at about the same time log
> rotation happens.  To handle this use case, I suggest the logging module
> needs a function named reopen(), which daemons can call upon receipt of
> a signal.  The reopen function will find all handlers with open files
> and ask them to reopen.

FWIW, such an operation is generally useless on a Windows system, as a file
can not be renamed while it is in use.

zope-dev went through this discussion recently, and in the interests of
still allowing an external tool (similar to logrotate) to have the rotation
logic instead of Python or Zope, we came up with a slightly different scheme
for Windows; when requested, Zope will temporarily close the file and rename
it to the original name + ".last" (ignoring any errors.)  It then reopens
the original name.  In most cases, this will leave a .last file around which
can be processed by an external tool, but is also safe in the face of
failure.

IMO, we should consider implementing something like the above; it is still
preferable to have a standard solution for Windows, even if different than
other platforms, rather than requiring everyone to solve this problem
themselves.  I'd even suggest sticking with the name 'reopen()' is still OK,
noting the differences in the documentation, but that is a secondary issue.

Cheers,

Mark


From andrew-pythondev at puzzling.org  Tue Jan 10 09:49:55 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue, 10 Jan 2006 19:49:55 +1100
Subject: [Python-Dev] buildbot
In-Reply-To: <43C36D3C.7050202@v.loewis.de>
References: <20060104.003058.48524852.warner@lothar.com>
	<43BD9B32.3030700@v.loewis.de>
	<20060108.173810.91561560.warner@lothar.com>
	<43C36D3C.7050202@v.loewis.de>
Message-ID: <20060110084955.GF10373@home.puzzling.org>

On Tue, Jan 10, 2006 at 09:15:56AM +0100, "Martin v. L?wis" wrote:
[...]
> 
> I know I could limit the Twisted webserver to localhost using
> firewalling/iptables (and I will need to if there is no other
> option); just having it generate static pages would have been
> more convenient.

For this part at least, it's easier than that; you can pass
interface='127.0.0.1' to the relevant listenTCP or TCPServer call, so that it
will only bind that port to localhost.

-Andrew.


From mal at egenix.com  Tue Jan 10 10:18:58 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 10 Jan 2006 10:18:58 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43BD8464.4060801@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>
	<43BD8464.4060801@v.loewis.de>
Message-ID: <43C37C02.8030005@egenix.com>

Martin v. L?wis wrote:
> Armin Rigo wrote:
>> This would do the right thing for <= 2.4, using ints everywhere; and the
>> Python.h version 2.5 would detect the #define and assume it's a
>> 2.5-compatible module, so it would override the #define with the real
>> thing *and* turn on the ssize_t interpretation of the '#' format
>> character.
> 
> This would be very similar to the PY_SIZE_T_CLEAN approach, except
> that it would also help to detect spelling mistakes.
> 
>>From an implementation point of view, the real challenge is to
> give PyArg_ParseTuple a different meaning; I do this be #defining
> it to PyArg_ParseTupleSsize_t (to preserve binary compatibility
> for the original interpretation of ParseTuple). Putting
> additional flags arguments in the entire code is also quite
> hackish.
> 
>> I still don't like the idea of a magic #define that changes the behavior
>> of '#include <Python.h>', but I admit I don't find any better solution.
>> I suppose I'll just blame C.
> 
> More precisely, the printf style of function calling, and varargs
> functions. ISO C is pretty type safe, but with varargs functions,
> you lose that completely.
> 
> I still hope I can write a C parser some day that does
> ParseTuple/BuildValue checking.

At least gcc does check the parameter types and generates
warnings for wrong vararg parameters passed to printf() et al.

We definitely need a clean solution for this. Otherwise,
authors who want to support more than just the latest Python
release will run into deep trouble.

Note that the branch also has other changes of output parameter
types which will cause problems in extensions (not only extensions
implementing the sequence protocol as the PEP suggests, but
also ones using it as well as extensions implementing or
using the buffer protocol and the slicing protocol). These are not
as easily avoidable as the PyArg_ParseTuple() problem which
could be dealt with by a a new parser marker for ssize_t
lengths (with the '#' marker casting the argument to an int).

I don't see a good solution for these other than introducing
a set of new APIs (and maybe doing some macro magic as Martin
did for PyArg_ParseTuple()). Due to the fact that changes in
the types of output parameters require changes in the
extension variable type layout itself, they introduce a large
number of type changes in the extension and make writing
backwards compatible extensions harder than necessary.

Furthermore, all extensions for Python 2.4 would have to be
ported to the new Python API and porting is not going to
be a simple recompile, but will require C skills.

We'd also have to make sure that old extensions don't
just import with a warning, since the change will introduce
buffer overflows and seg faults in extensions that are not
aware of the change.

Martin, please add the above points to the PEP. I'd also
like to see it published, because it's hard to track a PEP
in the mailing

Thanks,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mwh at python.net  Tue Jan 10 10:28:53 2006
From: mwh at python.net (Michael Hudson)
Date: Tue, 10 Jan 2006 09:28:53 +0000
Subject: [Python-Dev] [Python-checkins] r41972 -
 python/branches/ssize_t/Objects/funcobject.c
In-Reply-To: <43C2DF77.1060409@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Mon,
	09 Jan 2006 23:11:03 +0100")
References: <20060108060915.A69E01E4002@bag.python.org>
	<43C0DF8C.7050104@v.loewis.de>
	<1f7befae0601080842s24c98447k92484e6ad141569f@mail.gmail.com>
	<43C15167.7000702@v.loewis.de>
	<1f7befae0601081547h50cbce19w328e30aa6b1e1f5a@mail.gmail.com>
	<43C20F79.4020908@v.loewis.de>
	<1f7befae0601090948k47f782baq1e46ed6306be778e@mail.gmail.com>
	<ee2a432c0601091040h7c87d9dcj2aaa3174176c6cf2@mail.gmail.com>
	<43C2DF77.1060409@v.loewis.de>
Message-ID: <2m64osqvmy.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Well, you know that LOAD_CONST only supports 2**31 constants, so
> truncation to int is currently safe (I hope that the compiler detects
> cases where somebody tries to create more than 2**16 constants).

Easy enough to check:

   >>> eval(repr(range(100000)))

(It works).

Cheers,
mwh

-- 
  Q: What are 1000 lawyers at the bottom of the ocean?
  A: A good start.
  (A lawyer told me this joke.)
                                  -- Michael Str?der, comp.lang.python

From mwh at python.net  Tue Jan 10 10:32:22 2006
From: mwh at python.net (Michael Hudson)
Date: Tue, 10 Jan 2006 09:32:22 +0000
Subject: [Python-Dev] test_curses
In-Reply-To: <dpubjo$buj$1@sea.gmane.org> (Georg Brandl's message of "Mon,
	09 Jan 2006 19:55:52 +0100")
References: <dprnoe$p98$1@sea.gmane.org> <2m1wzhsqq5.fsf@starship.python.net>
	<dpubjo$buj$1@sea.gmane.org>
Message-ID: <2m1wzgqvh5.fsf@starship.python.net>

Georg Brandl <g.brandl-nospam at gmx.net> writes:

> Michael Hudson wrote:
>> Georg Brandl <g.brandl-nospam at gmx.net> writes:
>> 
>>> The call to curses.setupterm() leaves my terminal in a bad state.
>> 
>> Hmm.
>> 
>>> The reset program outputs:
>>> Erase set to delete.
>>> Kill set to control-U (^U).
>>> Interrupt set to control-C (^C).
>> 
>> It always says that :) (unless you've messed with stty, I guess)
>
> Well, when I do a reset without meddling with the terminal, it says nothing,
> at least on my box.

Oh.  Maybe I'm out of date.

> And, there's more: Ctrl+D doesn't work, Ctrl+C doesn't work.

Right, so it definitely sounds like the terminal is in raw mode.

>>> Doesn't the setupterm() have to be paired with something like shutdownterm()?
>> 
>> Not as far as my memory of curses goes.  From the man page:
>> 
>>        The setupterm routine reads in the terminfo database,
>>        initializing the terminfo structures, but does not set up the
>>        output virtualization structures used by curses.
>> 
>> What platform are you on?
>
> Linux 2.6, ncurses 5.5, TERM=xterm.

Well, this still has the faint whiff of impossibility about it.  Are
you sure it's setupterm() that's doing the damage?  Can you reproduce
interactively?

Cheers,
mwh

-- 
  You can remove divmod() when I'm dead.  Before then, it stays.
  I'm sure all will agree that's a reasonable compromise.
                               -- Tim Peters negotiating on python-dev

From skip at pobox.com  Tue Jan 10 15:46:22 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 10 Jan 2006 08:46:22 -0600
Subject: [Python-Dev] sudo security hole w/ potential Python connection
Message-ID: <17347.51390.16470.505543@montanaro.dyndns.org>


Got this from a Google alert overnight.  It's not really a Python problem
(it's a sudo problem), but it's probably not a bad idea to understand the
implications.

    >> SUDO Python Environment Cleaning Privilege Escalation ...
    >> Secunia - UK
    >> ... This can be exploited by a user with sudo access to a python script
    >> to gain access to an interactive python prompt via the "PYTHONINSPECT"
    >> environment variable ...
    >> <http://secunia.com/advisories/18358/>

Skip

From munna_tank at yahoo.co.in  Tue Jan 10 16:51:12 2006
From: munna_tank at yahoo.co.in (RASHMI TANK)
Date: Tue, 10 Jan 2006 15:51:12 +0000 (GMT)
Subject: [Python-Dev] Python-Dev Digest, Vol 30, Issue 32
In-Reply-To: <mailman.1206.1136885545.27774.python-dev@python.org>
Message-ID: <20060110155112.92853.qmail@web8514.mail.in.yahoo.com>



please stop email upto further request.
  thanking you

Send instant messages to your online friends http://in.messenger.yahoo.com 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060110/9aac8e8e/attachment.htm 

From jorodrig at sfwmd.gov  Tue Jan 10 17:35:45 2006
From: jorodrig at sfwmd.gov (Rodrigues, Joseph)
Date: Tue, 10 Jan 2006 11:35:45 -0500
Subject: [Python-Dev] Tkinter
Message-ID: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>

Hello,

I would like to know if Tkinter is being developed/maintained.  It
appears the latest book on Tkinter by John E. Grayson published in 2000
is the only book resource on Tkinter.

 

The publisher suggested that QtPy
(http://www.riverbankcomputing.co.uk/pyqt/)  and wxPython (
http://www.wxpython.org/ ) have replaced Tkinter.

 

Could you provide an update on Tkinter

 

Joseph

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060110/776eac80/attachment.html 

From anothermax at gmail.com  Mon Jan  9 20:40:00 2006
From: anothermax at gmail.com (Jeremy Maxfield)
Date: Mon, 9 Jan 2006 20:40:00 +0100
Subject: [Python-Dev] Sharing between multiple interpreters and
	restricted mode
In-Reply-To: <43BC0933.2060705@core-sdi.com>
References: <43BC0933.2060705@core-sdi.com>
Message-ID: <93dc9c320601091140h3338b085rb739f39e98ea1213@mail.gmail.com>

Hi Gabriel,

Apart from the problem of sharing code between interpreters (sounds
pretty dangerous to me) there's a bug in 2.4.1 (and 2.3.5 for that
matter) that causes problems with multiple interpreters in certain
siutations (esp. threads) which results in this 'restricted mode'
message - I encountered it when trying to do some work with JEP (Java
Embedded Python).
Python 2.4.2 fixed this problem for me.
I would suggest giving  2.4.2 a try too.

Good luck,
Max

On 1/4/06, Gabriel Becedillas <lists at core-sdi.com> wrote:
> Hi,
> At the company I work for we've embedded Python 2.4.1 in a C++
> application. We execute multiple scripts concurrenlty, each one in its
> own interpreter (created using Py_NewInterpreter()).
> We are sharing a certain instance between interpreters because its to
> expensive to instantiate that class every time an interpreter is
> created. The class is instantiated in the main interpreter (that is
> always alive) and every time a new interpreter is created, that
> instance is added to the interpreter's __builtins__ dict.
> The problem I'm having is that some methods of that class (when
> executed in an interpreter different from the one it was created in)
> complain about being in restricted execution mode.
> Assuming that there are no issues with this instance lifetime (coz it
> will always be referenced by the main interpreter), is there a safe way
> to do some sharing between interpreters ?.
> Thanks.
>
> --
>
> Gabriel Becedillas
> Developer
> CORE SECURITY TECHNOLOGIES
>
> Florida 141 - 2? cuerpo - 7? piso
> C1005AAC Buenos Aires - Argentina
> Tel/Fax: (54 11) 5032-CORE (2673)
> http://www.corest.com
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/anothermax%40gmail.com
>

From aahz at pythoncraft.com  Tue Jan 10 17:54:23 2006
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 10 Jan 2006 08:54:23 -0800
Subject: [Python-Dev] Tkinter
In-Reply-To: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
Message-ID: <20060110165423.GA10810@panix.com>

On Tue, Jan 10, 2006, Rodrigues, Joseph wrote:
>
> I would like to know if Tkinter is being developed/maintained.  It
> appears the latest book on Tkinter by John E. Grayson published in 2000
> is the only book resource on Tkinter.

This message is not appropriate for python-dev.  Please use
comp.lang.python (or python-list).
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From warner at lothar.com  Tue Jan 10 19:07:46 2006
From: warner at lothar.com (Brian Warner)
Date: Tue, 10 Jan 2006 10:07:46 -0800 (PST)
Subject: [Python-Dev] buildbot
In-Reply-To: <20060110084955.GF10373@home.puzzling.org>
References: <20060108.173810.91561560.warner@lothar.com>
	<43C36D3C.7050202@v.loewis.de>
	<20060110084955.GF10373@home.puzzling.org>
Message-ID: <20060110.100746.13886364.warner@lothar.com>

> For this part at least, it's easier than that; you can pass
> interface='127.0.0.1' to the relevant listenTCP or TCPServer call, so that it
> will only bind that port to localhost.

And I intend to add an option to the buildbot master.cfg file to make it easy
to do just that. It should be in the next release.

 -Brian (running busy today, hoping to respond to the rest tonight)

From guido at python.org  Tue Jan 10 19:10:50 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 10:10:50 -0800
Subject: [Python-Dev] Tkinter
In-Reply-To: <20060110165423.GA10810@panix.com>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
	<20060110165423.GA10810@panix.com>
Message-ID: <ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>

On 1/10/06, Aahz <aahz at pythoncraft.com> wrote:
> On Tue, Jan 10, 2006, Rodrigues, Joseph wrote:
> > I would like to know if Tkinter is being developed/maintained.  It
> > appears the latest book on Tkinter by John E. Grayson published in 2000
> > is the only book resource on Tkinter.
>
> This message is not appropriate for python-dev.  Please use
> comp.lang.python (or python-list).

But for sure, Tkinter is still being maintained by the Python
developers (mainly MvL I believe). If it appears stagnant that's
probably because Tcl/Tk itself isn't changing much.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From warner at lothar.com  Tue Jan 10 19:16:11 2006
From: warner at lothar.com (Brian Warner)
Date: Tue, 10 Jan 2006 10:16:11 -0800 (PST)
Subject: [Python-Dev] [Buildbot-devel] Re:  buildbot
In-Reply-To: <EA8E416C-607E-4D96-8789-3C323C4660DD@mac.com>
References: <20060108.173810.91561560.warner@lothar.com>
	<43C36D3C.7050202@v.loewis.de>
	<EA8E416C-607E-4D96-8789-3C323C4660DD@mac.com>
Message-ID: <20060110.101611.23734793.warner@lothar.com>

> I have security concerns as well, but not in buildbot itself.  My  
> project is restricted even withinz the company I work for so I need  
> the buildbot web server to only be available to certain people.   
> HTTPS access would be nice too.  TwistedWeb doesn't seem to have  
> support for either HTTPS or authentication so I've been forced to  
> "hide" it by putting it on a non-standard port.  Very weak.

Actually twisted-web provides pretty good support for both SSL and
password-based authentication, but my own web skills are too weak to really
figure out how to take advantage of them. I've been hoping to address this in
the "web status page cleanup" item on the TODO list, but other bugs and
features keep getting in the way.

There are a couple of examples of setting up an HTTPS website in the twisted
docs, in case you're interested. I think the changes would only have to go
into Waterfall.setup, probably just replacing the internet.TCPServer() call
to something involving SSL.

The authentication stuff is called "guard", and I think it would involve
inserting an additional password-checking resource between the top-level
server.Site and the primary html.StatusResource object. But, as I said, my
twisted-web-foo is not strong, so I could be completely wrong about this.

cheers,
 -Brian

From guido at python.org  Tue Jan 10 19:31:16 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 10:31:16 -0800
Subject: [Python-Dev] sudo security hole w/ potential Python connection
In-Reply-To: <17347.51390.16470.505543@montanaro.dyndns.org>
References: <17347.51390.16470.505543@montanaro.dyndns.org>
Message-ID: <ca471dc20601101031h5c7a3189vfb22709203c7a78@mail.gmail.com>

Methinks anyone using sudo to allow non-root-users to execute specific
scripts without giving them full root perms is relying on security by
obscurity at this point. (Ditto for setuid Python scripts BTW.)

--Guido

On 1/10/06, skip at pobox.com <skip at pobox.com> wrote:
>
> Got this from a Google alert overnight.  It's not really a Python problem
> (it's a sudo problem), but it's probably not a bad idea to understand the
> implications.
>
>     >> SUDO Python Environment Cleaning Privilege Escalation ...
>     >> Secunia - UK
>     >> ... This can be exploited by a user with sudo access to a python script
>     >> to gain access to an interactive python prompt via the "PYTHONINSPECT"
>     >> environment variable ...
>     >> <http://secunia.com/advisories/18358/>
>
> Skip
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at core-sdi.com  Tue Jan 10 19:27:02 2006
From: lists at core-sdi.com (Gabriel Becedillas)
Date: Tue, 10 Jan 2006 15:27:02 -0300
Subject: [Python-Dev] PyThreadState_Delete vs PyThreadState_DeleteCurrent
Message-ID: <43C3FC76.4010305@core-sdi.com>

Hi,
I started hitting Py_FatalError("Invalid thread state for this thread") 
in pystate.c when a thread bootstraps since my upgrade to Python 2.4.2 
(was using 2.4.1 previously).
I'm embedding Python in C++, using multiple interpreters and threads.
I think the problem is that PyThreadState_Delete (which is used when I 
destroy thread states and interpreters) is not making the same clean up 
that PyThreadState_DeleteCurrent is doing:

if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
	PyThread_delete_key_value(autoTLSkey);

If a thread id is reused (and this often happens), and the previous 
thread used PyThreadState_Delete instead of PyThreadState_DeleteCurrent, 
  an old and invalid value for autoTLSkey is stored, and that is 
triggering the Py_FatalError.
Thanks.

-- 


Gabriel Becedillas
Developer
CORE SECURITY TECHNOLOGIES

Florida 141 - 2? cuerpo - 7? piso
C1005AAC Buenos Aires - Argentina
Tel/Fax: (54 11) 5032-CORE (2673)
http://www.corest.com

From g.brandl-nospam at gmx.net  Tue Jan 10 20:06:48 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Tue, 10 Jan 2006 20:06:48 +0100
Subject: [Python-Dev] test_curses
In-Reply-To: <2m1wzgqvh5.fsf@starship.python.net>
References: <dprnoe$p98$1@sea.gmane.org>
	<2m1wzhsqq5.fsf@starship.python.net>	<dpubjo$buj$1@sea.gmane.org>
	<2m1wzgqvh5.fsf@starship.python.net>
Message-ID: <dq10k9$ctq$1@sea.gmane.org>

Michael Hudson wrote:
> Georg Brandl <g.brandl-nospam at gmx.net> writes:
> 
>> Michael Hudson wrote:
>>> Georg Brandl <g.brandl-nospam at gmx.net> writes:
>>> 
>>>> The call to curses.setupterm() leaves my terminal in a bad state.
>>> 
>>> Hmm.
>>> 
>>>> The reset program outputs:
>>>> Erase set to delete.
>>>> Kill set to control-U (^U).
>>>> Interrupt set to control-C (^C).
>>> 
>>> It always says that :) (unless you've messed with stty, I guess)
>>
>> Well, when I do a reset without meddling with the terminal, it says nothing,
>> at least on my box.
> 
> Oh.  Maybe I'm out of date.
> 
>> And, there's more: Ctrl+D doesn't work, Ctrl+C doesn't work.
> 
> Right, so it definitely sounds like the terminal is in raw mode.
> 
>>>> Doesn't the setupterm() have to be paired with something like shutdownterm()?
>>> 
>>> Not as far as my memory of curses goes.  From the man page:
>>> 
>>>        The setupterm routine reads in the terminfo database,
>>>        initializing the terminfo structures, but does not set up the
>>>        output virtualization structures used by curses.
>>> 
>>> What platform are you on?
>>
>> Linux 2.6, ncurses 5.5, TERM=xterm.
> 
> Well, this still has the faint whiff of impossibility about it.  Are
> you sure it's setupterm() that's doing the damage?  Can you reproduce
> interactively?

Yep.
Alone, the setupterm call [curses.setupterm(sys.__stdout__.fileno())] does
nothing remarkable, but when it is done inside of curses.initscr() / curses.endwin()
the terminal is not restored properly.

If setupterm() is to be tested, I think it should be executed before initscr().

regards,
Georg









From martin at v.loewis.de  Tue Jan 10 20:17:19 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 20:17:19 +0100
Subject: [Python-Dev] Py_ssize_t output parameters (Was:
	[Python-checkins] r41971 ...)
In-Reply-To: <43C37C92.3060302@egenix.com>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>
	<43C2DD44.2040908@v.loewis.de> <43C37C92.3060302@egenix.com>
Message-ID: <43C4083F.3070001@v.loewis.de>

M.-A. Lemburg wrote:
>>I don't believe the change is major. It only affects a few extensions,
>>and for these, it is only a minor change. A single line of changing
>>will be enough.
> 
> 
> This is true for all the changes related to parameters passed by
> value. It is not true for output parameters. For these, the
> change will propagate into the extension and make quite a few
> changes necessary (for the same reason you have to change
> so many variables to Py_ssize_t).

For the output parameter, just the variable receiving
the value needs to be changed, and you are done.

> We should make it possible to have a simple recompile of old
> extensions continue to work with Python 2.5 (albeit without
> them supporting 64-bit indexes) and without introducing
> segfaults or buffer overflows that way.

I cannot see how this could reasonably achieved for arbitrary
extension modules.

Regards,
Martin

From fredrik at pythonware.com  Tue Jan 10 20:29:26 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 10 Jan 2006 20:29:26 +0100
Subject: [Python-Dev] Tkinter
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov><20060110165423.GA10810@panix.com>
	<ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>
Message-ID: <dq11uo$i72$1@sea.gmane.org>

Guido van Rossum wrote:

> But for sure, Tkinter is still being maintained by the Python
> developers (mainly MvL I believe). If it appears stagnant that's
> probably because Tcl/Tk itself isn't changing much.

afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're
probably a lot closer to a release than the Perl 6 guys...

    http://www.tcl.tk/software/tcltk/8.5.html

</F>




From fredrik at pythonware.com  Tue Jan 10 21:00:51 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 10 Jan 2006 21:00:51 +0100
Subject: [Python-Dev] r42003 -
	python/trunk/Lib/test/outstanding_bugs.pypython/trunk/Lib/test/outstanding_crashes.py
References: <20060110192928.F114B1E4002@bag.python.org>
Message-ID: <dq13pl$pee$1@sea.gmane.org>

time to synchronize your activities a bit ?

> Author: neal.norwitz
> Date: Tue Jan 10 08:49:41 2006
> New Revision: 42000
>
> Added:
>    python/trunk/Lib/test/crashers/
>    python/trunk/Lib/test/crashers/README
>    python/trunk/Lib/test/crashers/coerce.py
>    python/trunk/Lib/test/crashers/weakref_in_del.py
>    python/trunk/Lib/test/crashers/xml_parsers.py
> Log:
> As I threatened on python-dev, add a directory which contains all known
> bugs which cause the interpreter to crash.  I'm sure we can find a few
> more.  Many missing bugs deal with variations on unchecked infinite recursion
> (like coerce.py).

> Author: georg.brandl
> Date: Tue Jan 10 20:29:24 2006
> New Revision: 42003
>
> Added:
>    python/trunk/Lib/test/outstanding_crashes.py
> Modified:
>    python/trunk/Lib/test/outstanding_bugs.py
> Log:
> Add outstanding_crashes.py with tests for crashes.




From martin at v.loewis.de  Tue Jan 10 21:05:45 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 21:05:45 +0100
Subject: [Python-Dev] buildbot
In-Reply-To: <20060110084955.GF10373@home.puzzling.org>
References: <20060104.003058.48524852.warner@lothar.com>
	<43BD9B32.3030700@v.loewis.de>
	<20060108.173810.91561560.warner@lothar.com>
	<43C36D3C.7050202@v.loewis.de>
	<20060110084955.GF10373@home.puzzling.org>
Message-ID: <43C41399.9090707@v.loewis.de>

Andrew Bennetts wrote:
> For this part at least, it's easier than that; you can pass
> interface='127.0.0.1' to the relevant listenTCP or TCPServer call, so that it
> will only bind that port to localhost.

Thanks for the suggestion! works fine for me.

Regards,
Martin

From martin at v.loewis.de  Tue Jan 10 21:28:43 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 21:28:43 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C37C02.8030005@egenix.com>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>	<43BD8464.4060801@v.loewis.de>
	<43C37C02.8030005@egenix.com>
Message-ID: <43C418FB.7010007@v.loewis.de>

M.-A. Lemburg wrote:
> I don't see a good solution for these other than introducing
> a set of new APIs (and maybe doing some macro magic as Martin
> did for PyArg_ParseTuple()). Due to the fact that changes in
> the types of output parameters require changes in the
> extension variable type layout itself, they introduce a large
> number of type changes in the extension and make writing
> backwards compatible extensions harder than necessary.

That's not true. It is very easy to write extensions that
receive such values and are still backwards-compatible.

Suppose you had

  int pos;
  PyObject *k, *v;

  PyDict_Next(dict, &pos, &k, &v);

You just change this to

  /* beginning of file */
  #ifdef Py_HEX_VERSION < 2.5
  typedef int Py_ssize_t;
  #endif

  /* later */
  Py_ssize_t pos;
  PyObject *k, *v;

  PyDict_Next(dict, &pos, &k, &v);

That's it!

> Furthermore, all extensions for Python 2.4 would have to be
> ported to the new Python API and porting is not going to
> be a simple recompile, but will require C skills.

Not all extensions. Only those that call functions that expect
int* output parameters - which is fairly uncommon.

> Martin, please add the above points to the PEP. I'd also
> like to see it published, because it's hard to track a PEP
> in the mailing

It's very difficult to get a PEP number assigned. I wrote
peps at python.org with no response.

Regards,
Martin

From guido at python.org  Tue Jan 10 21:34:47 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 12:34:47 -0800
Subject: [Python-Dev] Tkinter
In-Reply-To: <dq11uo$i72$1@sea.gmane.org>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
	<20060110165423.GA10810@panix.com>
	<ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>
	<dq11uo$i72$1@sea.gmane.org>
Message-ID: <ca471dc20601101234p40f135fcp74415a41c2491042@mail.gmail.com>

On 1/10/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> > But for sure, Tkinter is still being maintained by the Python
> > developers (mainly MvL I believe). If it appears stagnant that's
> > probably because Tcl/Tk itself isn't changing much.
>
> afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're
> probably a lot closer to a release than the Perl 6 guys...
>
>     http://www.tcl.tk/software/tcltk/8.5.html

Well to compare to Perl 6 you should really look at the progress of
Tcl/Tk 9.0, which sounds like it's never going to happen. So Perl 6
still wins. :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Tue Jan 10 21:37:53 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 21:37:53 +0100
Subject: [Python-Dev] Tkinter
In-Reply-To: <dq11uo$i72$1@sea.gmane.org>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov><20060110165423.GA10810@panix.com>	<ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>
	<dq11uo$i72$1@sea.gmane.org>
Message-ID: <43C41B21.40500@v.loewis.de>

Fredrik Lundh wrote:
>>But for sure, Tkinter is still being maintained by the Python
>>developers (mainly MvL I believe). If it appears stagnant that's
>>probably because Tcl/Tk itself isn't changing much.
> 
> 
> afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're
> probably a lot closer to a release than the Perl 6 guys...
> 
>     http://www.tcl.tk/software/tcltk/8.5.html

My maintenance focuses on integrating user contributions, plus looking
for the latest Tcl release when doing a Windows build. So new options
to existing widgets, or entire new widgets don't get "automatically"
wrapper classes in Tkinter. Instead, users wanting to make use of
such features are expected to contribute a patch.

I usually catch up with such patches before the next major release,
atleast if the patches are straight-forward.

Regards,
Martin

From theller at python.net  Tue Jan 10 21:55:02 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 10 Jan 2006 21:55:02 +0100
Subject: [Python-Dev] Include ctypes into core Python?
Message-ID: <u0cbyfa1.fsf@python.net>

I would like to suggest to include ctypes into core Python, starting
with the 2.5 release.

ctypes is nearing the 1.0 release, and works on Windows, OS X, linux,
possibly other platforms (provided that the processor is supported by
libffi), and just recently even on Windows CE.

ctypes does not depend on a libfii library installed on the system,
instead the libfii source code is compiled into the extension module.
libffi is copyright Red Hat, Inc, and released under a MIT style
license.

Up-to date docs for ctypes still have to be written, but I assume that
the 2.5 release timeframe leaves enough time for that.

Thomas



From mal at egenix.com  Tue Jan 10 22:05:39 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 10 Jan 2006 22:05:39 +0100
Subject: [Python-Dev] Py_ssize_t output parameters (Was:
	[Python-checkins] r41971 ...)
In-Reply-To: <43C4083F.3070001@v.loewis.de>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>
	<43C2DD44.2040908@v.loewis.de>	<43C37C92.3060302@egenix.com>
	<43C4083F.3070001@v.loewis.de>
Message-ID: <43C421A3.2030806@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>>> I don't believe the change is major. It only affects a few extensions,
>>> and for these, it is only a minor change. A single line of changing
>>> will be enough.
>>
>> This is true for all the changes related to parameters passed by
>> value. It is not true for output parameters. For these, the
>> change will propagate into the extension and make quite a few
>> changes necessary (for the same reason you have to change
>> so many variables to Py_ssize_t).
> 
> For the output parameter, just the variable receiving
> the value needs to be changed, and you are done.

... and then the type change of that variable propagates
throughout the extension.

You basically end up having to convert the whole extension
to Py_ssize_t.

Don't get me wrong: I don't mind doing this for the eGenix
extensions, but I'm worried about the gazillion other
useful extensions out there which probably won't get
upgraded in time to be used with Python 2.5.

>> We should make it possible to have a simple recompile of old
>> extensions continue to work with Python 2.5 (albeit without
>> them supporting 64-bit indexes) and without introducing
>> segfaults or buffer overflows that way.
> 
> I cannot see how this could reasonably achieved for arbitrary
> extension modules.

I think all it takes is a set of new APIs for functions
that use Py_ssize_t as output parameter and which are
mapped to the regular API names if and only if the
extension #defines PY_SSIZE_T_CLEAN (or some other
capability flag).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From skip at pobox.com  Tue Jan 10 22:07:48 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 10 Jan 2006 15:07:48 -0600
Subject: [Python-Dev] Tkinter
In-Reply-To: <ca471dc20601101234p40f135fcp74415a41c2491042@mail.gmail.com>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
	<20060110165423.GA10810@panix.com>
	<ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>
	<dq11uo$i72$1@sea.gmane.org>
	<ca471dc20601101234p40f135fcp74415a41c2491042@mail.gmail.com>
Message-ID: <17348.8740.582103.283482@montanaro.dyndns.org>


    Guido> Well to compare to Perl 6 you should really look at the progress
    Guido> of Tcl/Tk 9.0, which sounds like it's never going to happen. So
    Guido> Perl 6 still wins. :-)

So the grand race is between Tcl/Tk 9.0, Perl 6 and Python 3?  Maybe you,
Larry and whoever the Tcl/Tk master is these days (still Ousterhout?) could
agree to release all three on the same day.  <wink>

Skip

From guido at python.org  Tue Jan 10 22:11:33 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 13:11:33 -0800
Subject: [Python-Dev] Tkinter
In-Reply-To: <17348.8740.582103.283482@montanaro.dyndns.org>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
	<20060110165423.GA10810@panix.com>
	<ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>
	<dq11uo$i72$1@sea.gmane.org>
	<ca471dc20601101234p40f135fcp74415a41c2491042@mail.gmail.com>
	<17348.8740.582103.283482@montanaro.dyndns.org>
Message-ID: <ca471dc20601101311m2fe2fe72q733b658a850bfe32@mail.gmail.com>

On 1/10/06, skip at pobox.com <skip at pobox.com> wrote:
>
>     Guido> Well to compare to Perl 6 you should really look at the progress
>     Guido> of Tcl/Tk 9.0, which sounds like it's never going to happen. So
>     Guido> Perl 6 still wins. :-)
>
> So the grand race is between Tcl/Tk 9.0, Perl 6 and Python 3?  Maybe you,
> Larry and whoever the Tcl/Tk master is these days (still Ousterhout?) could
> agree to release all three on the same day.  <wink>

Or we could have another bake-off and pie-throwing contest. :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jan 10 22:14:13 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 13:14:13 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <u0cbyfa1.fsf@python.net>
References: <u0cbyfa1.fsf@python.net>
Message-ID: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>

On 1/10/06, Thomas Heller <theller at python.net> wrote:
> I would like to suggest to include ctypes into core Python, starting
> with the 2.5 release.

On the one hand I agree that this is a useful module, popular, mature etc.

On the other hand it breaks one of the most fundamental Python
guidelines: if you get a core dump (segfault etc.) it's a bug in
Python or in a 3rd party extension, not in *your* Python code. An
exception would have to be made for any code that uses ctypes, as it
is usually trivial to cause core dumps with ctypes (I'd venture it's
hard to avoid them ;-).

I don't expect this to count against including ctypes; but I do want
it to be dealt with somehow!

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From trentm at ActiveState.com  Tue Jan 10 22:15:16 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Tue, 10 Jan 2006 13:15:16 -0800
Subject: [Python-Dev] Tkinter
In-Reply-To: <17348.8740.582103.283482@montanaro.dyndns.org>
References: <14A2A120D369B6469BB154B2D2DC34D203537050@EXCHVS01.ad.sfwmd.gov>
	<20060110165423.GA10810@panix.com>
	<ca471dc20601101010m12fc154bu8023b04a3f1b86a@mail.gmail.com>
	<dq11uo$i72$1@sea.gmane.org>
	<ca471dc20601101234p40f135fcp74415a41c2491042@mail.gmail.com>
	<17348.8740.582103.283482@montanaro.dyndns.org>
Message-ID: <20060110211516.GA8360@activestate.com>

[skip at pobox.com wrote]
> ...and whoever the Tcl/Tk master is these days (still Ousterhout?)...

That's Jeff Hobbs. He sits behind me. I'll see if I can pester him to
give some Tcl/Tk and _tkinter thoughts.

Cheers,
Trent

-- 
Trent Mick
trentm at activestate.com

From jorodrig at sfwmd.gov  Tue Jan 10 22:17:12 2006
From: jorodrig at sfwmd.gov (Rodrigues, Joseph)
Date: Tue, 10 Jan 2006 16:17:12 -0500
Subject: [Python-Dev] Tkinter
Message-ID: <14A2A120D369B6469BB154B2D2DC34D203537052@EXCHVS01.ad.sfwmd.gov>

Thanks for your reply.  (I won't bug you on this with lots of email) but I would really like a comment or two on the other python GUIs viz. wxPython and QtPy.

Would you say wxPython and QtPy are competing with TKinter or vice versa?

Why wxPyton and/or QtPy when TKinter is alive and well? 


Why am I asking about this?
I am working with an application that my agency started with TKinter and is investigating if we should stay with it or consider the others and have not been able to get the kind of direction anywhere.  In addition, we purchased the John E. Grayson book but realized it is from 2000 and is wondering what is missing that has been developed in TKinter since 2000?  Why has the book become so dated with no updates?  This led me to consider that TKinter may be on its way out and we need to consider the other GUI toolkits.

I am glad for you response and understand if you cannot dwell too long on this topic.

Sincerely
Joseph

-----Original Message-----
From: "Martin v. L?wis" [mailto:martin at v.loewis.de] 
Sent: Tuesday, January 10, 2006 3:38 PM
To: Fredrik Lundh
Cc: python-dev at python.org; Rodrigues, Joseph
Subject: Re: [Python-Dev] Tkinter

Fredrik Lundh wrote:
>>But for sure, Tkinter is still being maintained by the Python
>>developers (mainly MvL I believe). If it appears stagnant that's
>>probably because Tcl/Tk itself isn't changing much.
> 
> 
> afaict, Tcl/Tk 8.5 isn't moving quite as fast as Python 2.5, but they're
> probably a lot closer to a release than the Perl 6 guys...
> 
>     http://www.tcl.tk/software/tcltk/8.5.html

My maintenance focuses on integrating user contributions, plus looking
for the latest Tcl release when doing a Windows build. So new options
to existing widgets, or entire new widgets don't get "automatically"
wrapper classes in Tkinter. Instead, users wanting to make use of
such features are expected to contribute a patch.

I usually catch up with such patches before the next major release,
atleast if the patches are straight-forward.

Regards,
Martin



From mal at egenix.com  Tue Jan 10 22:20:36 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 10 Jan 2006 22:20:36 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C418FB.7010007@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>	<43BD8464.4060801@v.loewis.de>	<43C37C02.8030005@egenix.com>
	<43C418FB.7010007@v.loewis.de>
Message-ID: <43C42524.7000802@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>> I don't see a good solution for these other than introducing
>> a set of new APIs (and maybe doing some macro magic as Martin
>> did for PyArg_ParseTuple()). Due to the fact that changes in
>> the types of output parameters require changes in the
>> extension variable type layout itself, they introduce a large
>> number of type changes in the extension and make writing
>> backwards compatible extensions harder than necessary.
> 
> That's not true. It is very easy to write extensions that
> receive such values and are still backwards-compatible.
> 
> Suppose you had
> 
>   int pos;
>   PyObject *k, *v;
> 
>   PyDict_Next(dict, &pos, &k, &v);
> 
> You just change this to
> 
>   /* beginning of file */
>   #ifdef Py_HEX_VERSION < 2.5
>   typedef int Py_ssize_t;
>   #endif
> 
>   /* later */
>   Py_ssize_t pos;
>   PyObject *k, *v;
> 
>   PyDict_Next(dict, &pos, &k, &v);
> 
> That's it!

If it were this easy, I wouldn't have objections. But it's
not.

The variables you use with these APIs tend to propagate
through the extension, you use them in other calls,
make assignments, etc.

If you implement extension types, you end up having to
convert all the length related struct variables to
Py_ssize_t.

If you're writing against 3rd party APIs which don't
use ssize_t or size_t, you have to convert Py_ssize_t
to int where necessary.

All this is fine, but it's also a lot of work which
can be made easier. Recompiling an extension is well
within range of many Python users, manually checking,
fixing and porting it to the new API is certainly not.

>> Furthermore, all extensions for Python 2.4 would have to be
>> ported to the new Python API and porting is not going to
>> be a simple recompile, but will require C skills.
> 
> Not all extensions. Only those that call functions that expect
> int* output parameters - which is fairly uncommon.

The set of functions that will require Py_ssize_t
is getting larger in your branch which is why I started
this discussion.

In the first checkin you only had
the rarely used slice APIs converted. In the meantime
the buffer APIs, the Unicode APIs and others have
been added to the list.

These APIs are used a lot more often than the slice
APIs.

I'm not saying that it's a bad idea to adjust these
to Py_ssize_t, it's just the backwards incompatible
way this is done which bothers me.

>> Martin, please add the above points to the PEP. I'd also
>> like to see it published, because it's hard to track a PEP
>> in the mailing
> 
> It's very difficult to get a PEP number assigned. I wrote
> peps at python.org with no response.

Would it be possible to host the PEP in the python.org
wiki or maybe in the sandbox on svn.python.org ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 10 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From martin at v.loewis.de  Tue Jan 10 22:23:42 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 22:23:42 +0100
Subject: [Python-Dev] Py_ssize_t output parameters (Was:
	[Python-checkins] r41971 ...)
In-Reply-To: <43C421A3.2030806@egenix.com>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>
	<43C2DD44.2040908@v.loewis.de>	<43C37C92.3060302@egenix.com>
	<43C4083F.3070001@v.loewis.de> <43C421A3.2030806@egenix.com>
Message-ID: <43C425DE.2050001@v.loewis.de>

M.-A. Lemburg wrote:
> ... and then the type change of that variable propagates
> throughout the extension.

That depends on the usage of the code. If the variable
is passed by value, no further changes are necessary.
If a pointer to the variable is passed, you could replace
it with

  Py_ssize_t x64; int x;
  foo(&x64);
  x = x64;

Then use x as you did with the original code.

> You basically end up having to convert the whole extension
> to Py_ssize_t.

That is not necessary. Can you give in example of a module
where you think it is necessary?

> Don't get me wrong: I don't mind doing this for the eGenix
> extensions, but I'm worried about the gazillion other
> useful extensions out there which probably won't get
> upgraded in time to be used with Python 2.5.

I agree that it is not acceptable to require immediate
whole-sale updates of every modules. However, I do
believe that the number of modules that need any change
at all is small, and that those modules can be modified
with minimal effort to "get them working again,
backwards-compatible" (i.e. with the only exception that
they would fail if indices run above 2**31).

> I think all it takes is a set of new APIs for functions
> that use Py_ssize_t as output parameter and which are
> mapped to the regular API names if and only if the
> extension #defines PY_SSIZE_T_CLEAN (or some other
> capability flag).

That is not enough. You also need to deal with the
function pointers that change.

Also, others have rejected/expressed dislike of the
PY_SIZE_T_CLEAN macro already, so they would probably
dislike further hackishness in that direction.

Anyway, I have installed the PEP onsite, and
added an "Open Issues" section, recording your
comments.

Regards,
Martin

From martin at v.loewis.de  Tue Jan 10 22:28:52 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 22:28:52 +0100
Subject: [Python-Dev] Tkinter
In-Reply-To: <14A2A120D369B6469BB154B2D2DC34D203537052@EXCHVS01.ad.sfwmd.gov>
References: <14A2A120D369B6469BB154B2D2DC34D203537052@EXCHVS01.ad.sfwmd.gov>
Message-ID: <43C42714.6060702@v.loewis.de>

Rodrigues, Joseph wrote:
> Would you say wxPython and QtPy are competing with TKinter or vice
> versa?

No. I never say that software "competes". I don't even say that products
compete. People may compete, or perhaps companies, but not things.

> Why wxPyton and/or QtPy when TKinter is alive and well?

Because they have their user communities.

> Why am I asking about this?

Not sure :-)

> I am working with an application that my agency started with TKinter
> and is investigating if we should stay with it or consider the others
> and have not been able to get the kind of direction anywhere.  In
> addition, we purchased the John E. Grayson book but realized it is
> from 2000 and is wondering what is missing that has been developed in
> TKinter since 2000?

Not much - code in that book should continue to work just fine.

> Why has the book become so dated with no updates?

You would have to ask John Grayson for that.

> This led me to consider that TKinter may be on its way out
> and we need to consider the other GUI toolkits.

Well, this is certainly off-topic for python-dev.

Regards,
Martin

From martin at v.loewis.de  Tue Jan 10 22:35:30 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 10 Jan 2006 22:35:30 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C42524.7000802@egenix.com>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>	<43BD8464.4060801@v.loewis.de>	<43C37C02.8030005@egenix.com>
	<43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com>
Message-ID: <43C428A2.9000703@v.loewis.de>

M.-A. Lemburg wrote:
> If it were this easy, I wouldn't have objections. But it's
> not.

It is so easy. Can you should me an example where it isn't?

> The variables you use with these APIs tend to propagate
> through the extension, you use them in other calls,
> make assignments, etc.

They only propage if you make them. You don't have to,
if you don't want to.

> If you implement extension types, you end up having to
> convert all the length related struct variables to
> Py_ssize_t.

Only if you want to. If not, the module will work
(nearly) unmodified. Of course, it will be limited
to 32-bit indices.

> All this is fine, but it's also a lot of work which
> can be made easier. Recompiling an extension is well
> within range of many Python users, manually checking,
> fixing and porting it to the new API is certainly not.

Sure. However, most users will compile it on 32-bit
systems. If they find they cannot get it to work on
a 64-bit system, they should ask the author for help,
or just use it in 32-bit mode (as 64-bit mode won't
gain them anything, anyway).

> The set of functions that will require Py_ssize_t
> is getting larger in your branch which is why I started
> this discussion.

How so? I did not add a single function that has
int* output values, AFAICT.

I am talking about the entirety of these functions,
and claim that they are rarely used (including the
Unicode and buffer APIs).

> Would it be possible to host the PEP in the python.org
> wiki or maybe in the sandbox on svn.python.org ?

I pinged the PEP editors again, and they assigned
the PEP number.

Hosting it in a Wiki would be besides the point of
the PEP process.

Regards,
Martin


From tim.peters at gmail.com  Tue Jan 10 22:49:11 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 10 Jan 2006 16:49:11 -0500
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C428A2.9000703@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
	<43B50B64.20904@v.loewis.de>
	<20060105194538.GB11736@code1.codespeak.net>
	<43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com>
	<43C418FB.7010007@v.loewis.de> <43C42524.7000802@egenix.com>
	<43C428A2.9000703@v.loewis.de>
Message-ID: <1f7befae0601101349l57e3ae35o65913fecc7112710@mail.gmail.com>

[Martin v. L?wis]
> ...
> I am talking about the entirety of these functions,
> and claim that they are rarely used (including the
> Unicode and buffer APIs).

This reminded me that I still owe you a reply about s# and t# format
codes.  It occurred to me that I've never used them, and probably
never will, so I really don't care how they work:  I'm only really
worried about widespread ugliness, meaning wide enough that it touches
me ;-)

From tdelaney at avaya.com  Wed Jan 11 00:08:27 2006
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Wed, 11 Jan 2006 10:08:27 +1100
Subject: [Python-Dev] Include ctypes into core Python?
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

> On 1/10/06, Thomas Heller <theller at python.net> wrote:
>> I would like to suggest to include ctypes into core Python, starting
>> with the 2.5 release.
> 
> On the one hand I agree that this is a useful module, popular, mature
> etc. 
> 
> On the other hand it breaks one of the most fundamental Python
> guidelines: if you get a core dump (segfault etc.) it's a bug in
> Python or in a 3rd party extension, not in *your* Python code. An
> exception would have to be made for any code that uses ctypes, as it
> is usually trivial to cause core dumps with ctypes (I'd venture it's
> hard to avoid them ;-).
> 
> I don't expect this to count against including ctypes; but I do want
> it to be dealt with somehow!

As was pointed out on c.l.py, the `dl` module suffers the exact same
problem (I don't know myself, as I've never used it). There are no
warnings about this in the `dl` module documentation.

I can't see how it would be possible to guarantee that such a module
could not cause crashes. I'm of the opinion that having a big red
warning at the top of the module documentation that this is a
contributed module, and incorrect use could cause segmentation
faults/crashes, etc would be sufficient.

Tim Delaney

From g.brandl-nospam at gmx.net  Wed Jan 11 00:10:22 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Wed, 11 Jan 2006 00:10:22 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
Message-ID: <dq1esu$64d$1@sea.gmane.org>

Delaney, Timothy (Tim) wrote:
> Guido van Rossum wrote:
> 
>> On 1/10/06, Thomas Heller <theller at python.net> wrote:
>>> I would like to suggest to include ctypes into core Python, starting
>>> with the 2.5 release.
>> 
>> On the one hand I agree that this is a useful module, popular, mature
>> etc. 
>> 
>> On the other hand it breaks one of the most fundamental Python
>> guidelines: if you get a core dump (segfault etc.) it's a bug in
>> Python or in a 3rd party extension, not in *your* Python code. An
>> exception would have to be made for any code that uses ctypes, as it
>> is usually trivial to cause core dumps with ctypes (I'd venture it's
>> hard to avoid them ;-).
>> 
>> I don't expect this to count against including ctypes; but I do want
>> it to be dealt with somehow!
> 
> As was pointed out on c.l.py, the `dl` module suffers the exact same
> problem (I don't know myself, as I've never used it). There are no
> warnings about this in the `dl` module documentation.
> 
> I can't see how it would be possible to guarantee that such a module
> could not cause crashes. I'm of the opinion that having a big red
> warning at the top of the module documentation that this is a
> contributed module, and incorrect use could cause segmentation
> faults/crashes, etc would be sufficient.

+1.

A warning for dl might not be the worst thing.

Georg


From guido at python.org  Wed Jan 11 00:12:43 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 15:12:43 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
Message-ID: <ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>

On 1/10/06, Delaney, Timothy (Tim) <tdelaney at avaya.com> wrote:
> Guido van Rossum wrote:
>
> > On 1/10/06, Thomas Heller <theller at python.net> wrote:
> >> I would like to suggest to include ctypes into core Python, starting
> >> with the 2.5 release.
> >
> > On the one hand I agree that this is a useful module, popular, mature
> > etc.
> >
> > On the other hand it breaks one of the most fundamental Python
> > guidelines: if you get a core dump (segfault etc.) it's a bug in
> > Python or in a 3rd party extension, not in *your* Python code. An
> > exception would have to be made for any code that uses ctypes, as it
> > is usually trivial to cause core dumps with ctypes (I'd venture it's
> > hard to avoid them ;-).
> >
> > I don't expect this to count against including ctypes; but I do want
> > it to be dealt with somehow!
>
> As was pointed out on c.l.py, the `dl` module suffers the exact same
> problem (I don't know myself, as I've never used it). There are no
> warnings about this in the `dl` module documentation.

Good point. I think there should be such warnings though.

> I can't see how it would be possible to guarantee that such a module
> could not cause crashes.

And I'm not asking for that.

> I'm of the opinion that having a big red
> warning at the top of the module documentation that this is a
> contributed module, and incorrect use could cause segmentation
> faults/crashes, etc would be sufficient.

Works for me.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tdelaney at avaya.com  Wed Jan 11 00:22:22 2006
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Wed, 11 Jan 2006 10:22:22 +1100
Subject: [Python-Dev] Include ctypes into core Python?
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F074324@au3010avexu1.global.avaya.com>

Guido van Rossum wrote:

>> As was pointed out on c.l.py, the `dl` module suffers the exact same
>> problem (I don't know myself, as I've never used it). There are no
>> warnings about this in the `dl` module documentation.
> 
> Good point. I think there should be such warnings though.

Added documentation patch 1402224:
http://sourceforge.net/tracker/index.php?func=detail&aid=1402224&group_i
d=5470&atid=305470

May need a little more work on the wording.

>> I can't see how it would be possible to guarantee that such a module
>> could not cause crashes.
> 
> And I'm not asking for that.
> 
>> I'm of the opinion that having a big red
>> warning at the top of the module documentation that this is a
>> contributed module, and incorrect use could cause segmentation
>> faults/crashes, etc would be sufficient.
> 
> Works for me.

Sounds like we agree on everything :) Maybe I should start channelling
... <wink>.

Tim Delaney

From thomas at xs4all.net  Wed Jan 11 00:23:24 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 11 Jan 2006 00:23:24 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
Message-ID: <20060110232324.GO18916@xs4all.nl>

On Tue, Jan 10, 2006 at 01:14:13PM -0800, Guido van Rossum wrote:

> On the other hand it breaks one of the most fundamental Python
> guidelines: if you get a core dump (segfault etc.) it's a bug in
> Python or in a 3rd party extension, not in *your* Python code. An
> exception would have to be made for any code that uses ctypes, as it
> is usually trivial to cause core dumps with ctypes (I'd venture it's
> hard to avoid them ;-).

Aside from 'dl', what was also pointed out in c.l.py was the crashability of
Python in general, even from pure Python code:

centurion:~ > python < .
Segmentation fault

[...]
>>> sys.setrecursionlimit(1<<30)
>>> f = lambda f:f(f)
>>> f(f)
Segmentation fault

There's more, all from Python itself. And sure, "well, don't do that then"
is a perfectly valid response to most of these harebrained tricks, but it
does put a lie to the 'uncrashable python' idea :)

Not-for-or-against-including-ctypes-anyway'ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From guido at python.org  Wed Jan 11 00:35:08 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 15:35:08 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060110232324.GO18916@xs4all.nl>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
Message-ID: <ca471dc20601101535x6956f0a8l9e5b5c445eca6a3b@mail.gmail.com>

On 1/10/06, Thomas Wouters <thomas at xs4all.net> wrote:
> Aside from 'dl', what was also pointed out in c.l.py was the crashability of
> Python in general, even from pure Python code:
>
> centurion:~ > python < .
> Segmentation fault

This I think ought to be fixed; it's just (I presume) the parser
stumbling over extremely invalid input.

> [...]
> >>> sys.setrecursionlimit(1<<30)
> >>> f = lambda f:f(f)
> >>> f(f)
> Segmentation fault

OK, point taken.

> There's more, all from Python itself. And sure, "well, don't do that then"
> is a perfectly valid response to most of these harebrained tricks, but it
> does put a lie to the 'uncrashable python' idea :)

I'm not saying it's uncrashable. I'm saying that if you crash it, it's
a bug unless proven harebrained.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Wed Jan 11 00:42:13 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 10 Jan 2006 17:42:13 -0600
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
Message-ID: <17348.18005.164441.346058@montanaro.dyndns.org>


    Tim> I'm of the opinion that having a big red warning at the top of the
    Tim> module documentation ...

Do we have semantic markup to support that?

<ducks>

Skip

From fredrik at pythonware.com  Wed Jan 11 00:42:13 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 11 Jan 2006 00:42:13 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
Message-ID: <dq1gon$c9p$1@sea.gmane.org>

I've been playing with "automatic module discovery" in an attempt to figure
out how complete the library reference really is.

I've created a simple script that tweaks the path to get rid of site-package
stuff, and then scans the path for python modules and extensions, and writes
the result to a text file.  By combining module lists from different sources,
you can trivially generate all sorts of custom module indexes (e.g. modules
available on typical linux machines, modules not available in 2.5 on windows,
etc).  For some examples etc, see:

    http://effbot.org/lib/index-24.html
    http://effbot.org/lib/index-23.html
    http://effbot.org/lib/index-win32.html

    http://online.effbot.org/2006_01_01_archive.htm#20060110
    http://online.effbot.org/2006_01_01_archive.htm#module-count

Most importantly, this makes it easy to track down missing reference pages,
and generate stubs for pages that "should exist".

My initial thought was that we could ask alpha testers to run this script on
their alpha builds, and report back, but it just struck me that the "buildbot"
already builds stuff on a couple of interesting platforms.

Can buildbot deal with custom test/validation scripts, and collect the output
somewhere ?

</F>




From raymond.hettinger at verizon.net  Wed Jan 11 00:42:25 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue, 10 Jan 2006 18:42:25 -0500
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ca471dc20601101535x6956f0a8l9e5b5c445eca6a3b@mail.gmail.com>
Message-ID: <000801c6163f$833558e0$e427a044@oemcomputer>

[Guido]
> I'm not saying it's uncrashable. I'm saying that if you crash it, it's
> a bug unless proven harebrained.

+1 QOTW


From crutcher at gmail.com  Wed Jan 11 00:47:18 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Tue, 10 Jan 2006 15:47:18 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when used as
	global dictionaries
Message-ID: <d49fe110601101547m1dce16c5n63233e7752516f5d@mail.gmail.com>

There is an inconsistancy in the way that dictionary subclasses behave
when they are used as as namespaces in execs.

Basically, while python 2.4 permits the usage of dictionary subclasses
for local environments, it still bypasses the subclass functions and
uses the C API for global environments. The attached patch (and
unittest!) addresses this issue.

I'm pretty sure we keep the fast path in this.
--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com
-------------- next part --------------
A non-text attachment was scrubbed...
Name: dictsubclassexec.patch
Type: application/octet-stream
Size: 6621 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060110/34be979a/attachment.obj 

From jcarlson at uci.edu  Wed Jan 11 00:57:28 2006
From: jcarlson at uci.edu (Josiah Carlson)
Date: Tue, 10 Jan 2006 15:57:28 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060110232324.GO18916@xs4all.nl>
References: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
Message-ID: <20060110154626.C003.JCARLSON@uci.edu>


Thomas Wouters <thomas at xs4all.net> wrote:
> centurion:~ > python < .
> Segmentation fault
> 
> [...]
> >>> sys.setrecursionlimit(1<<30)
> >>> f = lambda f:f(f)
> >>> f(f)
> Segmentation fault
> 
> There's more, all from Python itself. And sure, "well, don't do that then"
> is a perfectly valid response to most of these harebrained tricks, but it
> does put a lie to the 'uncrashable python' idea :)

Many of the "segfaulting Python" issues are platform specific.  On a
platform with a sane malloc, you get a MemoryError for the recursion. 
On a platform which doesn't see the current path as a readable file, you
get "Access is denied." on the redirection attempt.

As always, I'm sure that reasonable patches which work around such
segfaulting cases will be acceptable, though remember that it may not be
clear how to work around them.


 - Josiah


From thomas at xs4all.net  Wed Jan 11 01:03:10 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 11 Jan 2006 01:03:10 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060110154626.C003.JCARLSON@uci.edu>
References: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<20060110154626.C003.JCARLSON@uci.edu>
Message-ID: <20060111000310.GP18916@xs4all.nl>

On Tue, Jan 10, 2006 at 03:57:28PM -0800, Josiah Carlson wrote:

> As always, I'm sure that reasonable patches which work around such
> segfaulting cases will be acceptable, though remember that it may not be
> clear how to work around them.

Sorry, I missed the point I was aiming at, I guess. I wasn't aiming for
fixable bugs; I see these things as, with great effort, holding up your foot
at an awkward angle so that it ends up right at the business end of your
primed and lit medieval cannon. We can jump through hoops to stop those
particular examples, but I'd rather spend time and effort to fix *real* bugs
instead. I was just showing that if you want to crash Python, you need
neither 'ctypes' nor 'dl'.

I do agree a warning is suitable to both of those modules, though if anyone
wants to nitpick, similar modules might be added to marshal and cPickle (and
pickle, too!), when feeding invalid input. Those modules are more in the
awkward-foot-angle terrain for me, though.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From crutcher at gmail.com  Wed Jan 11 01:04:52 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Tue, 10 Jan 2006 16:04:52 -0800
Subject: [Python-Dev] Hello
Message-ID: <d49fe110601101604u462fa285p4075949a9b0dcb48@mail.gmail.com>

Sorry, sent a patch without an intro.

My name is Crutcher Dunnavant
I'm working on a doctorate in computer science (in modeling language practises).
Before I got my master's degree, I used to work for Red Hat in North Carolina as
an OS developer, and I now work in the San Fransisco Bay Area as a
system analyst.

--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com

From aahz at pythoncraft.com  Wed Jan 11 01:08:44 2006
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 10 Jan 2006 16:08:44 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when
	used as global dictionaries
In-Reply-To: <d49fe110601101547m1dce16c5n63233e7752516f5d@mail.gmail.com>
References: <d49fe110601101547m1dce16c5n63233e7752516f5d@mail.gmail.com>
Message-ID: <20060111000844.GA11919@panix.com>

On Tue, Jan 10, 2006, Crutcher Dunnavant wrote:
>
> There is an inconsistancy in the way that dictionary subclasses behave
> when they are used as as namespaces in execs.
> 
> Basically, while python 2.4 permits the usage of dictionary subclasses
> for local environments, it still bypasses the subclass functions and
> uses the C API for global environments. The attached patch (and
> unittest!) addresses this issue.

Please submit the patch to SourceForge and report back with the SF ID.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From crutcher at gmail.com  Wed Jan 11 01:25:07 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Tue, 10 Jan 2006 16:25:07 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when
	used as global dictionaries
In-Reply-To: <20060111000844.GA11919@panix.com>
References: <d49fe110601101547m1dce16c5n63233e7752516f5d@mail.gmail.com>
	<20060111000844.GA11919@panix.com>
Message-ID: <d49fe110601101625x6df3bbffmc7d7d1fd307189b2@mail.gmail.com>

1402289

On 1/10/06, Aahz <aahz at pythoncraft.com> wrote:
> On Tue, Jan 10, 2006, Crutcher Dunnavant wrote:
> >
> > There is an inconsistancy in the way that dictionary subclasses behave
> > when they are used as as namespaces in execs.
> >
> > Basically, while python 2.4 permits the usage of dictionary subclasses
> > for local environments, it still bypasses the subclass functions and
> > uses the C API for global environments. The attached patch (and
> > unittest!) addresses this issue.
>
> Please submit the patch to SourceForge and report back with the SF ID.
> --
> Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/
>
> "19. A language that doesn't affect the way you think about programming,
> is not worth knowing."  --Alan Perlis
>


--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com

From guido at python.org  Wed Jan 11 01:38:56 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 10 Jan 2006 16:38:56 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060111000310.GP18916@xs4all.nl>
References: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<20060110154626.C003.JCARLSON@uci.edu>
	<20060111000310.GP18916@xs4all.nl>
Message-ID: <ca471dc20601101638m59d1a662x18614df445d6a2e7@mail.gmail.com>

On 1/10/06, Thomas Wouters <thomas at xs4all.net> wrote:
> Sorry, I missed the point I was aiming at, I guess. I wasn't aiming for
> fixable bugs; I see these things as, with great effort, holding up your foot
> at an awkward angle so that it ends up right at the business end of your
> primed and lit medieval cannon. We can jump through hoops to stop those
> particular examples, but I'd rather spend time and effort to fix *real* bugs
> instead.

I'm not sure what makes those particular examples less than real bugs.
Users *will* trip over these things.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From crutcher at gmail.com  Wed Jan 11 02:09:27 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Tue, 10 Jan 2006 17:09:27 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when
	used as global dictionaries
In-Reply-To: <d49fe110601101625x6df3bbffmc7d7d1fd307189b2@mail.gmail.com>
References: <d49fe110601101547m1dce16c5n63233e7752516f5d@mail.gmail.com>
	<20060111000844.GA11919@panix.com>
	<d49fe110601101625x6df3bbffmc7d7d1fd307189b2@mail.gmail.com>
Message-ID: <d49fe110601101709r3bc1b122j32d73596f32a0eee@mail.gmail.com>

This is the unittest that checks it. On trunk, the global and fall
through cases will fail, but they pass with the patch.

import unittest
from test import test_support

class DictSubclass(dict):
    get_notify_func = None
    def __getitem__(self, key):
        if self.get_notify_func:
            self.get_notify_func(key)
        return dict.__getitem__(self, key)

    set_notify_func = None
    def __setitem__(self, key, value):
        if self.set_notify_func:
            self.set_notify_func(key)
        return dict.__setitem__(self, key, value)

    del_notify_func = None
    def __delitem__(self, key):
        if self.del_notify_func:
            self.del_notify_func(key)
        return dict.__delitem__(self, key)

class DictSubclassExecTest(unittest.TestCase):
    def test_subclassexec_setlocal(self):
        d = DictSubclass()
        l = []
        d.set_notify_func = l.append
        exec "a = 1" in d
        d.set_notify_func = None
        self.assert_(d['a'] == 1)
        self.assert_('a' in l)

    def test_subclassexec_getlocal(self):
        d = DictSubclass()
        l = []
        d.get_notify_func = l.append
        exec "a = 1; a" in d
        d.get_notify_func = None
        self.assert_(d['a'] == 1)
        self.assert_('a' in l)

    def test_subclassexec_dellocal(self):
        d = DictSubclass()
        l = []
        d.del_notify_func = l.append
        exec "a = 1; del a" in d
        d.del_notify_func = None
        self.assert_(not d.has_key('a'))
        self.assert_('a' in l)

    def test_subclassexec_setglobal(self):
        d = DictSubclass()
        l = []
        d.set_notify_func = l.append
        exec "a = 1\ndef foo():\n\tglobal a\n\tpass" in d
        d.set_notify_func = None
        self.assert_(d['a'] == 1)
        self.assert_('a' in l)

    def test_subclassexec_getglobal(self):
        d = DictSubclass()
        l = []
        d.get_notify_func = l.append
        exec "a = 1; a\ndef foo():\n\tglobal a\n\tpass" in d
        d.get_notify_func = None
        self.assert_(d['a'] == 1)
        self.assert_('a' in l)

    def test_subclassexec_delglobal(self):
        d = DictSubclass()
        l = []
        d.del_notify_func = l.append
        exec "a = 1; del a\ndef foo():\n\tglobal a\n\tpass" in d
        d.del_notify_func = None
        self.assert_(not d.has_key('a'))
        self.assert_('a' in l)

    def test_subclassexec_getfallthrough(self):
        ld = DictSubclass()
        ll = []
        ld.get_notify_func = ll.append
        gd = DictSubclass()
        gl = []
        gd.get_notify_func = gl.append
        gd['a'] = 1
        exec 'b = a' in gd, ld
        gd.del_notify_func = None
        ld.del_notify_func = None
        self.assert_(ld['b'] == 1)
        self.assert_('a' in ll)
        self.assert_('a' in gl)


def test_main():
        test_support.run_unittest(
                DictSubclassExecTest,
        )

if __name__ == "__main__":
        test_main()



On 1/10/06, Crutcher Dunnavant <crutcher at gmail.com> wrote:
> 1402289
>
> On 1/10/06, Aahz <aahz at pythoncraft.com> wrote:
> > On Tue, Jan 10, 2006, Crutcher Dunnavant wrote:
> > >
> > > There is an inconsistancy in the way that dictionary subclasses behave
> > > when they are used as as namespaces in execs.
> > >
> > > Basically, while python 2.4 permits the usage of dictionary subclasses
> > > for local environments, it still bypasses the subclass functions and
> > > uses the C API for global environments. The attached patch (and
> > > unittest!) addresses this issue.
> >
> > Please submit the patch to SourceForge and report back with the SF ID.
> > --
> > Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/
> >
> > "19. A language that doesn't affect the way you think about programming,
> > is not worth knowing."  --Alan Perlis
> >
>
>
> --
> Crutcher Dunnavant <crutcher at gmail.com>
> monket.samedi-studios.com
>


--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com

From trentm at ActiveState.com  Wed Jan 11 02:16:01 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Tue, 10 Jan 2006 17:16:01 -0800
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <dq1gon$c9p$1@sea.gmane.org>
References: <dq1gon$c9p$1@sea.gmane.org>
Message-ID: <20060111011601.GF4721@activestate.com>

[Fredrik Lundh wrote]
> Can buildbot deal with custom test/validation scripts, and collect the output
> somewhere ?

Yes, it should be able to. However, it might change the part of the the
master.cfg file that defines the build steps from being trivial
(probably something like:

    building_python_factory = factory.GNUAutoconf(
        s(SVN, svnurl="http://svn.python.org/projects/python/trunk"),
        test=["make", "test"],
    )

) to something a little less trivial.

The standard GNUAutoconf class would have to be subclassed to add that
extra step ("make check_for_missing_docs"?). That probably would be
fairly straightforward. Neil and Martin would probably know better. I
don't have access to the buildbot setup.

Trent

-- 
Trent Mick
trentm at activestate.com

From skip at pobox.com  Wed Jan 11 02:54:19 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 10 Jan 2006 19:54:19 -0600
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ca471dc20601101638m59d1a662x18614df445d6a2e7@mail.gmail.com>
References: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<20060110154626.C003.JCARLSON@uci.edu>
	<20060111000310.GP18916@xs4all.nl>
	<ca471dc20601101638m59d1a662x18614df445d6a2e7@mail.gmail.com>
Message-ID: <17348.25931.469196.19964@montanaro.dyndns.org>


    Guido> I'm not sure what makes those particular examples less than real
    Guido> bugs.  Users *will* trip over these things.

Feel free to hack away at:

    http://wiki.python.org/moin/CrashingPython

I have no idea where (if at all) it should be linked from elsewhere on the
wiki.

Skip

From barry at python.org  Wed Jan 11 03:09:49 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 10 Jan 2006 21:09:49 -0500
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <e04bdf310601090908m4633eeb8t@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
Message-ID: <1136945389.10328.7.camel@geddy.wooz.org>

On Mon, 2006-01-09 at 14:08 -0300, Facundo Batista wrote:

> We can rewrite ConfigParser to store everything and write it back in
> the exact order it took it, with new values at the end of each section
> (or where the user inserted it), but it'll took a big rewrite of the
> module: right now it's using dictionaries and should use lists to
> achieve that behaviour.
> 
> What I wanted to add to the module was predicatibility: a very needed
> feature when you're writing test cases (and that's where I got bite).

It's also important when you do crazy things like keep your
configuration files in a revision control system.  That's something that
I do as much as possible.  In general it works great for some thing, and
pretty poorly for others, but when it works it makes keeping systems in
sync or backed up much easier.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060110/bb0509a7/attachment.pgp 

From facundobatista at gmail.com  Wed Jan 11 03:49:18 2006
From: facundobatista at gmail.com (Facundo Batista)
Date: Tue, 10 Jan 2006 23:49:18 -0300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <1136945389.10328.7.camel@geddy.wooz.org>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
	<1136945389.10328.7.camel@geddy.wooz.org>
Message-ID: <e04bdf310601101849h7f4e01beh@mail.gmail.com>

2006/1/10, Barry Warsaw <barry at python.org>:

> It's also important when you do crazy things like keep your
> configuration files in a revision control system.  That's something that

Good point, Barry.

So, unless somebody screams a blatant "no", I'll apply my patch to
gain predictability and reject the other one because, if further work
is made, it should be done towards a controlled position structure
(a.k.a. use lists instead of dictionaries).

Thanks.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From t-meyer at ihug.co.nz  Wed Jan 11 05:08:29 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed, 11 Jan 2006 17:08:29 +1300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <e04bdf310601090908m4633eeb8t@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
Message-ID: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz>

[Guido]

>> I think it's moot unless you also preserve comments. Ideally would be
>> something that prserved everything (ordering, blank lines, comments
>> etc.) from how it was read in. Modifying a value should keep its
>> position. Adding a value should add it to the end of the section it's
>> in (unless there are cases where ordering is important to the
>> semantics -- are there?).

[Facundo Batista]
> We can rewrite ConfigParser to store everything and write it back in
> the exact order it took it, with new values at the end of each section
> (or where the user inserted it), but it'll took a big rewrite of the
> module: right now it's using dictionaries and should use lists to
> achieve that behaviour.

It's not at all that complicated.  Simply running through the  
original file when writing gets you the original order and comments.   
This is simple to do (a simple example is in the SpamBayes source  
code in the OptionsClass.py module).

Remember that there has been a lot of discussion about how  
ConfigParser should work in the past; for example (ignoring c.l.p):

<http://mail.python.org/pipermail/python-dev/2004-October/049454.html>
<http://mail.python.org/pipermail/python-dev/2004-October/049527.html>
<http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1483518>

=Tony.Meyer

From ronaldoussoren at mac.com  Wed Jan 11 07:45:03 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 11 Jan 2006 07:45:03 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060110154626.C003.JCARLSON@uci.edu>
References: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<20060110154626.C003.JCARLSON@uci.edu>
Message-ID: <AC5CE7EB-A8B7-41E0-B31F-03382D0E828A@mac.com>


On 11-jan-2006, at 0:57, Josiah Carlson wrote:

>
> On a platform which doesn't see the current path as a readable  
> file, you
> get "Access is denied." on the redirection attempt.

On my osx box using python 2.4.2:

$ cat < .
cat: stdin: Is a directory

$ python < .
Bus error

And finally:
$ python -c 'execfile(".")'
Traceback (most recent call last):
   File "<string>", line 1, in ?
IOError: [Errno 21] Is a directory: '.'


It seems there is a bug somewhere. Annoyingly 'gdb -tty=. python'  
doesn't crash.

Ronald

>
> As always, I'm sure that reasonable patches which work around such
> segfaulting cases will be acceptable, though remember that it may  
> not be
> clear how to work around them.
>
>
>  - Josiah
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com


From theller at python.net  Wed Jan 11 07:59:03 2006
From: theller at python.net (Thomas Heller)
Date: Wed, 11 Jan 2006 07:59:03 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
Message-ID: <y81ntfm0.fsf@python.net>

Guido van Rossum <guido at python.org> writes:

> On 1/10/06, Delaney, Timothy (Tim) <tdelaney at avaya.com> wrote:
>> Guido van Rossum wrote:
>>
>> > On 1/10/06, Thomas Heller <theller at python.net> wrote:
>> >> I would like to suggest to include ctypes into core Python, starting
>> >> with the 2.5 release.
>> >
>> > On the one hand I agree that this is a useful module, popular, mature
>> > etc.
>> >
>> > On the other hand it breaks one of the most fundamental Python
>> > guidelines: if you get a core dump (segfault etc.) it's a bug in
>> > Python or in a 3rd party extension, not in *your* Python code. An
>> > exception would have to be made for any code that uses ctypes, as it
>> > is usually trivial to cause core dumps with ctypes (I'd venture it's
>> > hard to avoid them ;-).
>> >
>> > I don't expect this to count against including ctypes; but I do want
>> > it to be dealt with somehow!
>>
>> As was pointed out on c.l.py, the `dl` module suffers the exact same
>> problem (I don't know myself, as I've never used it). There are no
>> warnings about this in the `dl` module documentation.
>
> Good point. I think there should be such warnings though.
>
>> I can't see how it would be possible to guarantee that such a module
>> could not cause crashes.
>
> And I'm not asking for that.
>
>> I'm of the opinion that having a big red
>> warning at the top of the module documentation that this is a
>> contributed module, and incorrect use could cause segmentation
>> faults/crashes, etc would be sufficient.
>
> Works for me.

Another possibility would be to emit a warning when the module (dl or
ctypes, if included) is imported.

warnings.warn("Incorrect usage of this module may crash Python",
              RuntimeWarning, stacklevel=2)

Thomas


From nyamatongwe at gmail.com  Wed Jan 11 08:14:55 2006
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Wed, 11 Jan 2006 18:14:55 +1100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <u0cbyfa1.fsf@python.net>
References: <u0cbyfa1.fsf@python.net>
Message-ID: <50862ebd0601102314r228a0d6hd977eb431ee0ec5c@mail.gmail.com>

   Won't ctypes completely replace dl? dl provides only a small subset
of the functionality of ctypes and is very restricted in the set of
argument types allowed.

   Neil

From nnorwitz at gmail.com  Wed Jan 11 08:22:56 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 10 Jan 2006 23:22:56 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060110232324.GO18916@xs4all.nl>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
Message-ID: <ee2a432c0601102322i985ceddwcc097ae0c1593177@mail.gmail.com>

On 1/10/06, Thomas Wouters <thomas at xs4all.net> wrote:
>
> centurion:~ > python < .
> Segmentation fault

I fixed that in Oct in head and 2.4 branch.  Although Skip filed a bug
since Py_FatalError() is called which generates a core dump in debug
builds at least.  http://python.org/sf/1353504

I'm not sure what else can be done except perhaps just print to stderr
and exit() rather than abort().

> >>> sys.setrecursionlimit(1<<30)
> >>> f = lambda f:f(f)
> >>> f(f)
> Segmentation fault

Added as Lib/test/crashers/recursive_call.py

I'm sure there are many more similar to this one though.

> There's more, all from Python itself.

We should at least try to clearly document limitations such as these. 
That's why I added the crashers directory.  Maybe we can even find
volunteers to fix these.

n

From nnorwitz at gmail.com  Wed Jan 11 08:49:23 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 10 Jan 2006 23:49:23 -0800
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <20060111011601.GF4721@activestate.com>
References: <dq1gon$c9p$1@sea.gmane.org>
	<20060111011601.GF4721@activestate.com>
Message-ID: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>

On 1/10/06, Trent Mick <trentm at activestate.com> wrote:
> [Fredrik Lundh wrote]
> > Can buildbot deal with custom test/validation scripts, and collect the output
> > somewhere ?
>
> Yes, it should be able to. However, it might change the part of the the
> master.cfg file that defines the build steps from being trivial
> to something a little less trivial.
>
> The standard GNUAutoconf class would have to be subclassed to add that
> extra step ("make check_for_missing_docs"?). That probably would be
> fairly straightforward. Neil and Martin would probably know better. I
> don't have access to the buildbot setup.

It looks like we could define a class similar to Test, such as:

class Catalog(ShellCommand):
    name = "catalog"
    warnOnFailure = 0 # this was 1, not sure which we want
    description = ["catalog"]
    descriptionDone = ["catalog"]
    command = ["/f cmd here", "and", "other", "flags"]

The factory is created like this:
        f = factory.GNUAutoconf(source, test=["make","testall"])

ISTM, we can just add this line:
       f.steps.append(s(Catalog, command=Catalog.command))

Does that make sense?  We would just need /f's script in SVN.

n

PS  If there was a method on ShellCommand, adding a step would be simpler, e.g.,

  def addStep(self, cls, **kwds):
    self.steps.append((cls, kwds))

then we could do: f.addStep(Catalog, command=Catalog.command)
which would simplify a bunch of code in process/factory.py since it is
called this way 10 times.  Didn't look elsewhere.

From warner at lothar.com  Wed Jan 11 09:36:09 2006
From: warner at lothar.com (Brian Warner)
Date: Wed, 11 Jan 2006 00:36:09 -0800 (PST)
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <20060107001221.GA13845@activestate.com>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
Message-ID: <20060111.003609.21623845.warner@lothar.com>

> > > To wipe out the build occassionally you could (presumably) add a
> > > starting step to the Python 'builder' (in the build master.cfg) to 
> > >     rm -rf $builddir
> > > every, say, Sunday night.

That would work, although to be honest the buildbot is more about repeatable
builds. My first suggestion would be to have two separate Builders, one of
which does incremental builds, the other which does full (from-scratch)
builds. Both modes are useful to test: I've seen build problems that affected
one but not the other (deleted source files that leave .o files in place,
Makefile changes that don't notice modified files correctly, etc).

If you want to minimize the compile load placed on the build slaves, you
could use the Schedulers to reduce the frequency of triggering the "full"
builder. A subclass of the default Scheduler class could enforce a
minimum-time-between-builds timer, and you could crank it up to only allow
one full build per hour, or per day, or whatever.

The reason I'd encourage you to let every build of a given Builder behave the
same way is that I hope to add more problem-tracking code to the Buildbot,
and that code will work better if builds really are reproducible and stable.
If the tests pass in build #9 and fail in build #10, you should be able to
blame the failure on the changes that went into build #10. If instead the
failure is due to the fact that build #10 just happened to be the one where
you do a clean build instead of an incremental one, then it will be harder to
automatically isolate the cause of the failure.

It isn't a big issue, though. Just thought I'd share what my design
philosophy is so you can decide which direction you want to go. The code is
flexible enough to handle it.

> The current problem with this is that I don't know if the build step
> objects have access to a local data store -- so it could, say, count how
> many builds have been done to know to clobber every tenth one. The
> current code just chooses to clobber for any build on Sunday. Not a very
> satisfying solution.
> 
> Brian,
> Is there a way for build steps to maintain some state?

In general, no.. the long-term status is kept in a separate BuildStatus
object, the idea is that BuildStatus gets persisted but Builds do not. You
can always have your Build subclass use the regular status interface to look
up an earlier BuildStatus, and base some decisions upon the data you find
there, although I haven't really provided a way to set arbitrary attributes
on the BuildStatus.

For this case, however, you can have the Build reach into its BuildStatus
object and look at the .number attribute, and then do a simple %10==0 test.
Not especially elegant but it would get the job done. From a BuildStep you
would use 'self.build.build_status.number % 10 == 0'.

Apart from that, your SVNEx subclass seems feasible. I'd be inclined to have
the subclass always implement this update_and_clobber_occassionally behavior
(rather than providing a special mode= switch for it), but that's just a
usage detail.

cheers,
 -Brian

From warner at lothar.com  Wed Jan 11 09:39:50 2006
From: warner at lothar.com (Brian Warner)
Date: Wed, 11 Jan 2006 00:39:50 -0800 (PST)
Subject: [Python-Dev] Buildbot questions
In-Reply-To: <20060107001651.GB13845@activestate.com>
References: <20060107001221.GA13845@activestate.com>
	<43BF07C1.3060608@v.loewis.de>
	<20060107001651.GB13845@activestate.com>
Message-ID: <20060111.003950.48501461.warner@lothar.com>

> > Ah, but that would require changes to the slaves, right? I would
> > prefer a solution that avoids that.
> 
> I don't think so. In my little test setup I didn't have to make any
> change to the slave.

The "update" and "clobber" mode parameters are implemented on the slave side.
Trent's patch changes which mode setting is used on the master side before
anything is sent to the slaves, so the slaves don't have to know anything
about it.

In general there is as little slave-side code as possible, to reduce the need
to upgrade or change the code on that side. The VC checkout/update operations
are an exception, since they may do several operations in a row (which would
mean a lot more latency to do them without support from code on the slave
side), and because I'm lazy and prefer to implement the algorithms in Python
instead of in Bourne shell :).

 -Brian


From warner at lothar.com  Wed Jan 11 10:24:29 2006
From: warner at lothar.com (Brian Warner)
Date: Wed, 11 Jan 2006 01:24:29 -0800 (PST)
Subject: [Python-Dev] buildbot
In-Reply-To: <43C36D3C.7050202@v.loewis.de>
References: <43BD9B32.3030700@v.loewis.de>
	<20060108.173810.91561560.warner@lothar.com>
	<43C36D3C.7050202@v.loewis.de>
Message-ID: <20060111.012429.45715174.warner@lothar.com>

> The reason I want static pages is for security concerns. It is not
> easy whether buildbot can be trusted to have no security flaws,
> which might allow people to start new processes on the master,
> or (perhaps worse) on any of the slaves.

These are excellent points. While it would take a complete security audit to
satisfy the kind of paranoia *I* tend to carry around, I can mention a couple
of points about the buildbot's security design that might help you make some
useful decisions about it:

 The buildmaster "owns" (spelled "pwns" or "0wnz0red" these days, according
 to my "leet-speak phrasebook" :) the buildslaves. It can make them run
 whatever shell commands it likes, therefore it has full control of the
 buildslave accounts. It is appropriate to give the buildslaves their own
 account, with limited privileges.

 The codebase "owns" the buildslaves too: most build commands will wind up
 running './configure' or 'make' or something which executes commands that
 are provided by the checked out source tree.

 Nobody is supposed to "own" the buildmaster: it performs build scheduling
 and status reporting according to its own design and configuration file. A
 compromised codebase cannot make the buildmaster do anything unusual, nor
 can a compromised buildslave. The worst that a buildslave can do is cause a
 DoS attack by sending overly large command-status messages, which can
 prevent the buildmaster from doing anything useful (in the worst case
 causing it to run out of memory), but cannot make it do anything it isn't
 supposed to.

 The top-level IRC functions can be audited by inspecting the command_
 methods, as you've already seen.

 The HTTP status page can be audited similarly, once you know how Twisted-Web
 works (there is a hierarchy of Resource objects, each component of the URL
 path uses Resource.getChild to obtain the child node in this tree, once the
 final child is retrieved then the 'render' method is called to produce
 HTML). The Waterfall resource and all its children get their capabilities
 from two objects: Status (which provides read-only status information about
 all builds) and Control (which is the piece that allows things like "force
 build"). The knob that disables the "Force Build" button does so by creating
 the Waterfall instance with control=None. If you can verify that the code
 doesn't go out of its way to acquire a Control reference through some
 private-use-only attribute, then you can be reasonably confident that it
 isn't possible to make the web server do anything to trigger a build. It's
 not restricted-execution mode or anything, but it's designed with
 capability-based security in mind, and that may help someone who wishes to
 audit it.

 The PBListener status interface is similar: PB guarantees that only remote_*
 methods can be invoked by a remote client, and the PBListener object only
 has a reference to the top-level Status object.

 The slave->master connection (via the 'slaveport') uses PB, so it can be
 audited the same way. Only the remote_* (and perspective_*) methods of
 objects which are provided to the buildslave can be invoked. The buildslaves
 are allowed to do two things to the top-level buildmaster: force a build
 that is run on their own machine, and invoke an empty 'keepalive' method.
 During a build, they can send remote_update and remote_complete messages to
 the current BuildStep: this is how they deliver status information
 (specifically the output of shell commands). By inspecting
 buildbot.process.step.RemoteCommand.remote_update, you can verify that the
 update is appended to a logfile and nothing else.

 PB's serialization is designed specifically to be safe (in explicit contrast
 to pickle). Arbitrary classes cannot be sent over the wire. The worst-case
 attack is DoS, specifically memory exhaustion.

Any application which can talk to the outside world is a security concern.
The tools that we have to insure that these applications only do what we
intended them to do are not as fully developed as we would like (I hang out
with the developers of E, and would love to implement the buildbot in a
capability-secure form of Python, but such a beast is not available right
now, and I'm spending too much time writing Buildbot code to get around to
writing a more secureable language too). So we write our programs in as clear
a way as possible, and take advantage of tools that have been developed or
inspected by people we respect.

These days my paranoia tells me to trust a webserver written in Python more
than one written in C. Buffer overruns are the obvious thing, but another
important distinction is how Twisted's web server architecture treats the URL
as a path of edges in a tree of Resource instances rather than as a pathname
to a file on the disk. I don't need to worry about what kind of URLs might
give access to the master.cfg file (which could contain debugging passwords
or something), as long as I can tell that none of the Resource instances give
access to it. This also makes preventing things like
http://foo/../../oops.txt much much easier.

Preferring a Twisted web server over Apache reveals my bias, both in favor of
Python and the developers of Twisted and Twisted's web server, and I quite
understand if you don't share that bias. I think it would be quite possible
to create a 'StaticWaterfall' status class, which would write HTML to a tree
of files each time something changed. There are a number of status event
delivery APIs in the buildbot which could cause a method to be called each
time a Step was started or finished, and these could just write new HTML to a
file. It would consume a bit more disk space, but would allow an external
webserver to provide truly read-only access to build status. If you'd like me
to spend some cycles on this, please let me know.. perhaps others would
prefer this style of status delivery too.


cheers,
 -Brian

From mwh at python.net  Wed Jan 11 12:56:32 2006
From: mwh at python.net (Michael Hudson)
Date: Wed, 11 Jan 2006 11:56:32 +0000
Subject: [Python-Dev] test_curses
In-Reply-To: <dq10k9$ctq$1@sea.gmane.org> (Georg Brandl's message of "Tue,
	10 Jan 2006 20:06:48 +0100")
References: <dprnoe$p98$1@sea.gmane.org> <2m1wzhsqq5.fsf@starship.python.net>
	<dpubjo$buj$1@sea.gmane.org> <2m1wzgqvh5.fsf@starship.python.net>
	<dq10k9$ctq$1@sea.gmane.org>
Message-ID: <2mslrvrn9r.fsf@starship.python.net>

Georg Brandl <g.brandl-nospam at gmx.net> writes:

>> Well, this still has the faint whiff of impossibility about it.  Are
>> you sure it's setupterm() that's doing the damage?  Can you reproduce
>> interactively?
>
> Yep.
> Alone, the setupterm call [curses.setupterm(sys.__stdout__.fileno())] does
> nothing remarkable, but when it is done inside of curses.initscr() / curses.endwin()
> the terminal is not restored properly.

Ah!  That does indeed make some small amount of sense.

> If setupterm() is to be tested, I think it should be executed before initscr().

Sounds good.  What are you waiting for? :)

Cheers,
mwh

-- 
  <etrepum> Jokes around here tend to get followed by implementations.
                                                -- from Twisted.Quotes

From barry at python.org  Wed Jan 11 13:59:50 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jan 2006 07:59:50 -0500
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <y81ntfm0.fsf@python.net>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
	<y81ntfm0.fsf@python.net>
Message-ID: <1136984390.12149.6.camel@geddy.wooz.org>

On Wed, 2006-01-11 at 07:59 +0100, Thomas Heller wrote:

> Another possibility would be to emit a warning when the module (dl or
> ctypes, if included) is imported.
> 
> warnings.warn("Incorrect usage of this module may crash Python",
>               RuntimeWarning, stacklevel=2)

BTW, although I'm pretty sure the answer is "no" (at least, I hope it
is), is anyone aware of a situation where the mere importation of a
module can cause Python to crash?

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060111/0eedc77f/attachment.pgp 

From thomas at xs4all.net  Wed Jan 11 14:54:40 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 11 Jan 2006 14:54:40 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <1136984390.12149.6.camel@geddy.wooz.org>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
	<y81ntfm0.fsf@python.net> <1136984390.12149.6.camel@geddy.wooz.org>
Message-ID: <20060111135440.GQ18916@xs4all.nl>

On Wed, Jan 11, 2006 at 07:59:50AM -0500, Barry Warsaw wrote:

> BTW, although I'm pretty sure the answer is "no" (at least, I hope it
> is), is anyone aware of a situation where the mere importation of a
> module can cause Python to crash?

Well, I assume you aren't importing any 'hostile' code, nor running in an
uncontrolled environment so I guess you mean other than running out of
memory, or the module you are importing actually executing one of the ways
to crash Python? Or the module being an extension module that crashes on
import? Or another extension module having corrupted the Python environment
to a point where a simple import crashes Python? Or a non-extension module
using one of the vulnerabilities (in, say, marshal, or pickle) to corrupt
the Python environment? Or stuff in os.environ, like LD_* variables, that
interfere with library linking? Ponder, ponder, no, can't think of any. :)

The pickle vulnerability came up last year, when someone on #python was
subclassing a builtin type (string or dict, I think the latter) that was
using a magical invocation of (IIRC) __new__ on unpickle. The subclassed
__new__ didn't handle this right, so the baseclass __new__ wasn't getting
called right, and the new object's addressspace was not initialized. This
lead to crashes. I don't remember the details exactly, and my continuous
advice of not subclassing builtin types unless you know what you're doing
solved the issue (there was no actual need to subclass, there), and I have
no idea whether that specific issue was solved or not, but I'm trying to
find it again :)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From barry at python.org  Wed Jan 11 15:02:50 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 11 Jan 2006 09:02:50 -0500
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060111135440.GQ18916@xs4all.nl>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
	<y81ntfm0.fsf@python.net> <1136984390.12149.6.camel@geddy.wooz.org>
	<20060111135440.GQ18916@xs4all.nl>
Message-ID: <1136988170.12149.12.camel@geddy.wooz.org>

On Wed, 2006-01-11 at 14:54 +0100, Thomas Wouters wrote:
> On Wed, Jan 11, 2006 at 07:59:50AM -0500, Barry Warsaw wrote:
> 
> > BTW, although I'm pretty sure the answer is "no" (at least, I hope it
> > is), is anyone aware of a situation where the mere importation of a
> > module can cause Python to crash?
> 
> Well, I assume you aren't importing any 'hostile' code, nor running in an
> uncontrolled environment so I guess you mean other than running out of
> memory, or the module you are importing actually executing one of the ways
> to crash Python? 

Correct.

> Or the module being an extension module that crashes on
> import? Or another extension module having corrupted the Python environment
> to a point where a simple import crashes Python? Or a non-extension module
> using one of the vulnerabilities (in, say, marshal, or pickle) to corrupt
> the Python environment? Or stuff in os.environ, like LD_* variables, that
> interfere with library linking? Ponder, ponder, no, can't think of any. :)

Let's keep it to modules in the standard library, although that includes
extension modules, and let's further say that it's a controlled enough
environment that you won't have stray evil modules floating around in
your sys.path.

> The pickle vulnerability came up last year, when someone on #python was
> subclassing a builtin type (string or dict, I think the latter) that was
> using a magical invocation of (IIRC) __new__ on unpickle. The subclassed
> __new__ didn't handle this right, so the baseclass __new__ wasn't getting
> called right, and the new object's addressspace was not initialized. This
> lead to crashes. I don't remember the details exactly, and my continuous
> advice of not subclassing builtin types unless you know what you're doing
> solved the issue (there was no actual need to subclass, there), and I have
> no idea whether that specific issue was solved or not, but I'm trying to
> find it again :)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060111/0d8a82fb/attachment.pgp 

From mwh at python.net  Wed Jan 11 15:03:48 2006
From: mwh at python.net (Michael Hudson)
Date: Wed, 11 Jan 2006 14:03:48 +0000
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ca471dc20601101638m59d1a662x18614df445d6a2e7@mail.gmail.com>
	(Guido van Rossum's message of "Tue, 10 Jan 2006 16:38:56 -0800")
References: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<20060110154626.C003.JCARLSON@uci.edu>
	<20060111000310.GP18916@xs4all.nl>
	<ca471dc20601101638m59d1a662x18614df445d6a2e7@mail.gmail.com>
Message-ID: <2moe2isvy3.fsf@starship.python.net>

Guido van Rossum <guido at python.org> writes:

> On 1/10/06, Thomas Wouters <thomas at xs4all.net> wrote:
>> Sorry, I missed the point I was aiming at, I guess. I wasn't aiming for
>> fixable bugs; I see these things as, with great effort, holding up your foot
>> at an awkward angle so that it ends up right at the business end of your
>> primed and lit medieval cannon. We can jump through hoops to stop those
>> particular examples, but I'd rather spend time and effort to fix *real* bugs
>> instead.
>
> I'm not sure what makes those particular examples less than real bugs.

I think the sys.setrecursionlimit(1<<30) one is fairly well into the
terriotory of "don't do that", isn't it?

Cheers,
mwh

-- 
  <exarkun> today's lesson
  <exarkun> don't strace X in an xterm
                                                -- from Twisted.Quotes

From thomas at xs4all.net  Wed Jan 11 15:20:04 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 11 Jan 2006 15:20:04 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <20060111135440.GQ18916@xs4all.nl>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
	<y81ntfm0.fsf@python.net> <1136984390.12149.6.camel@geddy.wooz.org>
	<20060111135440.GQ18916@xs4all.nl>
Message-ID: <20060111142004.GR18916@xs4all.nl>

On Wed, Jan 11, 2006 at 02:54:40PM +0100, Thomas Wouters wrote:

> The pickle vulnerability came up last year, when someone on #python was
> subclassing a builtin type (string or dict, I think the latter) that was
> using a magical invocation of (IIRC) __new__ on unpickle. The subclassed
> __new__ didn't handle this right, so the baseclass __new__ wasn't getting
> called right, and the new object's addressspace was not initialized. This
> lead to crashes. I don't remember the details exactly, and my continuous
> advice of not subclassing builtin types unless you know what you're doing
> solved the issue (there was no actual need to subclass, there), and I have
> no idea whether that specific issue was solved or not, but I'm trying to
> find it again :)

Ah, found it: it was one of the datetime types. It has guards in place (some
back then, python2.3, more since 2.4) so I wasn't able to figure out why it
actually crashed Python, rather than produce a weird date. I couldn't find
anything obviously wrong with the data's handling (although the extra guards
are good.) I'll see if I can reproduce it anyway.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From skip at pobox.com  Wed Jan 11 15:37:49 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 11 Jan 2006 08:37:49 -0600
Subject: [Python-Dev] Limiting the recursion limit
In-Reply-To: <ee2a432c0601102322i985ceddwcc097ae0c1593177@mail.gmail.com>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<ee2a432c0601102322i985ceddwcc097ae0c1593177@mail.gmail.com>
Message-ID: <17349.6205.976505.883903@montanaro.dyndns.org>


    >>> sys.setrecursionlimit(1<<30)
    >>> f = lambda f:f(f)
    >>> f(f)
    Segmentation fault

Is there some way that Python can determine that 1<<30 is an unreasonable
recursion limit?

Skip

From guido at python.org  Wed Jan 11 18:34:28 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 11 Jan 2006 09:34:28 -0800
Subject: [Python-Dev] Limiting the recursion limit
In-Reply-To: <17349.6205.976505.883903@montanaro.dyndns.org>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<20060110232324.GO18916@xs4all.nl>
	<ee2a432c0601102322i985ceddwcc097ae0c1593177@mail.gmail.com>
	<17349.6205.976505.883903@montanaro.dyndns.org>
Message-ID: <ca471dc20601110934h4fef7d51y7716b50938d0db34@mail.gmail.com>

On 1/11/06, skip at pobox.com <skip at pobox.com> wrote:
>
>     >>> sys.setrecursionlimit(1<<30)
>     >>> f = lambda f:f(f)
>     >>> f(f)
>     Segmentation fault
>
> Is there some way that Python can determine that 1<<30 is an unreasonable
> recursion limit?

Yes, but that doesn't help -- there's some value in the middle where
you may or may not get a crash depending on random other stuff that is
going on, and the only way to know is to try. It's easy enough to code
a loop that tries higher and higher values until it finds a crash.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ronaldoussoren at mac.com  Wed Jan 11 19:04:51 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Wed, 11 Jan 2006 19:04:51 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <y81ntfm0.fsf@python.net>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
	<y81ntfm0.fsf@python.net>
Message-ID: <C0A77247-2490-4742-AEDD-32D5F6B3BCF6@mac.com>


On 11-jan-2006, at 7:59, Thomas Heller wrote:
>>
>>> I'm of the opinion that having a big red
>>> warning at the top of the module documentation that this is a
>>> contributed module, and incorrect use could cause segmentation
>>> faults/crashes, etc would be sufficient.
>>
>> Works for me.
>
> Another possibility would be to emit a warning when the module (dl or
> ctypes, if included) is imported.
>
> warnings.warn("Incorrect usage of this module may crash Python",
>               RuntimeWarning, stacklevel=2)

Yuck. Just adding a warning to the documentation should be good  
enough. Other modules allow you to make your machine unuseable (such  
a fork bomb using os.fork) and those don't warn either.

Ronald

>
> Thomas
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com


From nick at craig-wood.com  Wed Jan 11 19:16:41 2006
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Wed, 11 Jan 2006 18:16:41 +0000
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <y81ntfm0.fsf@python.net>
References: <2773CAC687FD5F4689F526998C7E4E5F074323@au3010avexu1.global.avaya.com>
	<ca471dc20601101512y1a148af0q791d1462c2d416b3@mail.gmail.com>
	<y81ntfm0.fsf@python.net>
Message-ID: <20060111181641.GA8149@craig-wood.com>

On Wed, Jan 11, 2006 at 07:59:03AM +0100, Thomas Heller wrote:
> Another possibility would be to emit a warning when the module (dl or
> ctypes, if included) is imported.
> 
> warnings.warn("Incorrect usage of this module may crash Python",
>               RuntimeWarning, stacklevel=2)

Arrgggh!  No!!  Warnings are one of the things I really hate about
perl and I'd hate to see them infecting python any more than they
already have.

I want my programming language to tell me, the programmer, about stuff
not go blurting to the user behind my back.

Death to warnings ;-)

(Yes I know about the warnings module!)
-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick

From turnbull at sk.tsukuba.ac.jp  Sun Jan  8 17:10:50 2006
From: turnbull at sk.tsukuba.ac.jp (Stephen J. Turnbull)
Date: Mon, 09 Jan 2006 01:10:50 +0900
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dpr983$95r$1@sea.gmane.org> (Fredrik Lundh's message of "Sun,
	8 Jan 2006 15:57:06 +0100")
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dpr983$95r$1@sea.gmane.org>
Message-ID: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Fredrik" == Fredrik Lundh <fredrik at pythonware.com> writes:

    Fredrik> many test frameworks support "expected failures" for this
    Fredrik> purpose.  how hard would it be to add a

    Fredrik>     unittest.FailingTestCase

    Fredrik> class that runs a TestCase, catches any errors in it, and
    Fredrik> signals an error ("test foo passed unexpectedly") if it
    Fredrik> runs cleanly ?

One can do even better than that.  unittest.FailingTestCase should
(except possibly for platform dependencies) know _how_ the TestCase is
expected to fail.  You also want to know if the error changes.


-- 
Graduate School of Systems and Information Engineering   University of Tsukuba
http://turnbull.sk.tsukuba.ac.jp/        Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
        Economics of Information Communication and Computation Systems
          Experimental Economics, Microeconomic Theory, Game Theory

From stephendavis at mac.com  Tue Jan 10 18:18:59 2006
From: stephendavis at mac.com (Stephen Davis)
Date: Tue, 10 Jan 2006 09:18:59 -0800
Subject: [Python-Dev] [Buildbot-devel] Re:  buildbot
In-Reply-To: <43C36D3C.7050202@v.loewis.de>
References: <20060104.003058.48524852.warner@lothar.com>
	<43BD9B32.3030700@v.loewis.de>
	<20060108.173810.91561560.warner@lothar.com>
	<43C36D3C.7050202@v.loewis.de>
Message-ID: <EA8E416C-607E-4D96-8789-3C323C4660DD@mac.com>

> The reason I want static pages is for security concerns. It is not
> easy whether buildbot can be trusted to have no security flaws,
> which might allow people to start new processes on the master,
> or (perhaps worse) on any of the slaves.

I have security concerns as well, but not in buildbot itself.  My  
project is restricted even withinz the company I work for so I need  
the buildbot web server to only be available to certain people.   
HTTPS access would be nice too.  TwistedWeb doesn't seem to have  
support for either HTTPS or authentication so I've been forced to  
"hide" it by putting it on a non-standard port.  Very weak.

I am no networking expert so the suggestions for using a reverse  
proxy are very welcome and I will look into that right away.  Just  
wanted to add my voice to the security concerns.

stephen

From trentm at ActiveState.com  Wed Jan 11 20:59:00 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 11 Jan 2006 11:59:00 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dpr983$95r$1@sea.gmane.org>
	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <20060111195900.GB17269@activestate.com>

[Stephen J. Turnbull wrote]
> >>>>> "Fredrik" == Fredrik Lundh <fredrik at pythonware.com> writes:
> 
>     Fredrik> many test frameworks support "expected failures" for this
>     Fredrik> purpose.  how hard would it be to add a
> 
>     Fredrik>     unittest.FailingTestCase
> 
>     Fredrik> class that runs a TestCase, catches any errors in it, and
>     Fredrik> signals an error ("test foo passed unexpectedly") if it
>     Fredrik> runs cleanly ?
> 
> One can do even better than that.  unittest.FailingTestCase should
> (except possibly for platform dependencies) know _how_ the TestCase is
> expected to fail.  You also want to know if the error changes.

How about this:

    http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/307970
    a better assertRaises() for unittest.py 

    When writing unit tests for Python using the standard unittest.py
    system the assertRaises() (aka failUnlessRaises()) method is used to
    test that a particular call raises the given exception. This recipe
    if for assertRaisesEx() that adds three things: (1) the ability to
    assert the raised exception's args; (2) the ability to test that the
    stringified exception matches a given regular expression; and (3)
    much better failure messages.

I haven't read this thread, so apologies is this doesn't really apply to
the discussion.

Cheers,
Trent

-- 
Trent Mick
trentm at activestate.com

From trentm at ActiveState.com  Wed Jan 11 21:05:28 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 11 Jan 2006 12:05:28 -0800
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>
References: <dq1gon$c9p$1@sea.gmane.org>
	<20060111011601.GF4721@activestate.com>
	<ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>
Message-ID: <20060111200528.GA23752@activestate.com>

[Neal Norwitz wrote]
> > [Fredrik Lundh wrote]
> > > Can buildbot deal with custom test/validation scripts, and collect the output
> > > somewhere ?
> >
> ...
> It looks like we could define a class similar to Test, such as:
> 
> class Catalog(ShellCommand):
>     name = "catalog"
>     warnOnFailure = 0 # this was 1, not sure which we want
>     description = ["catalog"]
>     descriptionDone = ["catalog"]
>     command = ["/f cmd here", "and", "other", "flags"]
> 
> The factory is created like this:
>         f = factory.GNUAutoconf(source, test=["make","testall"])
> 
> ISTM, we can just add this line:
>        f.steps.append(s(Catalog, command=Catalog.command))
> 
> Does that make sense?  We would just need /f's script in SVN.

Yup. Sounds right to me.



> PS  If there was a method on ShellCommand, adding a step would be simpler, e.g.,
> 
>   def addStep(self, cls, **kwds):
>     self.steps.append((cls, kwds))
> 
> then we could do: f.addStep(Catalog, command=Catalog.command)
> which would simplify a bunch of code in process/factory.py since it is
> called this way 10 times.  Didn't look elsewhere.

Add a buildbot patch/bug?
    http://sourceforge.net/tracker/?func=add&group_id=73177&atid=537003



Trent

-- 
Trent Mick
trentm at activestate.com

From martin at v.loewis.de  Wed Jan 11 21:06:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Jan 2006 21:06:59 +0100
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <20060111.003609.21623845.warner@lothar.com>
References: <20060106173110.GD15276@activestate.com>	<43BEE8F4.5080304@v.loewis.de>	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
Message-ID: <43C56563.9060107@v.loewis.de>

Brian Warner wrote:
> That would work, although to be honest the buildbot is more about repeatable
> builds. My first suggestion would be to have two separate Builders, one of
> which does incremental builds, the other which does full (from-scratch)
> builds.

My concern is that then the number of builders would go up from 8 to 16,
even though it is just four different machines that we use.

Regards,
Martin

From trentm at ActiveState.com  Wed Jan 11 21:45:33 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 11 Jan 2006 12:45:33 -0800
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <43C56563.9060107@v.loewis.de>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
	<43C56563.9060107@v.loewis.de>
Message-ID: <20060111204533.GB30010@activestate.com>

[Martin v. Loewis wrote]
> Brian Warner wrote:
> > That would work, although to be honest the buildbot is more about repeatable
> > builds. My first suggestion would be to have two separate Builders, one of
> > which does incremental builds, the other which does full (from-scratch)
> > builds.
> 
> My concern is that then the number of builders would go up from 8 to 16,
> even though it is just four different machines that we use.

Specifically are you concerned about the readability of the waterfall
page or other things (maintainability or something)? If the former,
perhaps we could get some mileage out of the query args that Brian
mentioned:

    [Brian Warner wrote]
    > I should mention a feature that you might find useful: you can add
    > some query arguments to the top-level Waterfall page's URL to
    > restrict the set of Builders that are displayed. If you append
    > "?show=full-2.3", you'll wind up with a waterfall display that
    > only contains the "full-2.3" Builder's column.  Appending
    > "?show=full-2.3&show=full-2.4" will show two columns, etc. You can
    > also create multiple Waterfall instances, each with a different
    > set of "categories", and use that to manage the complexity of
    > displaying status for lots of Builders. (with the current release
    > these multiple Waterfalls must all listen on different TCP ports,
    > unfortunately, but you can always hide this with the
    > reverse-proxy's URL mapping).


Trent

-- 
Trent Mick
trentm at activestate.com

From skip at pobox.com  Wed Jan 11 21:53:50 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 11 Jan 2006 14:53:50 -0600
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <20060111204533.GB30010@activestate.com>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
	<43C56563.9060107@v.loewis.de>
	<20060111204533.GB30010@activestate.com>
Message-ID: <17349.28766.364593.662110@montanaro.dyndns.org>


    Trent> Specifically are you concerned about the readability of the
    Trent> waterfall page or other things (maintainability or something)? If
    Trent> the former, perhaps we could get some mileage out of the query
    Trent> args that Brian mentioned:

One concern might be performance.  All buildbot slaves are contributed
hardware.  I don't mind the load on my Mac (it's a dual processor g5), but
it may be an issue for other people.  Does/can buildbot run at a lower
priority or is it just a matter of nice-ing the slave process at startup?

Skip

From nnorwitz at gmail.com  Wed Jan 11 22:02:03 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 11 Jan 2006 13:02:03 -0800
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <17349.28766.364593.662110@montanaro.dyndns.org>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
	<43C56563.9060107@v.loewis.de>
	<20060111204533.GB30010@activestate.com>
	<17349.28766.364593.662110@montanaro.dyndns.org>
Message-ID: <ee2a432c0601111302v2845bf60s59c56dbe666b0c64@mail.gmail.com>

On 1/11/06, skip at pobox.com <skip at pobox.com> wrote:
>
> One concern might be performance.  All buildbot slaves are contributed
> hardware.  I don't mind the load on my Mac (it's a dual processor g5), but
> it may be an issue for other people.

I've "contributed" 2 machines.  One is my personal box, the other is
the PSFs.  The PSF box is very lightly used except by automated stuff,
so it's not a problem.  My box is a dual opteron and it really isn't a
problem for me right now.  I suppose if it was building all the time,
I might mind, but so far, there's no issues.

Hopefully over time we will get more people contributing other boxes
and this might become a bigger issue.  But right now I don't think
it's a problem.

n

From trentm at ActiveState.com  Wed Jan 11 22:25:30 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 11 Jan 2006 13:25:30 -0800
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <17349.28766.364593.662110@montanaro.dyndns.org>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
	<43C56563.9060107@v.loewis.de>
	<20060111204533.GB30010@activestate.com>
	<17349.28766.364593.662110@montanaro.dyndns.org>
Message-ID: <20060111212530.GA6642@activestate.com>

[skip at pobox.com wrote]
> 
>     Trent> Specifically are you concerned about the readability of the
>     Trent> waterfall page or other things (maintainability or something)? If
>     Trent> the former, perhaps we could get some mileage out of the query
>     Trent> args that Brian mentioned:
> 
> One concern might be performance.  All buildbot slaves are contributed
> hardware.  I don't mind the load on my Mac (it's a dual processor g5), but
> it may be an issue for other people.  Does/can buildbot run at a lower
> priority or is it just a matter of nice-ing the slave process at startup?

As (I think) Brian mentioned, the scheduler for the (proposed) "full
builder" could be setup to just build, say, once per day or only a couple
of times per day where as the "incremental builder" would trigger for
every SVN checkin.

Nicing could probably also be done.

Trent

-- 
Trent Mick
trentm at activestate.com

From fredrik at pythonware.com  Wed Jan 11 22:45:06 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 11 Jan 2006 22:45:06 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
References: <dq1gon$c9p$1@sea.gmane.org><20060111011601.GF4721@activestate.com>
	<ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>
Message-ID: <dq3u93$s4g$1@sea.gmane.org>

Neal Norwitz wrote:

> Does that make sense?  We would just need /f's script in SVN.

in python/Tools/something or sandbox/something ?

</F>




From martin at v.loewis.de  Wed Jan 11 22:50:48 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Jan 2006 22:50:48 +0100
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <17349.28766.364593.662110@montanaro.dyndns.org>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
	<43C56563.9060107@v.loewis.de>
	<20060111204533.GB30010@activestate.com>
	<17349.28766.364593.662110@montanaro.dyndns.org>
Message-ID: <43C57DB8.7060702@v.loewis.de>

skip at pobox.com wrote:
> One concern might be performance.  All buildbot slaves are contributed
> hardware.  I don't mind the load on my Mac (it's a dual processor g5), but
> it may be an issue for other people.  Does/can buildbot run at a lower
> priority or is it just a matter of nice-ing the slave process at startup?

nice-ing the slave process at startup would be the way to do it, right?

Regards,
Martin

From martin at v.loewis.de  Wed Jan 11 22:53:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Jan 2006 22:53:20 +0100
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <20060111204533.GB30010@activestate.com>
References: <20060106173110.GD15276@activestate.com>
	<43BEE8F4.5080304@v.loewis.de>
	<20060107001221.GA13845@activestate.com>
	<20060111.003609.21623845.warner@lothar.com>
	<43C56563.9060107@v.loewis.de>
	<20060111204533.GB30010@activestate.com>
Message-ID: <43C57E50.2090700@v.loewis.de>

Trent Mick wrote:
> Specifically are you concerned about the readability of the waterfall
> page or other things (maintainability or something)? If the former,
> perhaps we could get some mileage out of the query args that Brian
> mentioned:

Indeed - that would be a solution. However, I wonder how many people
would look at the (say) nightly builds.

Anyway, I'll try that.

Regards,
Martin

From martin at v.loewis.de  Wed Jan 11 22:59:16 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Jan 2006 22:59:16 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <1f7befae0601101349l57e3ae35o65913fecc7112710@mail.gmail.com>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>	<43BD8464.4060801@v.loewis.de>
	<43C37C02.8030005@egenix.com>	<43C418FB.7010007@v.loewis.de>
	<43C42524.7000802@egenix.com>	<43C428A2.9000703@v.loewis.de>
	<1f7befae0601101349l57e3ae35o65913fecc7112710@mail.gmail.com>
Message-ID: <43C57FB4.4050900@v.loewis.de>

Tim Peters wrote:
> This reminded me that I still owe you a reply about s# and t# format
> codes.  It occurred to me that I've never used them, and probably
> never will, so I really don't care how they work:  I'm only really
> worried about widespread ugliness, meaning wide enough that it touches
> me ;-)

Ok, given that Armin also "gave in", I think I keep that at a macro.

Regards,
Martin

From martin at v.loewis.de  Wed Jan 11 23:09:04 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 11 Jan 2006 23:09:04 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
Message-ID: <43C58200.20803@v.loewis.de>

Guido van Rossum wrote:
> On the other hand it breaks one of the most fundamental Python
> guidelines: if you get a core dump (segfault etc.) it's a bug in
> Python or in a 3rd party extension, not in *your* Python code. An
> exception would have to be made for any code that uses ctypes, as it
> is usually trivial to cause core dumps with ctypes (I'd venture it's
> hard to avoid them ;-).
> 
> I don't expect this to count against including ctypes; but I do want
> it to be dealt with somehow!

I think the strongest point *for* ctypes is the inclusion of dl
in the source distribution, which has the very same flaws as
ctypes in terms of crashability.

So as for dealing with it "somehow": I would make ctypes a dynamically
loaded module (ctypes.pyd), so administrators could remove it, and
I could also make it a separate option in the Windows installer,
so administrators could reject to install it.

Regards,
Martin

From warner at lothar.com  Wed Jan 11 23:10:18 2006
From: warner at lothar.com (Brian Warner)
Date: Wed, 11 Jan 2006 14:10:18 -0800 (PST)
Subject: [Python-Dev] Buildbot: doing occasional full builds
In-Reply-To: <43C57DB8.7060702@v.loewis.de>
References: <20060111204533.GB30010@activestate.com>
	<17349.28766.364593.662110@montanaro.dyndns.org>
	<43C57DB8.7060702@v.loewis.de>
Message-ID: <20060111.141018.74733221.warner@lothar.com>

> nice-ing the slave process at startup would be the way to do it, right?

Yup. We run the twisted buildslaves under 'nice', and it works pretty well.
It also reveals problems in tests that use absolute timeouts which fail when
the test runs slower than the author thought it was supposed to.

cheers,
 -Brian

From trentm at ActiveState.com  Wed Jan 11 23:30:39 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 11 Jan 2006 14:30:39 -0800
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <dq3u93$s4g$1@sea.gmane.org>
References: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>
	<dq3u93$s4g$1@sea.gmane.org>
Message-ID: <20060111223039.GC6642@activestate.com>

> > Does that make sense?  We would just need /f's script in SVN.
> 
> in python/Tools/something or sandbox/something ?

python/Doc/tools/something?

Trent

-- 
Trent Mick
trentm at activestate.com

From tdelaney at avaya.com  Thu Jan 12 00:08:25 2006
From: tdelaney at avaya.com (Delaney, Timothy (Tim))
Date: Thu, 12 Jan 2006 10:08:25 +1100
Subject: [Python-Dev] Include ctypes into core Python?
Message-ID: <2773CAC687FD5F4689F526998C7E4E5F4DB8B3@au3010avexu1.global.avaya.com>

"Martin v. L?wis" wrote:

> So as for dealing with it "somehow": I would make ctypes a dynamically
> loaded module (ctypes.pyd), so administrators could remove it, and
> I could also make it a separate option in the Windows installer,
> so administrators could reject to install it.

I like this solution. Of course, Thomas (author of both py2exe and ctypes) may like the ability to have ctypes built into python.dll ...

Tim Delaney

From facundobatista at gmail.com  Thu Jan 12 00:14:04 2006
From: facundobatista at gmail.com (Facundo Batista)
Date: Wed, 11 Jan 2006 20:14:04 -0300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
	<089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz>
Message-ID: <e04bdf310601111514n20e5c63aj@mail.gmail.com>

2006/1/11, Tony Meyer <t-meyer at ihug.co.nz>:

> Remember that there has been a lot of discussion about how
> ConfigParser should work in the past; for example (ignoring c.l.p):
>
> <http://mail.python.org/pipermail/python-dev/2004-October/049454.html>
> <http://mail.python.org/pipermail/python-dev/2004-October/049527.html>
> <http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1483518>

My god. Read it all (which took me near a thermo flask (or whatever
you call to an insulated bottle to keep liquids cold or hot) of
"mate"!).

I see two paths here:

- Good one
- Bad one

No, sorry, seriously:

- Rewrite ConfigParser entirely.
- Apply my patch.

The first option is the better, though it's the one which will take
more work. Someone needs to make a PEP. Michael Chermside, in the
first mail of the first link that you sent, was working in a solution,
but don't know what is done. In this case, both patchs I talked before
should be rejected.

The second one is the quickest.

Both paths solves my problem. I think we should go with the second
option in the short term, and with the first one in the long term.

What do you think?

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From crutcher at gmail.com  Thu Jan 12 01:03:11 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Wed, 11 Jan 2006 16:03:11 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics when
	used as global dictionaries
In-Reply-To: <d49fe110601101625x6df3bbffmc7d7d1fd307189b2@mail.gmail.com>
References: <d49fe110601101547m1dce16c5n63233e7752516f5d@mail.gmail.com>
	<20060111000844.GA11919@panix.com>
	<d49fe110601101625x6df3bbffmc7d7d1fd307189b2@mail.gmail.com>
Message-ID: <d49fe110601111603q336dc718v9381024c7a8a2237@mail.gmail.com>

Is there any objection to this patch? Any support?

On 1/10/06, Crutcher Dunnavant <crutcher at gmail.com> wrote:
> 1402289
>
> On 1/10/06, Aahz <aahz at pythoncraft.com> wrote:
> > On Tue, Jan 10, 2006, Crutcher Dunnavant wrote:
> > >
> > > There is an inconsistancy in the way that dictionary subclasses behave
> > > when they are used as as namespaces in execs.
> > >
> > > Basically, while python 2.4 permits the usage of dictionary subclasses
> > > for local environments, it still bypasses the subclass functions and
> > > uses the C API for global environments. The attached patch (and
> > > unittest!) addresses this issue.
> >
> > Please submit the patch to SourceForge and report back with the SF ID.
> > --
> > Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/
> >
> > "19. A language that doesn't affect the way you think about programming,
> > is not worth knowing."  --Alan Perlis
> >
>
>
> --
> Crutcher Dunnavant <crutcher at gmail.com>
> monket.samedi-studios.com
>


--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com

From warner at lothar.com  Thu Jan 12 01:04:27 2006
From: warner at lothar.com (Brian Warner)
Date: Wed, 11 Jan 2006 16:04:27 -0800 (PST)
Subject: [Python-Dev] building a module catalogue with buildbot
Message-ID: <20060111.160427.07674399.warner@lothar.com>

> PS If there was a method on ShellCommand, adding a step would be simpler,
> e.g.,
>
>   def addStep(self, cls, **kwds):
>     self.steps.append((cls, kwds))

Ooh! I like that.

> then we could do: f.addStep(Catalog, command=Catalog.command)

It would be even simpler: f.addStep(Catalog). The command= argument is
provided by the class-level attributes of your Catalog class (it can be set
by either an attribute or by a constructor argument, but if it's always going
to be the same thing then the attribute is easier).

If you make a patch for this, I'll commit it. If not, I'll write one before I
get the next release out.

thanks,
 -Brian

From goodger at python.org  Thu Jan 12 04:51:19 2006
From: goodger at python.org (David Goodger)
Date: Wed, 11 Jan 2006 22:51:19 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <20060112033316.E24C31E4002@bag.python.org>
References: <20060112033316.E24C31E4002@bag.python.org>
Message-ID: <43C5D237.3040006@python.org>

> Author: david.goodger
> Date: Thu Jan 12 04:33:16 2006
> New Revision: 42015
> 
> Modified:
>    peps/trunk/   (props changed)
> Log:
> add external link to Docutils public repo -- always up-to-date

I just deleted the static copy of the "docutils" directory from the "peps"
repository, and added in an external link (svn:externals 'docutils
svn://svn.berlios.de/docutils/trunk/docutils/docutils').  This way, the external
code should always be up-to-date.  You may need to manually delete your
peps/trunk/docutils directory to get this to work though -- SVN leaves
subdirectories behind which hinder the externals update.

Please let me know if this causes any problems.  Thanks.

-- 
David Goodger <http://python.net/~goodger>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060111/43d05d63/attachment.pgp 

From exarkun at divmod.com  Thu Jan 12 06:35:11 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Thu, 12 Jan 2006 00:35:11 -0500
Subject: [Python-Dev] [Buildbot-devel] Re:  buildbot
In-Reply-To: <EA8E416C-607E-4D96-8789-3C323C4660DD@mac.com>
Message-ID: <20060112053511.26200.797800731.divmod.quotient.127@ohm>

On Tue, 10 Jan 2006 09:18:59 -0800, Stephen Davis <stephendavis at mac.com> wrote:
>> The reason I want static pages is for security concerns. It is not
>> easy whether buildbot can be trusted to have no security flaws,
>> which might allow people to start new processes on the master,
>> or (perhaps worse) on any of the slaves.
>
>I have security concerns as well, but not in buildbot itself.  My
>project is restricted even withinz the company I work for so I need
>the buildbot web server to only be available to certain people.
>HTTPS access would be nice too.  TwistedWeb doesn't seem to have
>support for either HTTPS or authentication so I've been forced to
>"hide" it by putting it on a non-standard port.  Very weak.

Inexcusably weak.  Fortunately not exactly accurate, though: Twisted's web server supports both HTTPS and authentication.  I won't bore the list with the details, but you can read more here: <http://twistedmatrix.com/documents/current/api/twisted.internet.interfaces.IReactorSSL.html> and <http://twistedmatrix.com/projects/web/documentation/examples/simpleguardex2.py>

Jean-Paul

From martin at v.loewis.de  Thu Jan 12 07:19:08 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Jan 2006 07:19:08 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <dq1gon$c9p$1@sea.gmane.org>
References: <dq1gon$c9p$1@sea.gmane.org>
Message-ID: <43C5F4DC.4010604@v.loewis.de>

Fredrik Lundh wrote:
> My initial thought was that we could ask alpha testers to run this script on
> their alpha builds, and report back, but it just struck me that the "buildbot"
> already builds stuff on a couple of interesting platforms.
> 
> Can buildbot deal with custom test/validation scripts, and collect the output
> somewhere ?

Collecting the output is the real challenge. Basically, you are
restricted to putting stdout of a process into a file. So a result
that is a set of linked HTML files is not supported.

Regards,
Martin

From andrew-pythondev at puzzling.org  Thu Jan 12 07:42:20 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu, 12 Jan 2006 17:42:20 +1100
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <43C5F4DC.4010604@v.loewis.de>
References: <dq1gon$c9p$1@sea.gmane.org> <43C5F4DC.4010604@v.loewis.de>
Message-ID: <20060112064220.GE4555@home.puzzling.org>

On Thu, Jan 12, 2006 at 07:19:08AM +0100, "Martin v. L?wis" wrote:
> Fredrik Lundh wrote:
> > My initial thought was that we could ask alpha testers to run this script on
> > their alpha builds, and report back, but it just struck me that the "buildbot"
> > already builds stuff on a couple of interesting platforms.
> > 
> > Can buildbot deal with custom test/validation scripts, and collect the output
> > somewhere ?
> 
> Collecting the output is the real challenge. Basically, you are
> restricted to putting stdout of a process into a file. So a result
> that is a set of linked HTML files is not supported.

A limited solution is just to make the script put the files where they will be
published by something other than buildbot.  e.g. Twisted's docs are built from
SVN by one of our buildslaves, and placed in a directory published at
http://twistedmatrix.com/users/warner/doc-latest/

-Andrew.


From martin at v.loewis.de  Thu Jan 12 07:58:00 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Jan 2006 07:58:00 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <20060112064220.GE4555@home.puzzling.org>
References: <dq1gon$c9p$1@sea.gmane.org> <43C5F4DC.4010604@v.loewis.de>
	<20060112064220.GE4555@home.puzzling.org>
Message-ID: <43C5FDF8.9010002@v.loewis.de>

Andrew Bennetts wrote:
> A limited solution is just to make the script put the files where they will be
> published by something other than buildbot.  e.g. Twisted's docs are built from
> SVN by one of our buildslaves, and placed in a directory published at
> http://twistedmatrix.com/users/warner/doc-latest/

Right. Whether this could work for us depends on how the files could be
transferred. I would not want to see rsync, ftp, scp or some such to
put files on python.org, but I could imagine getting them "through"
the twisted connection, if that were possible somehow (i.e. with the
master itself putting the files into the places where I want them).

Regards,
Martin

From raymond.hettinger at verizon.net  Thu Jan 12 10:01:37 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 12 Jan 2006 04:01:37 -0500
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused
 as global dictionaries
In-Reply-To: <d49fe110601111603q336dc718v9381024c7a8a2237@mail.gmail.com>
Message-ID: <001201c61756$cc2d8780$5916c797@oemcomputer>

> Is there any objection to this patch? Any support?

It is assigned to me.  When I have time, will go through it in detail
(the given use cases, more detailed timings, and a close reading of the
code).

If accepted, it will be for Py2.5, as it is a new feature.  There is
nothing broken about the existing eval() version, it just doesn't apply
as broadly as you would have liked.



Raymond


From mal at egenix.com  Thu Jan 12 11:50:00 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jan 2006 11:50:00 +0100
Subject: [Python-Dev] Py_ssize_t output parameters (Was:
	[Python-checkins] r41971 ...)
In-Reply-To: <43C425DE.2050001@v.loewis.de>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>
	<43C2DD44.2040908@v.loewis.de>	<43C37C92.3060302@egenix.com>
	<43C4083F.3070001@v.loewis.de>	<43C421A3.2030806@egenix.com>
	<43C425DE.2050001@v.loewis.de>
Message-ID: <43C63458.10804@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>> ... and then the type change of that variable propagates
>> throughout the extension.
> 
> That depends on the usage of the code. If the variable
> is passed by value, no further changes are necessary.
> If a pointer to the variable is passed, you could replace
> it with
> 
>   Py_ssize_t x64; int x;
>   foo(&x64);
>   x = x64;
> 
> Then use x as you did with the original code.

What if x64 has a 64-bit value ? How do you catch
and process the truncation error ?

To do this down-casting correctly, you'd need to write
code that does bounds checks and integrate into the
functions error handling.

>> You basically end up having to convert the whole extension
>> to Py_ssize_t.
> 
> That is not necessary. Can you give in example of a module
> where you think it is necessary?

If you want to port the extension to Py_ssize_t, this
is essentially the case.

You might want to take the unicodeobject.c file as
example.

>> Don't get me wrong: I don't mind doing this for the eGenix
>> extensions, but I'm worried about the gazillion other
>> useful extensions out there which probably won't get
>> upgraded in time to be used with Python 2.5.
> 
> I agree that it is not acceptable to require immediate
> whole-sale updates of every modules. However, I do
> believe that the number of modules that need any change
> at all is small, and that those modules can be modified
> with minimal effort to "get them working again,
> backwards-compatible" (i.e. with the only exception that
> they would fail if indices run above 2**31).
> 
>> I think all it takes is a set of new APIs for functions
>> that use Py_ssize_t as output parameter and which are
>> mapped to the regular API names if and only if the
>> extension #defines PY_SSIZE_T_CLEAN (or some other
>> capability flag).
> 
> That is not enough. You also need to deal with the
> function pointers that change.

We could use the type flags for these. much like we
do for the new style numbers.

> Also, others have rejected/expressed dislike of the
> PY_SIZE_T_CLEAN macro already, so they would probably
> dislike further hackishness in that direction.

That's easy to say... by not providing an easy way
to upgrade extensions, you basically just move the
requirement for hacks into the extensions.

I wouldn't consider that a good solution.

If you don't like the macro approach, why not simply
leave the two separate sets of APIs visible. Old
extensions would then use and link against the existing
APIs. All Py_ssize_t aware and compatible extensions
would use the new APIs instead. The old-style APIs
should then take care of the proper down-casting and
error reporting.

In Py3K we could then remove the old-style ones
and rename the new ones to the old names. Porting
an already Py_ssize_t compatible extension to the
renamed new style APIs would then become a simple
task of search&replace.

> Anyway, I have installed the PEP onsite, and
> added an "Open Issues" section, recording your
> comments.

Thanks.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Jan 12 12:12:09 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jan 2006 12:12:09 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C428A2.9000703@v.loewis.de>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>	<43BD8464.4060801@v.loewis.de>	<43C37C02.8030005@egenix.com>
	<43C418FB.7010007@v.loewis.de>	<43C42524.7000802@egenix.com>
	<43C428A2.9000703@v.loewis.de>
Message-ID: <43C63989.6070500@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>> If it were this easy, I wouldn't have objections. But it's
>> not.
> 
> It is so easy. Can you should me an example where it isn't?
> 
>> The variables you use with these APIs tend to propagate
>> through the extension, you use them in other calls,
>> make assignments, etc.
> 
> They only propage if you make them. You don't have to,
> if you don't want to.
> 
>> If you implement extension types, you end up having to
>> convert all the length related struct variables to
>> Py_ssize_t.
> 
> Only if you want to. If not, the module will work
> (nearly) unmodified. Of course, it will be limited
> to 32-bit indices.

See my other reply on this topic.

>> All this is fine, but it's also a lot of work which
>> can be made easier. Recompiling an extension is well
>> within range of many Python users, manually checking,
>> fixing and porting it to the new API is certainly not.
> 
> Sure. However, most users will compile it on 32-bit
> systems. If they find they cannot get it to work on
> a 64-bit system, they should ask the author for help,
> or just use it in 32-bit mode (as 64-bit mode won't
> gain them anything, anyway).

I wonder how you are going to import a 32-bit
extension into a 64-bit binary of Python.
It simply doesn't work.

>> The set of functions that will require Py_ssize_t
>> is getting larger in your branch which is why I started
>> this discussion.
> 
> How so? I did not add a single function that has
> int* output values, AFAICT.

No, but there are quite a few APIs with Py_ssize_t*
output values.

> I am talking about the entirety of these functions,
> and claim that they are rarely used (including the
> Unicode and buffer APIs).

I wouldn't say that PyString_AsStringAndSize() is rarely
used and neither is PyArg_ParseTuple().

I agree that other APIs are certainly more domain
specific and can be dealt with in the extension, but
those two APIs need special care and so do the type
slot functions.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From mal at egenix.com  Thu Jan 12 12:27:46 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jan 2006 12:27:46 +0100
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <43C5D237.3040006@python.org>
References: <20060112033316.E24C31E4002@bag.python.org>
	<43C5D237.3040006@python.org>
Message-ID: <43C63D32.9070206@egenix.com>

David Goodger wrote:
>> Author: david.goodger
>> Date: Thu Jan 12 04:33:16 2006
>> New Revision: 42015
>>
>> Modified:
>>    peps/trunk/   (props changed)
>> Log:
>> add external link to Docutils public repo -- always up-to-date
> 
> I just deleted the static copy of the "docutils" directory from the "peps"
> repository, and added in an external link (svn:externals 'docutils
> svn://svn.berlios.de/docutils/trunk/docutils/docutils').  This way, the external
> code should always be up-to-date.  You may need to manually delete your
> peps/trunk/docutils directory to get this to work though -- SVN leaves
> subdirectories behind which hinder the externals update.
> 
> Please let me know if this causes any problems.  Thanks.

Question: why do we need docutils in the peps/trunk/ directory
in the first place ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From goodger at python.org  Thu Jan 12 14:31:25 2006
From: goodger at python.org (David Goodger)
Date: Thu, 12 Jan 2006 08:31:25 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <43C63D32.9070206@egenix.com>
References: <20060112033316.E24C31E4002@bag.python.org>
	<43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com>
Message-ID: <43C65A2D.2050807@python.org>

[M.-A. Lemburg]
> Question: why do we need docutils in the peps/trunk/ directory
> in the first place ?

It's a convenience, so that a separate Docutils download & install
(& maintain) isn't necessary for those who process reST-format PEPs.

-- 
David Goodger <http://python.net/~goodger>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060112/1d6be3a8/attachment.pgp 

From mal at egenix.com  Thu Jan 12 16:37:49 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jan 2006 16:37:49 +0100
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <43C65A2D.2050807@python.org>
References: <20060112033316.E24C31E4002@bag.python.org>	<43C5D237.3040006@python.org>
	<43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org>
Message-ID: <43C677CD.3030709@egenix.com>

David Goodger wrote:
> [M.-A. Lemburg]
>> Question: why do we need docutils in the peps/trunk/ directory
>> in the first place ?
> 
> It's a convenience, so that a separate Docutils download & install
> (& maintain) isn't necessary for those who process reST-format PEPs.

Hmm, shouldn't these things be tracked under external/ ?!

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From goodger at python.org  Thu Jan 12 16:56:18 2006
From: goodger at python.org (David Goodger)
Date: Thu, 12 Jan 2006 10:56:18 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <43C677CD.3030709@egenix.com>
References: <20060112033316.E24C31E4002@bag.python.org>
	<43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com>
	<43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com>
Message-ID: <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>

On 1/12/06, M.-A. Lemburg <mal at egenix.com> wrote:
> Hmm, shouldn't these things be tracked under external/ ?!

What do you mean exactly? A new "external" directory?

SVN provides a built-in mechanism for tracking external
repositories, via the "svn:externals" property, and that's
what I used.

--
David Goodger <http://python.net/~goodger>

From mal at egenix.com  Thu Jan 12 17:15:26 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jan 2006 17:15:26 +0100
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>
References: <20060112033316.E24C31E4002@bag.python.org>	
	<43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com>	
	<43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com>
	<4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>
Message-ID: <43C6809E.306@egenix.com>

David Goodger wrote:
> On 1/12/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> Hmm, shouldn't these things be tracked under external/ ?!
> 
> What do you mean exactly? A new "external" directory?

Yes.

> SVN provides a built-in mechanism for tracking external
> repositories, via the "svn:externals" property, and that's
> what I used.

I know, but I wouldn't expect SVN to query other servers
than svn.python.org inside the standard repository
directories.

AFAIK, this is a first in the Python repository. Not
sure if it's such a good idea. Branching and tagging
doesn't work with external resources in Subversion,
so things can become inconsistent.

Also, what if you break the code in the berlios repo
or the server is not reachable ? A release copy in the
external/ dir would solve all these issues.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From fredrik at pythonware.com  Thu Jan 12 17:53:13 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 12 Jan 2006 17:53:13 +0100
Subject: [Python-Dev] r42015 - peps/trunk
References: <20060112033316.E24C31E4002@bag.python.org>	<43C5D237.3040006@python.org>
	<43C63D32.9070206@egenix.com>	<43C65A2D.2050807@python.org>
	<43C677CD.3030709@egenix.com><4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>
	<43C6809E.306@egenix.com>
Message-ID: <dq61hr$dgh$1@sea.gmane.org>

M.-A. Lemburg wrote:

> AFAIK, this is a first in the Python repository. Not
> sure if it's such a good idea. Branching and tagging
> doesn't work with external resources in Subversion,
> so things can become inconsistent.
>
> Also, what if you break the code in the berlios repo
> or the server is not reachable ? A release copy in the
> external/ dir would solve all these issues.

using svn:external is usually a lousy idea.  using it in a public repository
is a really, really, really lousy idea.

</F>




From theller at python.net  Thu Jan 12 17:55:34 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 12 Jan 2006 17:55:34 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <2773CAC687FD5F4689F526998C7E4E5F4DB8B3@au3010avexu1.global.avaya.com>
Message-ID: <lkxle67t.fsf@python.net>

"Delaney, Timothy (Tim)" <tdelaney at avaya.com> writes:

> "Martin v. L?wis" wrote:
>
>> So as for dealing with it "somehow": I would make ctypes a dynamically
>> loaded module (ctypes.pyd), so administrators could remove it, and
>> I could also make it a separate option in the Windows installer,
>> so administrators could reject to install it.
>
> I like this solution. Of course, Thomas (author of both py2exe and
> ctypes) may like the ability to have ctypes built into python.dll ...

It is getting offtopic, but I don't care too much about that.  I
requested that zlib be changed to a builtin module too allow easier
bootstrapping of py2exe'd apps which have a compressed library archive.

The nearly only reason for me to implement the single-file option for
py2exe was that the implementation simulates a static linked Python-dll,
which allows for several totally-isolated interpreters in the same
process.

Thomas


From goodger at python.org  Thu Jan 12 18:04:51 2006
From: goodger at python.org (David Goodger)
Date: Thu, 12 Jan 2006 12:04:51 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <43C6809E.306@egenix.com>
References: <20060112033316.E24C31E4002@bag.python.org>
	<43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com>
	<43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com>
	<4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>
	<43C6809E.306@egenix.com>
Message-ID: <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com>

On 1/12/06, M.-A. Lemburg <mal at egenix.com> wrote:
> I know, but I wouldn't expect SVN to query other servers
> than svn.python.org inside the standard repository
> directories.
>
> AFAIK, this is a first in the Python repository.

True, and worth discussing.  Luckily the PEPs repository is a
low-traffic, non-core area, so it's not urgent.

> Not sure if it's such a good idea.

>From my point of view, it's a time-saver.  No more manual updates!
They were a pain, and rarely got done.

> Branching and tagging
> doesn't work with external resources in Subversion,
> so things can become inconsistent.

How so?  The "svn:externals" property is treated the same as any
other, and is copied along with everything else by "svn copy".

> Also, what if you break the code in the berlios repo
> or the server is not reachable ?

If the code in the repo ever breaks, it will be fixed within minutes.
The server being unreachable is only a problem for initial checkouts;
updates will just keep the code that was already there.  In my
experience, the berlios.de SVN server has rarely been unreachable.  If
there's a problem, we can always back out the svn:externals property
and install the package.

That having been said, I do see the value of installing a packaged
release.  We just released Docutils 0.4 a few days ago; I could
install that statically.  An alternative is to use svn:externals to
link to a specific revision (via the -r option); I could link to the
0.4 release revision.  This would solve the repo-breakage issue, but
not the server-unreachability issue.

I don't think these issues are major, but I wouldn't mind terribly if
we decide to go with a static install.

> A release copy in the
> external/ dir would solve all these issues.

That's basically what we had before (except no explicity "external"
directory), but it was always out of date.  The "docutils" package was
directly in the trunk, for ease of import by the pep2html.py front end
(easy to work around, but why bother?).

Another minor nit with the old way: updates "polluted" the
python-checkins list.

--
David Goodger <http://python.net/~goodger>

From theller at python.net  Thu Jan 12 17:58:15 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 12 Jan 2006 17:58:15 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de>
Message-ID: <fynte63c.fsf@python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Guido van Rossum wrote:
>> On the other hand it breaks one of the most fundamental Python
>> guidelines: if you get a core dump (segfault etc.) it's a bug in
>> Python or in a 3rd party extension, not in *your* Python code. An
>> exception would have to be made for any code that uses ctypes, as it
>> is usually trivial to cause core dumps with ctypes (I'd venture it's
>> hard to avoid them ;-).
>> 
>> I don't expect this to count against including ctypes; but I do want
>> it to be dealt with somehow!
>
> I think the strongest point *for* ctypes is the inclusion of dl
> in the source distribution, which has the very same flaws as
> ctypes in terms of crashability.
>
> So as for dealing with it "somehow": I would make ctypes a dynamically
> loaded module (ctypes.pyd), so administrators could remove it, and
> I could also make it a separate option in the Windows installer,
> so administrators could reject to install it.

Sounds good, although it should be noted that ctypes is a package now,
with a ctypes.wrap subpackage (contains the code generator), a
ctypes.macholib subpackage (contains Bob Ippolitos macholib for OS X),
and ctypes.test subpackage whcih contains several test modules.  Plus
the _ctypes.(dll|so) extension module.

Thomas


From fredrik at pythonware.com  Thu Jan 12 18:08:57 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 12 Jan 2006 18:08:57 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
References: <dq1gon$c9p$1@sea.gmane.org> <43C5F4DC.4010604@v.loewis.de>
Message-ID: <dq62fb$hem$1@sea.gmane.org>

Martin v. Löwis wrote:

> > My initial thought was that we could ask alpha testers to run this script on
> > their alpha builds, and report back, but it just struck me that the "buildbot"
> > already builds stuff on a couple of interesting platforms.
> >
> > Can buildbot deal with custom test/validation scripts, and collect the output
> > somewhere ?
>
> Collecting the output is the real challenge. Basically, you are
> restricted to putting stdout of a process into a file. So a result
> that is a set of linked HTML files is not supported.

The listmodule utility only outputs a plain list of available modules.  Just
being to able to scrape the lists off http://www.python.org/dev/buildbot/
would be good enough.

</F>




From skink at evhr.net  Thu Jan 12 18:29:16 2006
From: skink at evhr.net (Fabien Schwob)
Date: Thu, 12 Jan 2006 18:29:16 +0100
Subject: [Python-Dev] PEP and stdlib
Message-ID: <43C691EC.3000503@evhr.net>

Hello,

I've often read new PEP that are published, and they are often about new 
additions to the core language. Why not using them with the standard 
library.

Guido often say that he don't want to include new module that aren't 
widely used in the community. It's a good thing, but it lead to the 
creation of a lot of API incompatible modules. Why not using PEP in 
order to create standard API like the Java world do with JSRs (Java 
Specification Requests) ?

What do you think about that ?

-- 
Fabien


From mal at egenix.com  Thu Jan 12 20:34:54 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 12 Jan 2006 20:34:54 +0100
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com>
References: <20060112033316.E24C31E4002@bag.python.org>	<43C5D237.3040006@python.org>
	<43C63D32.9070206@egenix.com>	<43C65A2D.2050807@python.org>
	<43C677CD.3030709@egenix.com>	<4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>	<43C6809E.306@egenix.com>
	<4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com>
Message-ID: <43C6AF5E.7020803@egenix.com>

David Goodger wrote:
> On 1/12/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> I know, but I wouldn't expect SVN to query other servers
>> than svn.python.org inside the standard repository
>> directories.
>>
>> AFAIK, this is a first in the Python repository.
> 
> True, and worth discussing.  Luckily the PEPs repository is a
> low-traffic, non-core area, so it's not urgent.

True.

>> Not sure if it's such a good idea.
> 
>>From my point of view, it's a time-saver.  No more manual updates!
> They were a pain, and rarely got done.
> 
>> Branching and tagging
>> doesn't work with external resources in Subversion,
>> so things can become inconsistent.
> 
> How so?  The "svn:externals" property is treated the same as any
> other, and is copied along with everything else by "svn copy".

Right, but Subversion doesn't store the revision of the
external resource at the time you made the copy, so
when you checkout the branch or tag, you'll still get
the most recent version of the external resource
which can break things, e.g. say you upgrade docutils
to Python 2.5, then a checkout of a tag built at
a time when Python 2.3 was current would probably
no longer work (with Python 2.3).

At least that's how things were documented the last
time I had a look at svn:externals - could be that they
modified the behavior to make it a little more clever
since.

>> Also, what if you break the code in the berlios repo
>> or the server is not reachable ?
> 
> If the code in the repo ever breaks, it will be fixed within minutes.
> The server being unreachable is only a problem for initial checkouts;
> updates will just keep the code that was already there.  In my
> experience, the berlios.de SVN server has rarely been unreachable.  If
> there's a problem, we can always back out the svn:externals property
> and install the package.
> 
> That having been said, I do see the value of installing a packaged
> release.  We just released Docutils 0.4 a few days ago; I could
> install that statically.  An alternative is to use svn:externals to
> link to a specific revision (via the -r option); I could link to the
> 0.4 release revision.  This would solve the repo-breakage issue, but
> not the server-unreachability issue.

Interesting. Last time I looked these external revisions
were not possible (or I overread the possibility).

> I don't think these issues are major, but I wouldn't mind terribly if
> we decide to go with a static install.

There are two other nits with the external reference:

* connecting to a server that people probably don't
  recognize as being an official Python source and thus
  probably don't trust right away (though this is well
  hidden by subversion, firewall software will alarm the
  user)

* being able to download the complete repository of Python
  with all the history - since the external resource is not
  part of the history, users would have to track down
  the repository of the external resource (and this might
  not be readily available for download)

>> A release copy in the
>> external/ dir would solve all these issues.
> 
> That's basically what we had before (except no explicity "external"
> directory), but it was always out of date.  The "docutils" package was
> directly in the trunk, for ease of import by the pep2html.py front end
> (easy to work around, but why bother?).

I guess that only PSF license covered software should
go into the main directories of the repository. Everything
else should first go in externals/ and then get copied
into the main trunks. It's only one additional step,
but will help a lot in the future if we ever have to
track the copyright status of things in the main
trunks.

This is not an issue for docutils, I guess, since it's
public domain, but then again, there are lawyers who
believe that there's no such thing as public domain...

http://www.linuxjournal.com/article/6225

> Another minor nit with the old way: updates "polluted" the
> python-checkins list.

I guess if you just update to new release versions, then
this will be minimal. I, for one, don't mind at all.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 12 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From tim.peters at gmail.com  Thu Jan 12 20:41:56 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 12 Jan 2006 14:41:56 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <43C6AF5E.7020803@egenix.com>
References: <20060112033316.E24C31E4002@bag.python.org>
	<43C5D237.3040006@python.org> <43C63D32.9070206@egenix.com>
	<43C65A2D.2050807@python.org> <43C677CD.3030709@egenix.com>
	<4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>
	<43C6809E.306@egenix.com>
	<4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com>
	<43C6AF5E.7020803@egenix.com>
Message-ID: <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com>

[David Goodger]
...
> An alternative is to use svn:externals to link to a specific
> revision (via the -r option);

Yes. please.  svn:externals should always pin to a specific revision
(or at least to a "frozen" tag).  Updating to a new revision then is
just a propedit away (to change the revision number); ditto for
backing off to an older revision if the newer one has problems.

As to the rest, you're using externals in a vanilla way for their
intended purpose, and I have no problems with that.

From fredrik at pythonware.com  Thu Jan 12 21:13:05 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 12 Jan 2006 21:13:05 +0100
Subject: [Python-Dev] r42015 - peps/trunk
References: <20060112033316.E24C31E4002@bag.python.org><43C5D237.3040006@python.org>
	<43C63D32.9070206@egenix.com><43C65A2D.2050807@python.org>
	<43C677CD.3030709@egenix.com><4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com><43C6809E.306@egenix.com><4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com><43C6AF5E.7020803@egenix.com>
	<1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com>
Message-ID: <dq6d8j$vav$1@sea.gmane.org>

Tim Peters wrote:

> Yes. please.  svn:externals should always pin to a specific revision
> (or at least to a "frozen" tag).  Updating to a new revision then is
> just a propedit away (to change the revision number); ditto for
> backing off to an older revision if the newer one has problems.

wasn't the whole point for this exercise to make sure that the included
version was always up to date.  if you pin it to a given version, all we
got from this was a dependence on another SVN server.

(and a can of SVN worms, including issues with locking/cleanup, etc)

</F>




From tim.peters at gmail.com  Thu Jan 12 21:24:23 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 12 Jan 2006 15:24:23 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <dq6d8j$vav$1@sea.gmane.org>
References: <20060112033316.E24C31E4002@bag.python.org>
	<43C63D32.9070206@egenix.com> <43C65A2D.2050807@python.org>
	<43C677CD.3030709@egenix.com>
	<4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>
	<43C6809E.306@egenix.com>
	<4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com>
	<43C6AF5E.7020803@egenix.com>
	<1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com>
	<dq6d8j$vav$1@sea.gmane.org>
Message-ID: <1f7befae0601121224g1e9efc6ew33bb0791d2d9c701@mail.gmail.com>

[Tim Peters]
>> Yes. please.  svn:externals should always pin to a specific revision
>> (or at least to a "frozen" tag).  Updating to a new revision then is
>> just a propedit away (to change the revision number); ditto for
>> backing off to an older revision if the newer one has problems.

[Fredrik Lundh]
> wasn't the whole point for this exercise

Sorry, I don't know what the whole point was.

> to make sure that the included version was always up to date.  if you pin
> it to a given version, all we got from this was a dependence on another SVN
> server.

And very easy updating to a new version ("a propedit away ..."), and
transparency about exactly what it is we're depending on ("revision
12345 of project XYZ").

> (and a can of SVN worms, including issues with locking/cleanup, etc)

I haven't experienced any of that.  I've experienced plenty of
locking/cleanup problems when a project switches _between_ making its
own copies and using svn:externals, but none so long as it sticks to
one way of doing that.  Even on Windows ;-)

From crutcher at gmail.com  Thu Jan 12 22:54:24 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Thu, 12 Jan 2006 13:54:24 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused
	as global dictionaries
In-Reply-To: <001201c61756$cc2d8780$5916c797@oemcomputer>
References: <d49fe110601111603q336dc718v9381024c7a8a2237@mail.gmail.com>
	<001201c61756$cc2d8780$5916c797@oemcomputer>
Message-ID: <d49fe110601121354u6628da58j10de76961514e463@mail.gmail.com>

I sorta disagree about it not being broken. Adding a feature which
works for eval but not for exec seems pretty broken. It's difficult to
reason about what will happen in the exec context, so I can't see what
fixing it would endanger; but I'd deffinately like to see it for 2.5.

I've run rough timings on the code, ('test make time'), detailed in
the patch discussion, and it seems completely lost in the noise.

On 1/12/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> > Is there any objection to this patch? Any support?
>
> It is assigned to me.  When I have time, will go through it in detail
> (the given use cases, more detailed timings, and a close reading of the
> code).
>
> If accepted, it will be for Py2.5, as it is a new feature.  There is
> nothing broken about the existing eval() version, it just doesn't apply
> as broadly as you would have liked.
>
>
>
> Raymond
>
>


--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com

From t-meyer at ihug.co.nz  Thu Jan 12 22:57:03 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Fri, 13 Jan 2006 10:57:03 +1300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <e04bdf310601111514n20e5c63aj@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
	<089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz>
	<e04bdf310601111514n20e5c63aj@mail.gmail.com>
Message-ID: <56E4F12D-9477-4F58-9D85-3D159B3F1B71@ihug.co.nz>

> I see two paths here:
>
> - Rewrite ConfigParser entirely.
> - Apply my patch.

Allowing 'surgical' editing of configuration files, as has been  
proposed many times both here and c.l.p would not require  
ConfigParser to be entirely rewritten (just more extensive  
modification of the write() method).

> Someone needs to make a PEP. Michael Chermside, in the
> first mail of the first link that you sent, was working in a solution,
> but don't know what is done.

I believe that this always trails off without anything being achieved.

IMO what would be best would be a pronouncement (or something of  
similar weight) about the direction that should be taken wrt to  
configuration file parsing support in the stdlib.

   * The status quo.
   * The status quo, but output files in a fixed order (but unrelated  
to the read order); i.e. http://python.org/sf/1399309
   * 'Surgical' editing of files; i.e. preserve comments, whitespace,  
and ordering, appending new options/sections as needed.
   * A more complex configuration system (perhaps like others that  
exist in the wild).

I'm happy to either submit or help with a patch for the third option  
(and maybe the fourth) if need be.

I haven't personally found the lack of a more complex system in the  
stdlib a problem (although I suppose that the various more complex  
systems I have worked with could have been simplified if there was  
one).  I do believe that failing to preserve comments and ordering is  
a problem.

=Tony.Meyer


From martin at v.loewis.de  Thu Jan 12 23:36:01 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Jan 2006 23:36:01 +0100
Subject: [Python-Dev] Py_ssize_t output parameters (Was:
	[Python-checkins] r41971 ...)
In-Reply-To: <43C63458.10804@egenix.com>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>
	<43C2DD44.2040908@v.loewis.de>	<43C37C92.3060302@egenix.com>
	<43C4083F.3070001@v.loewis.de>	<43C421A3.2030806@egenix.com>
	<43C425DE.2050001@v.loewis.de> <43C63458.10804@egenix.com>
Message-ID: <43C6D9D1.7040101@v.loewis.de>

M.-A. Lemburg wrote:
> What if x64 has a 64-bit value ? How do you catch
> and process the truncation error ?

We were *both* discussing a scenario where no sizes
exceed 2**31, right?

Under such a scenario, this just won't happen.

OTOH, if you were discussing a scenario where sizes
might exceed 2**31, then I don't see why you are worried
about Py_ssize_t* parameters alone: Even

  PyString_Size()

might (of course!) return a value > 2**31 - so it
is not just output parameters, but also return values.

For more information, please read "Conversion guidelines"
in

http://www.python.org/peps/pep-0353.html

>>That is not necessary. Can you give in example of a module
>>where you think it is necessary?
> 
> 
> If you want to port the extension to Py_ssize_t, this
> is essentially the case.
> 
> You might want to take the unicodeobject.c file as
> example.

unicodeobject.c is not an extension. We were discussing
existing extension modules.

> We could use the type flags for these. much like we
> do for the new style numbers.

Would you like to write a specification for that?

> If you don't like the macro approach, why not simply
> leave the two separate sets of APIs visible.

To simplify the porting.

> All Py_ssize_t aware and compatible extensions
> would use the new APIs instead. The old-style APIs
> should then take care of the proper down-casting and
> error reporting.

That is not possible. Some API does not support
error reporting (like PyString_Size()). So callers
don't expect it to fail.

Regards,
Martin

From brett at python.org  Thu Jan 12 23:36:51 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Jan 2006 14:36:51 -0800
Subject: [Python-Dev] Ph.D. dissertation ideas?
Message-ID: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>

It is time once again in my educational career to come to python-dev
for help for major project ideas.  This time, though, it is for my
Ph.D. dissertation (and thus can have larger scope than my masters
thesis) but there are funding restrictions (and thus only certain
areas I have possible funding for).

First off, there are two areas where I have possible funding: XML
integration into programming languages and game scripting support (I
have a third, AOP for anything, but  I don't think AOP is a good match
for Python and thus not considering it for Python work).  The XML
integration I have no direct ideas since I don't use XML.  There is
the possibility of doing something like the LINQ project
(http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp)
or something, but I don't know how useful that could be.  It would
have to be just as generic, though, as the LINQ project in terms of
working with intefaces to be Pythonic so it might turn out to be more
about language support for querying data.

For the game scripting, trying out the active objects for Python is a
possibility (or something else that would help with concurrency). 
Since that could allow for agent objects to be more concurrent it can
be argued that it would help with game scripting.  I don't have any
other ideas for the area of game scripting support.

Now I am not locked down to these areas, but they are what I have
possible grant money for.  If through some miracle someone out there
wants to fund my Ph.D. and wants something else worked on for Python,
don't be shy.  =)  Unfortunately I don't think the PSF is up for
funding my Ph.D., else I would say I would work on whatever python-dev
decided they wanted to worked on that could be done as a dissertation.

Anyway, any ideas are appreciated.  There is no rush on this, either. 
Just getting this out there now while I am tentatively exploring
possible topics.

-Brett

From martin at v.loewis.de  Thu Jan 12 23:44:27 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 12 Jan 2006 23:44:27 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C63989.6070500@egenix.com>
References: <43B3ECEE.9060300@v.loewis.de>	<20051229173041.GA19575@code1.codespeak.net>	<43B50B64.20904@v.loewis.de>	<20060105194538.GB11736@code1.codespeak.net>	<43BD8464.4060801@v.loewis.de>	<43C37C02.8030005@egenix.com>
	<43C418FB.7010007@v.loewis.de>	<43C42524.7000802@egenix.com>
	<43C428A2.9000703@v.loewis.de> <43C63989.6070500@egenix.com>
Message-ID: <43C6DBCB.70305@v.loewis.de>

M.-A. Lemburg wrote:
>>Sure. However, most users will compile it on 32-bit
>>systems. If they find they cannot get it to work on
>>a 64-bit system, they should ask the author for help,
>>or just use it in 32-bit mode (as 64-bit mode won't
>>gain them anything, anyway).
> 
> 
> I wonder how you are going to import a 32-bit
> extension into a 64-bit binary of Python.
> It simply doesn't work.

No. They will need to run the 32-bit binary of Python
on the 64-bit system.

> I wouldn't say that PyString_AsStringAndSize() is rarely
> used and neither is PyArg_ParseTuple().

I can't find much usage of PyString_AsStringAndSize.
Even in the Python core, it is only called 8 times
outside string and strop.

> I agree that other APIs are certainly more domain
> specific and can be dealt with in the extension, but
> those two APIs need special care and so do the type
> slot functions.

For PyArg_ParseTuple, there is special care, because
there won't be compiler warnings otherwise.

For PyString_AsStringAndSize, the compiler will
warn about incorrect pointer types.

Regards,
Martin

From pje at telecommunity.com  Fri Jan 13 00:00:58 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu, 12 Jan 2006 18:00:58 -0500
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.co
 m>
Message-ID: <5.1.1.6.0.20060112174613.01e4ce10@mail.telecommunity.com>

At 02:36 PM 1/12/2006 -0800, Brett Cannon wrote:
>(I have a third, AOP for anything, but  I don't think AOP is a good match
>for Python and thus not considering it for Python work)

For what it's worth, I've been doing what I'll loosely refer to as 
"research" on implementing a non-traditional form of AOP in Python via the 
combination of extensible generic functions and contextual variables -- two 
concepts that already exist in Python but which can be extended far beyond 
what is in current use.

The combination can be used to implement an "explicit is better than 
implicit" form of AOP, that is usually handled by a "weaving language" in 
classic AOP implementations.  I don't know of any published AOP work that 
focuses on the use of these kinds of tools; the communities that talk about 
the respective technologies appear to be disjoint.

Anyway, extensible generic operations in Python like 'len()', 'copy()' and 
so on are fairly common, but it's rare that people define their own such 
functions, as opposed to using ones in the stdlib.  A typical workaround in 
larger frameworks is to use interfaces and adaptation, but extensible 
generic operations make it easier to separate concerns orthogonally.

So, although you might think that AOP isn't "a good match for Python", the 
truth is that there's actually a fair amount of AOP-like things being done 
with Python.  It's just that classic structuring of AOP into things like 
aspects and weavers and all that crud is overcomplication.  If you view it 
simply in terms of extensible operations, which can also make decisions 
based on context, then you can achieve all the same goals of "classic" AOP 
without using any of its concepts or terminology.

One definition of AOP claims that it allows asserting qualified statements 
over oblivious code.  Python has this basic ability natively -- 
monkeypatching suffices to meet that definition.  The interesting question 
is, how can you do it *without* monkeypatching?


From martin at v.loewis.de  Fri Jan 13 00:06:09 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Jan 2006 00:06:09 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <fynte63c.fsf@python.net>
References: <u0cbyfa1.fsf@python.net>	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>	<43C58200.20803@v.loewis.de>
	<fynte63c.fsf@python.net>
Message-ID: <43C6E0E1.5010705@v.loewis.de>

Thomas Heller wrote:
> Sounds good, although it should be noted that ctypes is a package now,
> with a ctypes.wrap subpackage (contains the code generator), a
> ctypes.macholib subpackage (contains Bob Ippolitos macholib for OS X),
> and ctypes.test subpackage whcih contains several test modules.  Plus
> the _ctypes.(dll|so) extension module.

Ok. The installer should then combine them all into a feature.

Still, the admin could disable all of this just by removing _ctypes.dll,
right?

Regards,
Martin

From martin at v.loewis.de  Fri Jan 13 00:12:31 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Jan 2006 00:12:31 +0100
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused
 as global dictionaries
In-Reply-To: <d49fe110601121354u6628da58j10de76961514e463@mail.gmail.com>
References: <d49fe110601111603q336dc718v9381024c7a8a2237@mail.gmail.com>	<001201c61756$cc2d8780$5916c797@oemcomputer>
	<d49fe110601121354u6628da58j10de76961514e463@mail.gmail.com>
Message-ID: <43C6E25F.2040606@v.loewis.de>

Crutcher Dunnavant wrote:
> I sorta disagree about it not being broken. Adding a feature which
> works for eval but not for exec seems pretty broken.

You "seem" to have a different notion of "to be broken", then.

Python is broken, if and only if
- the interpreter crashes, for any Python input
- the implementation does not do what the documentation says
  it would do
- the BDFL pronounces it is broken

In this specific change, the change did precisely what the requestor
of the feature wanted it to do (that eval could accept non-exact
dicts was a new feature back then also)

> It's difficult to
> reason about what will happen in the exec context, so I can't see what
> fixing it would endanger; but I'd deffinately like to see it for 2.5.

It would make Python code run which is currently rejected with an
exception. Therefore, it is a new feature (a behaviour change).

Applications relying on that feature would have to specify that
they require "2.4.3"; people would find that code that runs fine
in 2.4.3 fails in 2.4.2. This is not acceptable.

Regards,
Martin

From martin at v.loewis.de  Fri Jan 13 00:14:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Fri, 13 Jan 2006 00:14:24 +0100
Subject: [Python-Dev] PEP and stdlib
In-Reply-To: <43C691EC.3000503@evhr.net>
References: <43C691EC.3000503@evhr.net>
Message-ID: <43C6E2D0.9040407@v.loewis.de>

Fabien Schwob wrote:
> What do you think about that ?

See PEP 2.

Regards,
Martin

From Scott.Daniels at Acm.Org  Fri Jan 13 01:27:29 2006
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Thu, 12 Jan 2006 16:27:29 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <20060111195900.GB17269@activestate.com>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>
	<20060111195900.GB17269@activestate.com>
Message-ID: <dq6s50$mc4$1@sea.gmane.org>

OK I carried the code I offered earlier in this whole thread (tweaked in
reaction to some comments) over to comp.lang.python, gathered some
feedback, and put up a recipe on the cookbook.  After a week or so for
more comment, I'll be happy to submit a patch to include the broken_test
decorator function in unittest.

Here is where the recipe is, for those who want to comment further:
     http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288

--Scott David Daniels
Scott.Daniels at Acm.Org


From brett at python.org  Fri Jan 13 01:58:20 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Jan 2006 16:58:20 -0800
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <Pine.LNX.4.44.0601121446380.3707-100000@shasta.stanford.edu>
References: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>
	<Pine.LNX.4.44.0601121446380.3707-100000@shasta.stanford.edu>
Message-ID: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>

On 1/12/06, Dennis Allison <allison at shasta.stanford.edu> wrote:
>
> Brett,
>
> Where are you doing your Phd and who will be your likely supervisor?
> It does make a difference.
>

University of British Columbia under Eric Wohlstadter in the Software
Practices Lab.

> Your dissertation idea list seems to me to focus on implementation
> projects and not on research.  Usually a dissertation proceeds from a
> hypothesis leading to an experiment, some measurements, and conclusions.
> Many universities tacitly expect a theorem and associated proof. The goal
> of dissertation research is a completed approved dissertation, not some
> intergalactic all encompassing contribution to human knowledge.
> Dissertation research should, IMHO, focus on a small, manageable problem.
>

The topics are a little on the implementation side partially from my
personal taste in projects; I have always had a major bent towards the
practical over theoretical (reason I love Python so much  =).  Plus
the lab I am in tends towards the practical and not into the
theoretical (e.g., the number of Eclipse plug-ins being developed as
part of theses and dissertations here).

But I am open for theoretical work (as long as it does not involve
type inference).

> Frequently, universities have a large cooperative project involving many
> graduate students, each of whom research a small, well defined topic areas
> related to the larger project.   Is this the case for your dissertation?
>

Nope.  Nothing going on at the moment here in terms of a large scoped
project with multiple parts to it.

> What are your interests?  Are you interested in language structure,
> language implementation, the formal interface between languages and
> applications, or what.
>

Tough question.  Language design is the over-reaching area that I
love.  Basically how can powerful language ideas be presented in a
easy-to-use fashion.  Could even generalize even farther to just
making programming easier through the language.  And for some reason
networking protocols and concurrent programming have always caught my
eye (have not delved deep into these topics but I always catch myself
doing stuff such as reading the ICMP standard or reading all of the
various arguments for or against threading).

> Like most folks who lurk on this list, I have my list of things I'd like
> to see someone research.  Tell us a bit more and I am sure you'll get lots
> of us to share.
>

That's the hope.  =)

-Brett

From janssen at parc.com  Fri Jan 13 02:10:41 2006
From: janssen at parc.com (Bill Janssen)
Date: Thu, 12 Jan 2006 17:10:41 PST
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: Your message of "Thu, 12 Jan 2006 16:58:20 PST."
	<bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com> 
Message-ID: <06Jan12.171050pst."58633"@synergy1.parc.xerox.com>

Brett,

How about building a system that compiles a Python program (possibly
annotated) to an AJAX program?  That is, it analyzes the program to
determine what's appropriate and possible for client-side and
server-side, figures out the optimal network API (reduce latency,
reduce calls, reduce data transfer, increase responsiveness),
generates server-side Jython to run in Tomcat (or Python for
mod_python), and client-side XHTML, CSS, and Javascript to implement
the browser-based portion of the application.  Oddities of browser
implementations and Javascript VMs would be contained in the compiler
and automatically pushed into the generated code.

This would be a tremendously useful advance in building distributed
systems that work with the Web.  The current approach to AJAX is a bit
like carpentry with old hand tools.  You'd probably also push Python
to higher levels of language-ness.

> Tough question.  Language design is the over-reaching area that I
> love.  Basically how can powerful language ideas be presented in a
> easy-to-use fashion.  Could even generalize even farther to just
> making programming easier through the language.  And for some reason
> networking protocols and concurrent programming have always caught my
> eye

Bill

From brett at python.org  Fri Jan 13 04:29:33 2006
From: brett at python.org (Brett Cannon)
Date: Thu, 12 Jan 2006 19:29:33 -0800
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <-8397309676539309595@unknownmsgid>
References: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>
	<-8397309676539309595@unknownmsgid>
Message-ID: <bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>

On 1/12/06, Bill Janssen <janssen at parc.com> wrote:
> Brett,
>
> How about building a system that compiles a Python program (possibly
> annotated) to an AJAX program?  That is, it analyzes the program to
> determine what's appropriate and possible for client-side and
> server-side, figures out the optimal network API (reduce latency,
> reduce calls, reduce data transfer, increase responsiveness),
> generates server-side Jython to run in Tomcat (or Python for
> mod_python), and client-side XHTML, CSS, and Javascript to implement
> the browser-based portion of the application.  Oddities of browser
> implementations and Javascript VMs would be contained in the compiler
> and automatically pushed into the generated code.
>
> This would be a tremendously useful advance in building distributed
> systems that work with the Web.  The current approach to AJAX is a bit
> like carpentry with old hand tools.  You'd probably also push Python
> to higher levels of language-ness.
>

Hmm.  It's an idea.  I also thought of Python -> JavaScript compiler
since JavaScript is not fun and getting to test using Python might be
cool.  But not sure how useful that would be.  Plus I bet someone has
does this with Java or something.

-Brett

From steve at holdenweb.com  Fri Jan 13 04:40:34 2006
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Jan 2006 03:40:34 +0000
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
References: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>	<-8397309676539309595@unknownmsgid>
	<bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
Message-ID: <dq77ff$k8j$1@sea.gmane.org>

Brett Cannon wrote:
> On 1/12/06, Bill Janssen <janssen at parc.com> wrote:
> 
>>Brett,
>>
>>How about building a system that compiles a Python program (possibly
>>annotated) to an AJAX program?  That is, it analyzes the program to
>>determine what's appropriate and possible for client-side and
>>server-side, figures out the optimal network API (reduce latency,
>>reduce calls, reduce data transfer, increase responsiveness),
>>generates server-side Jython to run in Tomcat (or Python for
>>mod_python), and client-side XHTML, CSS, and Javascript to implement
>>the browser-based portion of the application.  Oddities of browser
>>implementations and Javascript VMs would be contained in the compiler
>>and automatically pushed into the generated code.
>>
>>This would be a tremendously useful advance in building distributed
>>systems that work with the Web.  The current approach to AJAX is a bit
>>like carpentry with old hand tools.  You'd probably also push Python
>>to higher levels of language-ness.
>>
> 
> 
> Hmm.  It's an idea.  I also thought of Python -> JavaScript compiler
> since JavaScript is not fun and getting to test using Python might be
> cool.  But not sure how useful that would be.  Plus I bet someone has
> does this with Java or something.
> 
If you can persuade your supervisor to be interested in educational 
games, a Python learning system might be an interesting project with 
lots of scope for user modelling and goal-directed behaviour.

There's been some work done on bomb-proofing the interpreter to offer a 
"try-python" type site, but nobody seems to have thought about how to 
engage a neophyte with an specific teaching strategy (possibly based on 
personality analysis) and then present problems to verify comprehension 
and offer appropriate feedback.

That would be a terrific game!

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From tony.meyer at gmail.com  Thu Jan 12 05:03:35 2006
From: tony.meyer at gmail.com (Tony Meyer)
Date: Thu, 12 Jan 2006 17:03:35 +1300
Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-01 through
	2005-12-15
Message-ID: <6c63de570601112003oa772cc3l4eb31cf3aad314f3@mail.gmail.com>

=============
Announcements
=============

-----------------------------------------------------
Reminder: plain text documentation fixes are accepted
-----------------------------------------------------

Want to help out with the Python documentation? Don't know LaTeX? No
problem! Plain text or ReST fixes are also welcome. You won't be able
to produce a diff file like with a normal patch, but comments that
explain how to fix the docs are just as good. A form like "in section
XXX right before the paragraph starting with YYY, the documentation
should say ZZZ" will make it easy for the doc maintainers to apply
your fix.

Contributing thread:

 - `c.l.p post on docs
<http://mail.python.org/pipermail/python-dev/2005-December/058479.html>`__

[SJB]

--------------------------------------------
New-style exceptions patch requesting review
--------------------------------------------

With `PEP 352`_ ready_ for Guido's pronouncement, Michael Hudson has
asked for a few more developers to review his patch_ to make all
exceptions new-style. It should be basically ready, but it would be
nice to have a few eyes for sanity checks, documentation, and
compilations on the various platforms.

.. _PEP 352: http://www.python.org/peps/pep-0352.html
.. _ready: http://www.python.org/dev/summary/2005-10-16_2005-10-31.html#required-superclass-for-exceptions
.. _patch: http://bugs.python.org/1104669

Contributing threads:

 - `New-style exceptions patch (was Re: PEP 8 updates/clarifications)
<http://mail.python.org/pipermail/python-dev/2005-December/058542.html>`__
 - `New-style exceptions patch
<http://mail.python.org/pipermail/python-dev/2005-December/058543.html>`__

[SJB]

----------------------------------------
Debugging lib available from ActiveState
----------------------------------------

Trent Mick confirmed that ActiveState do distribute a separate
distribution of debug DLLs for each Python release.  These can be
found by filling in the version number in the URL
ftp://ftp.activestate.com/ActivePython/etc/ActivePython-<version>-win32-ix86-debug.zip

Contributing thread:

 - `Plea to distribute debugging lib
<http://mail.python.org/pipermail/python-dev/2005-December/058446.html>`__

[TAM]

=========
Summaries
=========

-------------------------------
Updating the Python style guide
-------------------------------

Ian Bicking kicked off a lengthy discussion about updating various
aspects of `PEP 8`_ (the Python code style guide), which resulted in a
substantial revision of the PEP.

PEP 8 stated that if a module defines a single exception raised for
all sorts of conditions, it is generally called "error" or "Error". 
Ian noted that other than some outlying cases (e.g. os.error,
socket.error), CapWords are always used.  He also wondered if
exceptions should have names that are relatively unique, or simply
unique within their namespace.  Finally, he requested a position be
taken on acronyms (e.g. HHTPRedirect or HttpRedirect).

Barry Warsaw pointed out that since exceptions are now classes, the
rules for naming classes should really apply; his preference is
CapWordsEndingInError rather than Error or error.  He also suggested
that acronym letters should be capitalised.  There was mostly
consensus on this, although some prefer shorter exception class names.

Guido wondered whether the function/method naming convention
(lower_case, mixedCase, or CapWords) was so controversial that the PEP
should not make a recommendation other than of consistency.  In the
end, however, the status quo (lower_case except for consistency
reasons) won out.  He was definite about module, package, and class
names, however, which should be all-lowercase without underscores
(modules/packages), or CapWords (class names).  (He noted that
standard library modules such as StringIO.py and UserDict.py that
break this rule should be changed).

PEP 8 recommended appending an underscore rather than corrupted
spelling when a name conflicts with a reserved word e.g. class_ rather
than klass).  Ian suggested that a specific recommendation for the
class argument to classmethods be made; out of cls, klass and class_
he preferred cls.  He further suggested that, as self is not used
outside of the first argument of instance methods, whatever spelling
of the class argument is used should not be used elsewhere (e.g. cls
for the class argument, and class_ elsewhere).  This was generally
accepted.

A further suggestion from Barry's `Mailman style guide`_ was also
made, that from-imports should follow non-from imports, and dotted
imports should follow non-dotted imports.  Non-dotted imports should
be grouped by increasing length, while dotted imports should be
grouped roughly alphabetically.  However, this was felt to be too
prescriptive; users should follow their own aesthetic taste.

Various other small modifications and improvements were suggested and
made, such as::

  * PEP 8 stated that modules designed for use via "from M import *"
should prefix their globals, internal functions, and internal classes
with an underscore.  Ian suggested that __all__ should be recommended
over using a leading underscore, which was done.

  * PEP 8 was fairly dated in discussing inheritance and
public/non-public methods.  Ian suggested that the language should be
updated, and that property() should be discussed, and this was done.

  * PEP 8 recommended that class-based exceptions are used rather than
string-based exceptions.  Ian suggested that language be changed to
recommend more strongly against string-based exceptions, which was
done.

  * To make the PEP more concise and succinct, references to Emacs were removed.

There was also a sub-thread that considered when when to use normal
instance variables and when to use accessor methods.  Jim Fulton
suggested that attributes should be used if the accessors are trivial,
whereas Skip Montanaro preferred more frequent use of accessor
methods, to avoid incorrect usage of classes.  Phillip J. Eby's
opinion was that normal instance variables should be used, until there
is a need to do something when they change, at which point properties
should be introduced.  PEP 8 now recommends simple attribute access
over accessors.

One distinction is how people interpret "public" and "private". 
Skip's opinion is that just because an attribute doesn't start with an
underscore doesn't mean that it is implicitly declared public, and
that people should familiarise themselves with the callable methods of
an object and only use direct access if the necessary methods aren't
provided.  Jim felt that the API should be documented, and people
should use that API (he considered that prepending an underscore
documents that the attribute is internal; although this does not cover
the "subclassing API").

Ian also brought up the section of PEP 8 that discussed private
attributes and double leading underscores.  PEP 8 was unclear whether
double leading attributes should only be used to avoid name conflicts
when subclassing, or whether indicating that an attribute was meant to
be private was a valid use.  This spun off a very long discussion
about private variables.

Several people spoke up against double-underscore prefixes.  On the
other hand, Raymond Hettinger felt that the PEP should not dictate how
variables should be named, especially for a convention with a long
history and language support.  Jim Fulton went so far as to suggest
that __private be deprecated, which gained some support.  However,
Guido stated that he not only liked __private, he liked the current
implementation.  Specifically, he felt that the typically-shallow
class hierarchies found in Python reduces the likelihood of accidental
reuse of class names (he also advocated that subclasses should not
share names with parent classes), and he liked that the name-mangling
algorithm was well-defined and simple (e.g. so that when in the
debugger it is easy to manually mangle/unmangle names). 
Interestingly, this is the main reason that Jeremy Hylton dislikes
mangled names: because the debugger is unaware of the names and he
can't remember how to type them, and also because it's annoying to
have to change every instance of __var to _var if the intended usage
changes (as compared to C++ private variables, where only the
declaration of visibility must change).  He noted, though, that these
are problems that tools (debuggers, editors, IDEs) could solve.

Others felt that keeping mangling was fine, but that it should be
changed so that collisions were harder (e.g. use a GUID instead of the
class name).  However, this violates one of Guido's reasons for liking
the current implementation (although it's possible that having better
support for the feature in common tools would remove this necessity).

Tim Peters gave a nice explanation of why the name mangling is provided::

  If your utility or mixin base class `A` has a data member named `n`,
  nobody deriving from `A` dare name one of their data members `n` too,
  and it's unreasonable to expect everyone deriving from `A` to learn and
  avoid all the names `A` uses internally.  It's even more unreasonable
  for A's author to have to promise, after A's first release, never to
  change the name of, or introduce any new, attribute (A's author dare
  not, lest the new name conflict with a name someone else's subclass used).
  If A's author names the attribute `__n` instead, all those problems go
  away, provided only that some subclass doesn't also name itself `A`.

Contributing threads:

 - `PEP 8 updates/clarifications
<http://mail.python.org/pipermail/python-dev/2005-December/058532.html>`__
 - `Deprecate __ private (was Re: PEP 8 updates/clarifications)
<http://mail.python.org/pipermail/python-dev/2005-December/058555.html>`__
 - `Import order (was Re: PEP 8 updates/clarifications)
<http://mail.python.org/pipermail/python-dev/2005-December/058671.html>`__
 - `Deprecate __ private (was Re: PEP 8updates/clarifications)
<http://mail.python.org/pipermail/python-dev/2005-December/058694.html>`__
 - `PEP 8 updates/clarifications, function/method style
<http://mail.python.org/pipermail/python-dev/2005-December/058732.html>`__

 .. _PEP 8: http://www.python.org/dev/peps/pep-0008.html
 .. _Mailman style guide: http://barry.warsaw.us/software/STYLEGUIDE.txt

[TAM]

------------------------------------
ElementTree in the Python 2.5 stdlib
------------------------------------

Skip Montanaro forwarded to python-dev a bit of a comp.lang.python
thread asking why ElementTree wasn't part of the standard library. He
and others argued that it was "best of breed" with a wide user base,
and that it met the other requirements_ for stdlib inclusion like
having an active maintainer, Fredrik Lundh. After a relatively brief
discussion, Fredrik got the ball rolling, setting things up so that:

* The ``external`` directory at the top of the SVN tree contains
snapshots of his celementtree and elementtree packages
* The ``etree`` subpackage of the ``xml`` module contains the
ElementTree, cElementTree, ElementPath and ElementInclude modules

Mike Brown started a brief follow-up thread concerned that the XML-SIG
hadn't been consulted on this inclusion.  Martin v. L?wis and others
indicated that sidestepping XML-SIG had not been intensional, but also
that carrying the discussion on XML-SIG as well would not likely have
changed the outcome due to the strong community support for
ElementTree.

As a side effect of this discussion, magic in ``xml/__init__`` was
removed in favor of introducing an ``xmlcore`` package containing all
the xml modules found in the basic standard library, and an ``xml``
module which imports things from ``xmlcore`` or PyXML if it's
available.

.. _requirements:
http://mail.python.org/pipermail/python-dev/2003-April/034881.html

Contributing threads:

 - `ElementTree - Why not part of the core? (fwd)
<http://mail.python.org/pipermail/python-dev/2005-December/058512.html>`__
 - `stupid package tricks
<http://mail.python.org/pipermail/python-dev/2005-December/058622.html>`__
 - `ElementTree in stdlib
<http://mail.python.org/pipermail/python-dev/2005-December/058638.html>`__
 - `Sharing expat instances
<http://mail.python.org/pipermail/python-dev/2005-December/058692.html>`__
 - `"xml"; package in standard library
<http://mail.python.org/pipermail/python-dev/2005-December/058710.html>`__

[SJB]

--------------------
More work on the AST
--------------------

This fortnight saw implementations for the two main models proposed
for revamping the AST memory management. Martin v. L?wis created a
branch_ that converted all AST objects to PyObjects and used the
normal Python reference counting to manage them. This requires that
AST code:

* initialize all PyObjects to NULL
* Py_XDECREF() each PyObject on exit
* goto an error block for Py_DECREF()ing if there is a failure

Jeremy Hylton worked up a separate version of the AST code using an
arena API.  This requires that AST code:

* call PyArena_AddPyObject() for any allocated PyObject
* call PyArena_AddMallocPointer() for any malloc()ed memory

The arena code was merged into the trunk, though it seemed that work
on the PyObject branch would continue in order to be able to compare
the final outcomes of both strategies.

In a related issue, because the AST code is generated from
Parser/asdl_c.py, and because subversion sets the timestamp for each
file to the checkout time, trying to build the current trunk on a
machine without Python installed failed even when the AST C files had
been checked into subversion.  It looked like the best solution would
be to introduce a separate make rule for generating the AST code.

.. _branch: http://svn.python.org/projects/python/branches/ast-objects/

Contributing threads:

 - `ast-objects branch created
<http://mail.python.org/pipermail/python-dev/2005-December/058433.html>`__
 - `Memory management in the AST parser &amp; compiler
<http://mail.python.org/pipermail/python-dev/2005-December/058434.html>`__
 - `PyAST_FromNode returning PyTypeObject* ?
<http://mail.python.org/pipermail/python-dev/2005-December/058440.html>`__
 - `should I really have to install Python before I can build it ?
<http://mail.python.org/pipermail/python-dev/2005-December/058608.html>`__
 - `should I really have to install Python before Ican build it ?
<http://mail.python.org/pipermail/python-dev/2005-December/058618.html>`__
 - `should I really have to install Python before Icanbuild it ?
<http://mail.python.org/pipermail/python-dev/2005-December/058625.html>`__

[SJB]

-------------------
Python GC Refresher
-------------------

Gregoire Weber asked for a little more information on Python's garbage
collector since the links in ``Modules/gcmodule.c`` were broken.
Python's garbage collector is a combination of reference counting and
the periodic invocation of a cyclic garbage collector. Python's
reference counting means that each time an object P is referenced by
another object, P's refcount is incremented, and each time one of the
references to P is removed, P's refcount is decremented.  When P's
refcount reaches zero, the interpreter pauses to reclaim P and all
objects that were reachable only from P. In addition to this reference
counting, Python's cyclic garbage collector means that after a large
number of objects have been allocated (and not deallocated though
reference counting), the interpreter will pause to try to clear out
any unreferenced cycles.

Contributing thread:

 - `Documentation about Python's GC, python-dev list messages
referenced in Modules/gcmodule.c not reachable anymore
<http://mail.python.org/pipermail/python-dev/2005-December/058526.html>`__

[SJB]

-----------------------------------
Improving the documentation process
-----------------------------------

Skip Montanaro suggested that, in order to lower the barrier for
submitting documentation patches, it might be worth allowing anonymous
bug reports. Guido was against this, indicating that he thought the
problem was more the sourceforge sign-up hassle than the need to
provide an email address, and suggested that it might be time to
switch to roundup_. Martin v. L?wis was concerned about the transition
process - whether or not roundup could properly import the sourceforge
bug reports, and whether or not the python.org/sf/<sf bug id> redirect
would continue to work. The discussion trailed off before any final
decisions were made.

.. _roundup: http://roundup.sourceforge.net/

Contributing threads:

 - `Tracker anonymity
<http://mail.python.org/pipermail/python-dev/2005-December/058489.html>`__

[SJB]

------------------------
hasattr() and properties
------------------------

Thomas Lotze noticed that when applied to a class instance (but not an
object of that class), hasattr calls getattr and decides that the
attribute doesn't exist if the call raises any exception.  For
example::

>>> class Foo(object):
...     def get(self):
...         raise Exception
...     bar = property(get)
...
>>> hasattr(Foo, "bar")
True
>>> hasattr(Foo(), "bar")
False

He asked whether it would make more sense to only report a missing
attribute if an AttributeError is raised.  Greg Ewing agreed that this
would be an improvement, but felt that calling the property access
code as a side effect of hasattr seems like a misfeature.

Thomas also wondered whether it would make sense for properties to
look up the attribute in the same way that getattr would rather than
calling getattr.  Greg wondered if descripters need a forth slot for
hasattr customisation, removing the need to rely on catching
exceptions, so that the logic would be::

   if there is a descriptor for the attribute:
     if the descriptor's hasattr slot is populated:
       return the result of calling it
     else:
       return True
   else:
     look in the instance dict for the attribute

Guido indicated that he believes that a property that has a side
effect (other than caching, statistics, logging, and so forth) is a
misfeature anyway, so he doesn't have a problem with hasattr() trying
getattr() and reporting False if that raises an exception.  Discussion
died out before anything was resolved.

Contributing thread:

 - `hasattr and properties
<http://mail.python.org/pipermail/python-dev/2005-December/058498.html>`__

[TAM]

---------------------------------
Supporting iterables in xmlrpclib
---------------------------------

Skip Montanaro `presented a patch`_ to allow all currently supported
iterables (e.g. sets and arrays) to be marshalled as lists with
xmlrpclib.  The goals are to extend the set of sequences that can be
marshalled transparently, and keep the caller from caring as much
about the limitations of the XML-RPC datatypes.  Guido felt that this
was a bad idea, however; he feels that it's better to be aware of the
limitations in what XML-RPC can handle and try to handle it.

Contributing thread:

 - `Broader iterable support for xmlrpclib
<http://mail.python.org/pipermail/python-dev/2005-December/058474.html>`__

[TAM]

-----------------------------------------
Checking Jython support code into CPython
-----------------------------------------

Fredrik Lundh asked what the policy with respect to Jython-specific
modules in the standard library was.  Guido felt that as long as it
didn't do any harm (likely causing unit tests to fail) to CPython, it
would be fine.  He noted that traditionally Jython-specific code has
been checked into Jython's own source control, however, and Martin v.
L?wis indicated that this is what he would prefer.

 - `Jython and CPython
<http://mail.python.org/pipermail/python-dev/2005-December/058680.html>`__

[TAM]

-----------------------------------------
Getting rid of __builtins__ in Python 3.0
-----------------------------------------

Neal Norwitz asked Guido whether improving the confusing system of
having both __builtins__ and __builtin__ could be begun.  Guido
clarified that having both is clearly a bad idea, that he's not sure
that renaming builtin to __builtin__ was correct (and that perhaps it
should be changed back), that __builtins__ always being a dict would
simplify some code but need special handling of vars() in interactive
mode, and that another alternative might be to merge the __builtin__
and __builtins__ functionality (under the __builtin__ name).

Guido asked that people mull this over, but hasn't had any responses so far.

 - `__builtin__ vs __builtins__
<http://mail.python.org/pipermail/python-dev/2005-December/058652.html>`__

[TAM]

================
Deferred Threads
================

 - `Linked lists
<http://mail.python.org/pipermail/python-dev/2005-December/058754.html>`__


===============
Skipped Threads
===============

 - `Python bug day this Sunday
<http://mail.python.org/pipermail/python-dev/2005-December/058432.html>`__
 - `Subject lines of commit email
<http://mail.python.org/pipermail/python-dev/2005-December/058447.html>`__
 - `Proposed additional keyword argument in logging calls
<http://mail.python.org/pipermail/python-dev/2005-December/058449.html>`__
 - `os.normpath may change the meaning of the path if it contains
symbolic links?
<http://mail.python.org/pipermail/python-dev/2005-December/058452.html>`__
 - `SVN backup?
<http://mail.python.org/pipermail/python-dev/2005-December/058456.html>`__
 - `svn problem - can't get log info for a specific revision
<http://mail.python.org/pipermail/python-dev/2005-December/058462.html>`__
 - `Patch reviews &amp; request for patch review
<http://mail.python.org/pipermail/python-dev/2005-December/058470.html>`__
 - `Dynamic Link Library
<http://mail.python.org/pipermail/python-dev/2005-December/058472.html>`__
 - `[Python-checkins] commit of r41586 - in python/trunk:
Lib/SimpleXMLRPCServer.py Misc/NEWS
<http://mail.python.org/pipermail/python-dev/2005-December/058476.html>`__
 - `Short-circuiting iterators
<http://mail.python.org/pipermail/python-dev/2005-December/058484.html>`__
 - `Bug bz2.BZ2File(...).seek(0,2) + patch
<http://mail.python.org/pipermail/python-dev/2005-December/058524.html>`__
 - `imaplib module with IDLE implememted via threads
<http://mail.python.org/pipermail/python-dev/2005-December/058531.html>`__
 - `Exception type on handling closed files
<http://mail.python.org/pipermail/python-dev/2005-December/058573.html>`__
 - `A missing piece of information in weakref documentation
<http://mail.python.org/pipermail/python-dev/2005-December/058585.html>`__
 - `Directory for packages maintained outside the core (was Re:
ElementTree - Why not part of the core?)
<http://mail.python.org/pipermail/python-dev/2005-December/058592.html>`__
 - `Incorporating external packages into Python's std distribution
<http://mail.python.org/pipermail/python-dev/2005-December/058600.html>`__
 - `On moving to new-style classes
<http://mail.python.org/pipermail/python-dev/2005-December/058688.html>`__
 - `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-December/058719.html>`__
 - `Website cruft
<http://mail.python.org/pipermail/python-dev/2005-December/058729.html>`__
 - `Add timeout to subprocess.py?
<http://mail.python.org/pipermail/python-dev/2005-December/058784.html>`__
 - `patch tracker ping: cross compile and mingw support
<http://mail.python.org/pipermail/python-dev/2005-December/058803.html>`__
 - `Needed tester for patch in urllib.py module
<http://mail.python.org/pipermail/python-dev/2005-December/058817.html>`__
 - `Location of .so files (Was: Sharing expat instances)
<http://mail.python.org/pipermail/python-dev/2005-December/058824.html>`__

From tony.meyer at gmail.com  Thu Jan 12 05:08:08 2006
From: tony.meyer at gmail.com (Tony Meyer)
Date: Thu, 12 Jan 2006 17:08:08 +1300
Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-01 through
	2005-12-15
Message-ID: <6c63de570601112008o69989401n99487d15c8f7cd5e@mail.gmail.com>

Opps.  I just sent out the draft summary for the first half of
December (which might only make it to the list after this one, since
it's very long) but forgot to say anything at the top.

No-doubt everyone knows the pitch by now, but if anyone is able to
take a look at the summary (or parts of it) and send any
comments/suggestions/corrections/additions to either me or Steve, that
would be great.  Responding off-list would be best, to keep noise down
(unless you want to revitalise one of the threads, of course).

As always, thanks to all for your help!  The second half of December
draft should follow tomorrow, and hopefully the first January one next
week.

=Tony.Meyer

From rkiendl at gmx.net  Thu Jan 12 11:18:43 2006
From: rkiendl at gmx.net (Robert Kiendl)
Date: Thu, 12 Jan 2006 11:18:43 +0100
Subject: [Python-Dev] Unicode print/stdoutput exceptions are not nice
Message-ID: <43C62D03.9050909@gmx.net>

<1137059888.538391.119110 at o13g2000cwo.googlegroups.com>
Martin v. L?wis schrieb:

 > Robert wrote:
 > > is in a PythonWin Interactive session - ok results for cyrillic chars
 > > (tolerant mbcs/utf-8 encoding!).
 > > But if I do this on Win console (as you probably mean), I get also
 > > encoding Errors - no matter if chcp1251, because cyrillic chars raise
 > > the encoding errors also.
 >
 > If you do "chcp 1251" (not "chcp1251") in the console, and then
 > run python.exe in the same console, what is the value of
 > sys.stdout.encoding?

correctly: 'cp1252' in my case; cyrillic-chars break "print"   (on PC 
linux 2.2 tty py24 sys.stdout.encoding is None (?); only 
locale.getdef... useful? )

I live with this in site(customize):

import codecs
_stdout=sys.stdout
if sys.platform=='win32' and not 
sys.modules.has_key('pywin.framework.startup'):
     _stdoutenc=getattr(_stdout,'encoding',None) or sys.getdefaultencoding()
     try: codecs.lookup(_stdoutenc)
     except LookupError: _stdoutenc=sys.getdefaultencoding()
     class StdOut:
         def write(self,s): 
_stdout.write(s.encode(_stdoutenc,'backslashreplace'))
     sys.stdout=StdOut()
elif sys.platform.startswith('linux'):
     import locale
     _stdoutenc=locale.getdefaultlocale()[1]
     try: codecs.lookup(_stdoutenc)
     except LookupError: _stdoutenc=sys.getdefaultencoding()
     class StdOut:
         def write(self,s): 
_stdout.write(s.encode(_stdoutenc,'backslashreplace'))
     sys.stdout=StdOut()


 > > I think this is not a good behaviour of python to be so picky.
 >
 > I think it it is good.
 >
 >    Errors should never pass silently.
 >    Unless explicitly silenced.

A political question. Arguments:

* Webbrowsers for example have to display defective HTML as good as 
possible, unknown unicode chars as "?" and so on... Users got very angry 
in the beginning of browsers when 'strict' programmers displayed their 
exception error boxes ...

* at least the "print" statement has to go through - the costs (for 
angry users and developers; e.g. 
http://blog.ianbicking.org/do-i-hate-unicode-or-do-i-hate-ascii.html) 
are much higher when apps suddenly break in simple print/display-output 
when the system picks up alien unicode chars somewhere (e.g. 
occasionally in filenames,...). No one is really angry when occasionally 
chinese chars are displayed cryptically on non-chinese computers. One 
can investigate, add fonts, ... to improve, or do nothing in most cases, 
but apps do not break on every print statement! This is not only true 
for tty-output, but also for log-file redirect and almost any common 
situation for print/normal stdout/file-(write)-output.

* anything is nice-printable in python by default, why not 
unicode-strings!? If the decision for default 'strict' encoding on 
stdout stands, we have at least to discuss about print-repr for unicode.

* the need for having technical strings 'strict' is much more rare. And 
programmers are anyway very aware in such situations . e.g. by 
asciifile.write( us.encode(xy,'strict') )   .

* on Windows for example the (good) mbcs_encode is anyway tolerant as 
it: unkown chars are mapped to '?' . I never had any objection to this.

Some recommendations - soft to hard:

* make print-repr for unicode strings tolerant  (and in PythonWin alwasy 
tolerant 'mbcs' encoding)

* make stdout/files to have 'replace'-mode encoding by default. (similar 
as done with my code above)

* set site.py/encoding=('ascii', 'replace')   # if not utf-8/mbcs/locale 
;enable a tuple
* save sys._setdefaultencoding by default

* I would also live perfectly with .encode(enc) to run 'replace' by 
default, and 'strict' on demand. None of my apps and scripts would break 
because of this, but win. A programmer is naturally very aware when he 
wants 'strict'. Can you name realistic cases where 'replace' behavior 
would be so critical that a program damages something?


 > > In
 > > 1136925967.990106.299760 at g44g2000cwa.googlegroups.com I showed, how I
 > > solved this so far. Any better/portable idea?
 >
 > Not sure why you aren't using sys.stdout.encoding on Linux. I would do
 >
 > try:
 >   c = codecs.getwriter(sys.stdout.encoding)
 > except:
 >   c = codecs.getwriter('ascii')
 > sys.stdout = c(sys.stdout, 'replace')
 >
 > Also, I wouldn't edit site.py, but instead add sitecustomize.py.

I have more problems with the shape of sys.path in different situations, 
multiple sitecustomize.py on other apps, environments, OS / users, 
cxfreeze,py2exe  ...   sitecustomize not stackable easily: a horror 
solution. The need is for a callable _function_ or for general change in 
python behaviour.

modifiying site.py is better and stable for me (I have my 
patch/module-todo-list handy each time i install a new python), as I 
always want tolerant behaviour. in code i check for 
site.encoding/_setdefaultencoding (I save this). Thus i get one central 
error if setup is not correct, but not evil unicode-errors somewhere 
deep in the app once on a russian computer in the future...

 > > Yes. But the original problem is, that occasionally unicode strings
 > > (filenames in my case) arise which are not defined in the local
 > > platform encodings, but have to be displayed (in 'replace' encoding 
mode)
 > > without breaking the app flow. Thats the pain of the default behaviour
 > > of current python - and there is no simple switch. Why should "print
 > > xy" not print something _always_ as good and as far as possible?
 >
 > Because the author of the application wouldn't know that there
 > is a bug in the application, and that information was silently
 > discarded. Users might only find out much later that they have
 > question marks in places where users originally entered data,
 > and they would have no way of retrieving the original data.
 >
 > If you can accept that data loss: fine, but you should silence
 > the errors explicitly.

this is black/white theoretical - not real and practical (as python 
wants to be). see above.

Robert

From gabriel.becedillas at corest.com  Thu Jan 12 17:47:31 2006
From: gabriel.becedillas at corest.com (Gabriel Becedillas)
Date: Thu, 12 Jan 2006 13:47:31 -0300
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
Message-ID: <43C68823.3000105@corest.com>

Hi,
At the company I work for, we've embedded Python in C++ application we 
develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting 
Py_FatalError("Invalid thread state for this thread") when using debug 
builds.
We use both multiple interpreters and thread states.

I think the problem is that PyThreadState_Delete (which is used when I
destroy thread states and interpreters) is not making the same clean up
that PyThreadState_DeleteCurrent is doing (which is used when we create 
threads from python code). If a thread id is reused (obviously not 
between two running threads), and the previous thread used 
PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old 
and invalid value for autoTLSkey is still stored, and that is
triggering the Py_FatalError when a call to PyThreadState_Swap is made.
If I add this code at the end of PyThreadState_Delete:

if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
	PyThread_delete_key_value(autoTLSkey);

then everything works fine.
Could you please confirm if this is a bug ?
Thanks a lot and have a nice day.

From tony.meyer at gmail.com  Thu Jan 12 23:46:02 2006
From: tony.meyer at gmail.com (Tony Meyer)
Date: Fri, 13 Jan 2006 11:46:02 +1300
Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-16 through
	2005-12-31
Message-ID: <6c63de570601121446p390477a7q97a1486bada53d0d@mail.gmail.com>

Here's the second December summary.  As always, if anyone can spare
some time to take a look over it and send any
comments/suggestions/corrections/additions to me or Steve that would
be great.  I'm not all that confident about the "default comparisons"
thread, so particular attention to that would be great.  Thanks!

=============
Announcements
=============

----------------------------
QOTF: Quote of the Fortnight
----------------------------

Python-dev is in love with Python, though sometimes too much, Fredrik
Lundh contends:

    ...in reality, some things are carefully thought out and craftily
implemented, some things are engineering tradeoffs made at a certain
time, and some things are just accidents -- but python-dev will
happily defend the current solution with the same energy, no matter
what it is.

Contributing thread:

- `a quit that actually quits
<http://mail.python.org/pipermail/python-dev/2005-December/059283.html>`__

[SJB]

-------------------------------------------------------
Reminder: plain text documentation patches are accepted
-------------------------------------------------------

Want to help out with the Python documentation? Don't know LaTeX? No
problem! Plain text or ReST fixes are also welcome. You won't be able
to produce a diff file like with a normal patch, but comments that
explain how to fix the docs are just as good. A form like "in section
XXX right before the paragraph starting with YYY, the documentation
should say ZZZ" will make it easy for the doc maintainers to apply
your fix.

Contributing thread:

 - `LaTeX and Python doc contributions
<http://mail.python.org/pipermail/python-dev/2005-December/059001.html>`__

[SJB]

---------------------------------------------------------------
PyPy Sprint: Palma de Mallorca (Spain) 23rd - 29th January 2006
---------------------------------------------------------------

The next PyPy_ sprint is scheduled to take place from the 23rd to the
29th of January 2006 in Palma De Mallorca, Balearic Isles, Spain. 
There will be newcomer-friendly introductions and the focus will
mainly be on current JIT work, garbage collection, alternative
threading models, logic programming and on improving the interface
with external functions.

 .. _PyPy: http://codespeak.net/pypy

Contributing thread:

 - `Next PyPy Sprint: Palma de Mallorca (Spain) 23rd - 29th January
2006 <http://mail.python.org/pipermail/python-dev/2005-December/058975.html>`__

[TAM]

--------------------------------------------------
SHA-224, -256, -384 and -512 support in Python 2.5
--------------------------------------------------

Ronald L. Rivest asked about the cryptographic hash function support
in Python, now that md5 and sha1 are broken in terms of
collision-resistance.  The new-in-Python-2.5 hashlib module was
pointed out (and somewhat belatedly added to the NEWS file), which
includes new implementations for SHA-224, -256, -384 and -512 (these,
including tests, are already in SVN).

Gregory P. Smith noted that although the code isn't currently
available for earlier versions of Python, he does plan to release a
standalone package for Python 2.3 and Python 2.4, when time permits.

Contributing thread:

 - `hashlib - faster md5/sha, adds sha256/512 support
<http://mail.python.org/pipermail/python-dev/2005-December/058850.html>`__

[TAM]

=========
Summaries
=========

-----------------------------------
Improving the documentation process
-----------------------------------

Fredrik Lundh asked about automatically updating the development docs,
so that users wouldn't be required to have a latex toolchain installed
to get an HTML version of the current docs. This spawned a long
discussion about using something other than LaTeX (e.g. microformats_
or ReST_) for markup. Though HTML has the advantage that many Python
users are already familiar with it, a number of people voiced concerns
about the ease of reading and writing it.  ReST is generally easy to
read and write, but some people suggested that for complicated
structured text (like Python function and class definitions) it was no
better than LaTeX. Fredrik Lundh presented some example code
side-by-side_ and argued that using HTML with something like
PythonDoc_ could better represent the documentation's intent. He also
produced an initial conversion_ of the Python docs to this format.

Fredrik also pointed out that currently the doc patch submission
procedure is rather tedious, and listed_ the rather large number of
steps required for end-users and developers to get their documentation
fixes added to the current Python docs. He claimed that a simple wiki,
discussion board, or "user edit" mechanism like that of roundup_,
combined with automated updates of the documentation from the Python
SVN trunk, could reduce this procedure to two or three simple steps.
As a partial effort towards this goal, Trent Mick started posting
`daily builds`_ of the Python HTML. This prompted Neal Norwitz to set
up the docs server in a similar manner so that it now produces
development documentation builds twice daily at
http://docs.python.org/dev/.

.. _ReST: http://docutils.sourceforge.net/rst.html
.. _microformats: http://microformats.org/wiki/microformats
.. _side-by-side:
http://mail.python.org/pipermail/python-dev/2005-December/059022.html
.. _PythonDoc: http://effbot.org/zone/pythondoc.htm
.. _conversion: http://effbot.org/zone/pythondoc-lib.htm
.. _listed: http://mail.python.org/pipermail/python-dev/2005-December/059311.html
.. _roundup: http://roundup.sourceforge.net
.. _daily builds: http://trentm.com/python/

Contributing threads:

 - `status of development documentation
<http://mail.python.org/pipermail/python-dev/2005-December/058910.html>`__
 - `documentation comments
<http://mail.python.org/pipermail/python-dev/2005-December/058991.html>`__
 - `[Doc-SIG] status of development documentation
<http://mail.python.org/pipermail/python-dev/2005-December/058999.html>`__
 - `reST limitations? (was Re: status of development documentation)
<http://mail.python.org/pipermail/python-dev/2005-December/059016.html>`__
 - `that library reference, again
<http://mail.python.org/pipermail/python-dev/2005-December/059087.html>`__
 - `[Doc-SIG] that library reference, again
<http://mail.python.org/pipermail/python-dev/2005-December/059294.html>`__

[SJB]

--------------------------------------
Making quit and exit more command-like
--------------------------------------

Fredrik Lundh kicked off a surprisingly long thread when he proposed
that typing "quit" or "exit" in the interactive prompt actually exit
(i.e. raises SystemError) rather than printing a message informing the
user how they can exit, to avoid the "if you knew what I wanted, why
didn't you just do it?" situation.  His proposal was to install a
custom excepthook that looks for the appropriate NameErrors at the top
level (in interactive mode only).

Early discussion revolved around the implementation.  Skip Montanaro
worried that multiple code sources wanting to change sys.excepthook
would step on one another's toes; Fredrik felt that if you add your
own excepthook, you take responsibility.  Martin was also concerned
about this; although Fredrik believed that the change was so early
that no code would try to add a hook prior to this, Martin suggested
that the code stand as an example of good practice and pass execution
on to any existing excepthook (if the command wasn't "exit" or
"quit").

Reinhold Birkenfeld suggested that exit and quit could be instances of
a class whose repr() raises SystemExit; however, this would cause an
exit to be incorrectly performed whenever the exit or quit objects
were examined (e.g. calling vars()).  Reinhold suggested a variation
where the SystemExit would only be raised if
sys._getframe(1).f_code.co_names was "exit" or "quit").  Having quit
and exit be functions (i.e. one would type "exit()" to exit) was also
suggested, with a plain "exit" printing a help message (as today). 
However, while this shortens the platform-independent method of
exiting ("raise SystemExit") to "exit()", it does nothing to address
the "why didn't you just do it?" problem.

Ka-Ping Yee was concerned that Fredrik's implementation would cause an
exit in surprising situations, such as ::

    print exit                  # seems surprising
    [3, 4, 5, exit]             # seems surprising
    'a' in 'xyz' or exit        # desirable or undesirable?
    del exit                    # actually happened to me
    x = exit                    # seems surprising

And that Reinhold's modified proposal would do likewise::

    print exit                  # seems surprising
    [3, 4, 5, exit]             # seems surprising
    'a' in 'xyz' or exit        # desirable or undesirable?
    def foo():
        print exit
    foo()                       # seems surprising

As such, Fredrik proposed adding a new sys attribute that contains the
most recent interpreter line, which could be examined to check that
the line entered was simply "exit" or "quit" by itself.  Skip
suggested that such an attribute be made clearly private magic (e.g.
by naming it sys._last_input).

Michael Hudson suggested that the clever hacks could be avoided by
simply having the equivalent of "if input.rstrip() == "exit": raise
SystemExit" in the implementation of the interactive interpreter. 
Fredrik felt that this would elevate "exit" and "quit" into reserved
keywords, in that this could occur::

    >>> def exit():
    ...      print "bye"

    >>> # what is it?
    >>> exit

    $ oops!

(However, one could simply type "repr(exit)" in this case).

Walter D?rwald suggested adding "sys.inputhook" (c.f. sys.displayhook,
sys.excepthook), which gets each statement entered and returns True if
it processed the line or False if normal handling should be done; Alex
Martelli made a similar suggestion.  This would not only allow special
handling for "exit", "quit", and "help", but also might make
implementing alternative shells easier.  Nick Coghlan added a default
handler to this suggestion, which checked a new dictionary,
"sys.aliases", for special statements of this type and called an
appropriate function if found.  Fredrik's concern with this method was
the same as with Michael Hudson's - that typing "exit" as a shortcut
for "print repr(exit)" (when the user had assigned a value to "exit")
would cause the shell to exit.  As a result, Nick modified his default
inputhook to check that the statement didn't exist in __main__'s
vars().  Fernando Perez pointed out that this begins to come close to
reimplementing ipython_.

Fernando also commented that he felt that the amount of code he
thought would be necessary to avoid any side-effects when determining
whether "exit" should exit or not would make the standard interpreter
too brittle, and that this should be left to third-party interpreters.
 He gave a lengthy explanation of the process that ipython went
through, including linking to `the _prefilter method` that performs
the appropriate magic in ipython.  His opinion was that interpreters
should have a separate command mode, entered by a particular character
(e.g. the '%' used in ipython); while this would simply code, '%exit'
would be even harder for people to guess, however.

Some people were concerned that "quit" and "exit" would be treated
specially (i.e. the standard way to do something is to "call a
function" and a special case for exiting the interpreter is
inappropriate); others pointed out that this would be there especially
for people who don't know how to program in Python, and might not know
that typing an EOF would exit the shell (or what the EOF character was
for their platform).

Fredrik's proposal gained a lot of early support, but later posters
were not so keen.  Stephen J. Turnbull suggested that the problem was
simply that the current "help"/"exit" messages are impolite, and
changing the language would solve the problem.  Similarly, Aahz
suggested that adding an explanation of how to quit to the startup
message would suffice, although others disagreed that users would
notice this, particularly when it was needed.

Despite the strong support, however, in the end, Guido stepped in and
stated that, in his opinion, nothing was wrong wih the status quo (or
that if anything were to be changed, the hints provided by typing
"exit" or "quit" should be removed).  This pretty much ended the
discussion.

.. _ipython: http://ipython.scipy.org
.. _`the _prefilter method`:
http://projects.scipy.org/ipython/ipython/file/ipython/trunk/IPython/iplib.py

Contributing threads:

 - `a quit that actually quits
<http://mail.python.org/pipermail/python-dev/2005-December/059094.html>`__

---------------------------------------
Exposing the Subversion revision number
---------------------------------------

Barry Warsaw proposed `a fairly simple patch`_ to expose the
Subversion revision number to Python, both in the Py_GetBuildInfo()
text, in a new Py_GetBuildNumber() C API function, and via a new
sys.build_number attribute.

A lengthy discussion took place about the correct method of
determining the "revision number" that should be used.  There are many
possibilities, from the originally proposed revision number in which
the directory was last modified, to the latest revision number within
the tree, to using subversion itself.  Each method requires a
different amount of processing in order to determine the correct
value.  Barry indicated a preference for the original, simple, method,
because he wished to keep the patch as simple as possible, didn't want
to depend on Python already being built, and felt that generally
developers will just 'svn up' at the top of the source tree then
rebuild, rather than 'svn up' a buried sub-tree, change back to the
top directory, and rebuild from there.

Phillip J. Eby's concern with this was that this would not indicate if
a change has been made locally and committed, unless the developer
also did a 'svn up'.  He also pointed out that the "Revision" from
'svn info' can change when the code doesn't change (because it will
change when code in branches, the sandbox, PEPs, and so forth change).

In the end, using the svnversion command was agreed on as a suitable
method.  The two main advantages are that this is the officially
sanctioned (by Subversion) method, and that it indicates when local
changes have been made.

Armin Rigo voiced a concern that people would use sys.build_number to
check for features in their programs, instead of using
sys.version_info (or duck-typing methods such as hasattr()) because it
seems that comparing a single number is easier than comparing a tuple.
 This was of especial concern considering that this value would only
be present in CPython.  He suggested that a new build_info attribute
could be added to sys instead, which would be a tuple of ("CPython",
"<SVN revision number>", "trunk"); that is, it would also include
which Python implementation the program is using (for particular use
by the test suite).  The name also has advantages in that the number
is actually a string (because 'M' will be present if there are local
modifications) and it is not tied to a specific version control
system.

Michael Hudson pointed out that this value will not be able to be
obtained when Python is built from an 'svn export' (where the .svn
files do not exist), which is probably the case for many Python
distributions.  He wondered whether a subversion trigger could put the
revision number into some file in the repository to cover this case,
but Martin indicated that this would be difficult (although if the
distribution is built from a tag, that the information would not be
necessary).  Barry suggested that the number would be the latest
revision at which getbuildinfo was changed if a .svn directory isn't
found.

As an aside to this patch, Martin suggested that the build number
should be dropped, as it stopped working on Windows some time ago and
no longer provides any useful information.  There was no opposition to
this proposal, and some support.

 .. _a fairly simple patch: http://python.org/sf/1382163

Contributing threads:

 - `Expose Subversion revision number to Python
<http://mail.python.org/pipermail/python-dev/2005-December/058832.html>`__

[TAM]

---------------------------------------
Python's lists, linked lists and deques
---------------------------------------

Martin Blais asked why Python didn't have native singly-linked or
doubly-linked list types.  Generally, it seemed that people couldn't
find use cases for them that weren't covered by Python's builtin
lists, collections.deque objects, or head-plus-tail-style two-tuples.
The proposal was redirected to comp.lang.python until it was more
developed.

In a related thread, Christian Tismer asked if Python's list type
could do a little usage-tracking and switch between the current
array-based implementation and a deque-based implementation if, say, a
large number of inserts or deletes began to occur at the beginning of
the list. He got a few responses suggesting that it would be more
complication that it was worth, but mostly his question got drowned
out by the linked-list discussion.

Contributing threads:

 - `Linked lists
<http://mail.python.org/pipermail/python-dev/2005-December/058754.html>`__
 - `deque alternative (was: Linked lists)
<http://mail.python.org/pipermail/python-dev/2005-December/059054.html>`__
 - `deque alternative
<http://mail.python.org/pipermail/python-dev/2005-December/059076.html>`__
 - `(back to): Linked lists -- (was: deque alternative)
<http://mail.python.org/pipermail/python-dev/2005-December/059119.html>`__

[SJB]

------------------------
set() memory consumption
------------------------

Noam Raphael noted that sets (and dicts) do not release allocated
memory after calls to pop(). He suggested that to keep pop() as
amortized O(1) per delete, all that was needed was to resize the table
to half its current size whenever it dropped to less that one quarter
full. People seemed generally to think that the dict implementation
(and therefore the set implementation which is based on it) were
already highly optimized for real-world performance, and that there
were too few use cases where a set would be filled and then emptied,
but not filled up again.

Contributing thread:

 - `When do sets shrink?
<http://mail.python.org/pipermail/python-dev/2005-December/059228.html>`__

[SJB]

---------------------------------------------------------
Changing the naming conventions of builtins in Python 3.0
---------------------------------------------------------

Ka-Ping Yee provocatively suggested that, to be consistent with the
recommended style guide, in Python 3.0 built-in constants (None, True,
False) should be in all caps (NONE, TRUE, FALSE), and built-in classes
(object, int, float, str, unicode, set, list, tuple, dict) should be
in CapWords (Object, Int, Float, Str, Unicode, Set, List, Tuple,
Dict).  However, Michael Chermside noted that there is a common
convention that the fundamental built-in types (not standard library
types; e.g. set, not sets.Set) are all in lowercase.

Almost no-one liked this idea, and many people were vehemently
opposed.  Guido quickly pronounced this dead for built-in types
(cleaning up and making the more consistent the standard library
classes is another matter).

Contributing thread:

 - `Naming conventions in Py3K
<http://mail.python.org/pipermail/python-dev/2005-December/059304.html>`__

[TAM]

-------------------
Default comparisons
-------------------

Jim Jewett pointed out that `PEP 3000`_ now suggests that dropping
default comparison has become more than an idle what-if, which means
that the common use case of comparisons to get a canonical order (not
necessarily one that makes sense) will no longer work in many cases. 
An alternative solution would be to promote a standard way to say
"just get me a canonical ordering of some sort".  "Comparison" would
raise a TypeError if the pair of objects were not comparable;
"Ordering" would continue on to "something consistent", just like
sorts do today.

Josiah Carlson suggested that this could be achieved by new
superclasses for all built-in types (except string and unicode, which
already subclass from basestring).  int, float, and complex would
subclass from basenumber; tuple and list would subclass from
basesequence; dict and set would subclass from basemapping.  Each of
these base classes define a group in which items are comparable; if
you end up in a situation in which the base classes of the compared
object differ, you compare their base class name.  If a user-defined
classes wanted to be compared against (e.g.) ints, floats, or complex,
the user would subclass from basenumber; if the user only wanted their
objects to compare against objects of its own type, they define their
own __cmp__.  However, Marc-Andre Lemburg pointed out that Python
already uses this 'trick' based on the type name (falling back to
id(), which is the problem), and that inheriting from (e.g.)
basenumber could result in unexpected side-effects.

.. _PEP 3000: http://www.python.org/peps/pep-3000.html

Contributing thread:

 - `Keep default comparisons - or add a second set?
<http://mail.python.org/pipermail/python-dev/2005-December/058899.html>`__

[TAM]

---------------------------------------
Converting open() to a factory function
---------------------------------------

Guido had previously_ indicated that while file() will always stay the
name of the file object, open() may be changed in the future to a
factory function (instead of just being an alias for file). Noting
that ``help(open)`` now just displays the file object documentation,
Aahz suggested that open() become a factory function now. Fredrik
Lundh chimed in to suggest that a textopen() function should also be
introduced, which would call codecs.open() or file() depending on
whether or not an encoding was supplied. There was mild support for
both proposals, but no patches were available at the time of this
writing.

.. _previously:
http://mail.python.org/pipermail/python-dev/2004-July/045921.html

Contributing thread:

 - `file() vs open(), round 7
<http://mail.python.org/pipermail/python-dev/2005-December/059073.html>`__

[SJB]

-----------------------------
NotImplemented and __iadd__()
-----------------------------

Facundo Batista presented a bug_ where using += on Decimal objects
returned NotImplemented instead of raising an exception. Armin Rigo
was able to supply a patch after adding the following conditions to
the documentation:

* PySequence_Concat(a, b) and operator.concat(a, b) mean "a + b", with
the difference that we check that both arguments appear to be
sequences (as checked with operator.isSequenceType()).
* PySequence_Repeat(a, b) and operator.repeat(a, b) mean "a * b",
where "a" is checked to be a sequence and "b" is an integer. Some
bounds can be enforced on "b" -- for CPython, it means that it must
fit in a C int.

.. _bug: http://www.python.org/sf/1355842

Contributing thread:

 - `NotImplemented reaching top-level
<http://mail.python.org/pipermail/python-dev/2005-December/059046.html>`__

[SJB]

---------------------------------------------------
Using buildbot to automate testing of Python builds
---------------------------------------------------

Contributing thread:

 - `Automated Python testing (was Re: status of development
documentation) <http://mail.python.org/pipermail/python-dev/2005-December/059060.html>`__

-----------------------------------------------------
Importing C++ extensions compiled with Visual C++ 8.0
-----------------------------------------------------

Ralf W. Grosse-Kunstleve indicated that while they could compile their
C++ extensions with Visual C++ 8.0, importing any such extensions
resulted in an error::

    This application has failed to start because MSVCP80.dll was not
found. Re-installing the application may fix this problem.

Though the combination of compiling Python with VS.NET2003 and an
extension with VS2005 is not supported, Ralf was able to get things to
work by adding ``/manifest`` to the linker options and envoking
``mt.exe`` to embed the manifest.

Contributing thread:

 - `Python + Visual C++ 8.0?
<http://mail.python.org/pipermail/python-dev/2005-December/059085.html>`__

[SJB]

-------------------------------
Using ssize_t as the index type
-------------------------------

Martin v. L?wis presented a new (as yet unnumbered) PEP to use ssize_t
as the index type in the CPython implementation. In Python 2.4,
indices of sequences are restricted to the C type int. On 64-bit
machines, sequences therefore cannot use the full address space, and
are restricted to 2**31 elements. The PEP proposes to change this,
introducing a platform-specific index type Py_ssize_t. An
implementation of the proposed change is in `an SVN branch`_; the
Py_ssize_t will have the same size as the compiler's size_t type, but
is signed (it will be a typedef for ssize_t where available).

Although at the moment one needs 16GiB to just hold the pointers of a
2GiElement list, these changes also allow strings and mmap objects
longer than 2GiB (and also improve the scipy_core (ne? Numarray and
Numerical) objects, which are already 64-bit clean).  Since the
proposed change will cause incompatibilities on 64-bit machines, the
reasoning is that this change should be done as early as possible
(i.e. while such machines are not in wide use).

Guido approved the PEP, indicating he believed it was long overdue. 
Several other people also indicated support for the PEP.

Note that people felt that Martin's PEP was of a particularly high
standard; people considering writing their own PEP should consider
reading this (once it has a number and is available online!) to gain
an understanding of how a good PEP is both extremely readable (even
when including arcane C programming points!) and well-organised.

 .. _an SVN branch: http://svn.python.org/projects/python/branches/ssize_t

Contributing thread:

 - `New PEP: Using ssize_t as the index type
<http://mail.python.org/pipermail/python-dev/2005-December/059266.html>`__

[TAM]

-----------------------------------------------------------------------------
Garbage collection based on object memory consumption instead of object count
-----------------------------------------------------------------------------

Andrea Arcangeli suggested that Python's garbage collection could be
improved by invoking a gc.collect() at least once every time the
amount of anonymous memory allocated by the interpreter increases by a
tunable factor (defaulting to 2, meaning the memory doubles, and with
a factor of 1 mimicking the current state, where collection is done if
1000 new objects are allocated).  Aahz and Martin v. L?wis indicated
that it was extremely unlikely that anything would be done about this
unless Andrea submitted a patch (with doc patches and tests).

Contributing thread:

 - `suggestion for smarter garbage collection in function of size
(gc.set_collect_mem_growth(2))
<http://mail.python.org/pipermail/python-dev/2005-December/059184.html>`__

[TAM]

----------------------------------
Adding zlib module to python25.dll
----------------------------------

Thomas Heller and Martin Heller had been discussing whether the zlib
module should become builtin, at least on Windows (i.e. part of
python25.dll). This would simplify both the build process and py2exe,
the latter could then bootstrap extraction from the compressed file
just with pythonxy.dll, but this is currently not done, because the
pythoncore.vcproj would then not be buildable anymore unless you also
have the right version of zlib on disk.  To solve this problem, Thomas
proposed that the Python release could incorporate a copy of zlib,
primarily for use on Windows (with the project files appropriately
adjusted).  This change was later made.

Contributing thread:

 - `Incorporation of zlib sources into Python subversion
<http://mail.python.org/pipermail/python-dev/2005-December/058873.html>`__

[TAM]

----------------------------------------
Procedure for fixing bugs in ElementTree
----------------------------------------

Neal Norwitz found a few bugs in ElementTree and asked what the
procedure was, since the ElementTree module is maintained externally
by Fredrik Lundh. The answer was to post the bug to sourceforge as
usual, and and assign it to Fredrik Lundh. The ElementTree in the
Python SVN trunk should only have critical patches committed - all
other patches would be applied by Fredrik to his externally
distributed ElementTree package, and imported to the Python SVN trunk
after the next release of ElementTree.

Contributing thread:

 - `ref leak in element tree/pyexpat
<http://mail.python.org/pipermail/python-dev/2005-December/058872.html>`__

[SJB]


===============
Skipped Threads
===============
 - `PEP 8 updates/clarifications, function/method style
<http://mail.python.org/pipermail/python-dev/2005-December/058835.html>`__
 - `Test branch for ssize_t changes
<http://mail.python.org/pipermail/python-dev/2005-December/058863.html>`__
 - `DRAFT: python-dev summary for 2005-11-16 to 2005-11-31
<http://mail.python.org/pipermail/python-dev/2005-December/058869.html>`__
 - `[Python-checkins] commit of r41497 -python/trunk/Lib/test
<http://mail.python.org/pipermail/python-dev/2005-December/058871.html>`__
 - `fresh checkout won't build
<http://mail.python.org/pipermail/python-dev/2005-December/058876.html>`__
 - `fixing log messages
<http://mail.python.org/pipermail/python-dev/2005-December/058883.html>`__
 - `synchronized enumerate
<http://mail.python.org/pipermail/python-dev/2005-December/058896.html>`__
 - `(no subject)
<http://mail.python.org/pipermail/python-dev/2005-December/058916.html>`__
 - `Build failure and problem on Windows
<http://mail.python.org/pipermail/python-dev/2005-December/058918.html>`__
 - `os.startfile with optional second parameter
<http://mail.python.org/pipermail/python-dev/2005-December/058919.html>`__
 - `[OT] Fwd: a new python port: iPod
<http://mail.python.org/pipermail/python-dev/2005-December/058920.html>`__
 - `Sets are mappings?
<http://mail.python.org/pipermail/python-dev/2005-December/058923.html>`__
 - `Patch to make unittest.TestCase easier to subclass
<http://mail.python.org/pipermail/python-dev/2005-December/058995.html>`__
 - `timeout options in high-level networking modules
<http://mail.python.org/pipermail/python-dev/2005-December/058996.html>`__
 - `Patch reviews &amp; request for patch review
<http://mail.python.org/pipermail/python-dev/2005-December/059027.html>`__
 - `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2005-December/059031.html>`__
 - `A few questions about setobject
<http://mail.python.org/pipermail/python-dev/2005-December/059091.html>`__
) - `Small any/all enhancement
<http://mail.python.org/pipermail/python-dev/2005-December/059127.html>`__
 - `floating point literals don't work in non-US locale in 2.5
<http://mail.python.org/pipermail/python-dev/2005-December/059176.html>`__
 - `JOB OPENING: Implementor for Python and Search
<http://mail.python.org/pipermail/python-dev/2005-December/059183.html>`__
 - `PyCon TX 2006: Early-bird registration ends Dec. 31!
<http://mail.python.org/pipermail/python-dev/2005-December/059195.html>`__
 - `set.copy documentation string
<http://mail.python.org/pipermail/python-dev/2005-December/059222.html>`__
 - `Bug in Py_InitModule4
<http://mail.python.org/pipermail/python-dev/2005-December/059272.html>`__
 - `floating point literals don't work in non-USlocale in 2.5
<http://mail.python.org/pipermail/python-dev/2005-December/059290.html>`__
 - `slight inconsistency in svn checkin email subject lines
<http://mail.python.org/pipermail/python-dev/2005-December/059335.html>`__
 - `Gaming on Sunday (Jan 1)
<http://mail.python.org/pipermail/python-dev/2005-December/059339.html>`__

From allison at shasta.stanford.edu  Fri Jan 13 00:07:21 2006
From: allison at shasta.stanford.edu (Dennis Allison)
Date: Thu, 12 Jan 2006 15:07:21 -0800 (PST)
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>
Message-ID: <Pine.LNX.4.44.0601121446380.3707-100000@shasta.stanford.edu>


Brett,

Where are you doing your Phd and who will be your likely supervisor? 
It does make a difference.

Your dissertation idea list seems to me to focus on implementation
projects and not on research.  Usually a dissertation proceeds from a
hypothesis leading to an experiment, some measurements, and conclusions.  
Many universities tacitly expect a theorem and associated proof. The goal
of dissertation research is a completed approved dissertation, not some
intergalactic all encompassing contribution to human knowledge.  
Dissertation research should, IMHO, focus on a small, manageable problem.

Frequently, universities have a large cooperative project involving many 
graduate students, each of whom research a small, well defined topic areas 
related to the larger project.   Is this the case for your dissertation?

What are your interests?  Are you interested in language structure, 
language implementation, the formal interface between languages and 
applications, or what.   

Like most folks who lurk on this list, I have my list of things I'd like 
to see someone research.  Tell us a bit more and I am sure you'll get lots 
of us to share.

-d

On Thu, 12 Jan 2006, Brett Cannon wrote:

> It is time once again in my educational career to come to python-dev
> for help for major project ideas.  This time, though, it is for my
> Ph.D. dissertation (and thus can have larger scope than my masters
> thesis) but there are funding restrictions (and thus only certain
> areas I have possible funding for).
> 
> First off, there are two areas where I have possible funding: XML
> integration into programming languages and game scripting support (I
> have a third, AOP for anything, but  I don't think AOP is a good match
> for Python and thus not considering it for Python work).  The XML
> integration I have no direct ideas since I don't use XML.  There is
> the possibility of doing something like the LINQ project
> (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp)
> or something, but I don't know how useful that could be.  It would
> have to be just as generic, though, as the LINQ project in terms of
> working with intefaces to be Pythonic so it might turn out to be more
> about language support for querying data.
> 
> For the game scripting, trying out the active objects for Python is a
> possibility (or something else that would help with concurrency). 
> Since that could allow for agent objects to be more concurrent it can
> be argued that it would help with game scripting.  I don't have any
> other ideas for the area of game scripting support.
> 
> Now I am not locked down to these areas, but they are what I have
> possible grant money for.  If through some miracle someone out there
> wants to fund my Ph.D. and wants something else worked on for Python,
> don't be shy.  =)  Unfortunately I don't think the PSF is up for
> funding my Ph.D., else I would say I would work on whatever python-dev
> decided they wanted to worked on that could be done as a dissertation.
> 
> Anyway, any ideas are appreciated.  There is no rush on this, either. 
> Just getting this out there now while I am tentatively exploring
> possible topics.
> 
> -Brett
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40sumeru.stanford.edu
> 

-- 


From goodger at python.org  Fri Jan 13 05:29:19 2006
From: goodger at python.org (David Goodger)
Date: Thu, 12 Jan 2006 23:29:19 -0500
Subject: [Python-Dev] r42015 - peps/trunk
In-Reply-To: <1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com>
References: <20060112033316.E24C31E4002@bag.python.org>	<43C5D237.3040006@python.org>
	<43C63D32.9070206@egenix.com>	<43C65A2D.2050807@python.org>
	<43C677CD.3030709@egenix.com>	<4335d2c40601120756l34c4a16fvf4b1d1b7b376d939@mail.gmail.com>	<43C6809E.306@egenix.com>	<4335d2c40601120904rd45400cy2c09bcc9d3d66197@mail.gmail.com>	<43C6AF5E.7020803@egenix.com>
	<1f7befae0601121141m313d836etb0ba912821844bdf@mail.gmail.com>
Message-ID: <43C72C9F.6040700@python.org>

> [David Goodger]
>> An alternative is to use svn:externals to link to a specific
>> revision (via the -r option);

[Tim Peters]
> Yes. please.

Done.

-- 
David Goodger <http://python.net/~goodger>

-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 249 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060112/19055cab/attachment.pgp 

From aahz at pythoncraft.com  Fri Jan 13 05:36:49 2006
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 12 Jan 2006 20:36:49 -0800
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <43C68823.3000105@corest.com>
References: <43C68823.3000105@corest.com>
Message-ID: <20060113043649.GA16672@panix.com>

On Thu, Jan 12, 2006, Gabriel Becedillas wrote:
>
> If I add this code at the end of PyThreadState_Delete:
> 
> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
> 	PyThread_delete_key_value(autoTLSkey);
> 
> then everything works fine.
> Could you please confirm if this is a bug ?

Regardless of whether you get any other responses, please go ahead and
file a SourceForge bug report so that we don't lose track of this.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From theller at python.net  Fri Jan 13 07:51:20 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 13 Jan 2006 07:51:20 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <fynte63c.fsf@python.net>
	<43C6E0E1.5010705@v.loewis.de>
Message-ID: <1wzc62on.fsf@python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Thomas Heller wrote:
>> Sounds good, although it should be noted that ctypes is a package now,
>> with a ctypes.wrap subpackage (contains the code generator), a
>> ctypes.macholib subpackage (contains Bob Ippolitos macholib for OS X),
>> and ctypes.test subpackage whcih contains several test modules.  Plus
>> the _ctypes.(dll|so) extension module.
>
> Ok. The installer should then combine them all into a feature.
>
> Still, the admin could disable all of this just by removing _ctypes.dll,
> right?

Right.

Thomas


From theller at python.net  Fri Jan 13 07:56:56 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 13 Jan 2006 07:56:56 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-16 through
	2005-12-31
References: <6c63de570601121446p390477a7q97a1486bada53d0d@mail.gmail.com>
Message-ID: <vewo4nuv.fsf@python.net>

Tony Meyer <tony.meyer at gmail.com> writes:

> ----------------------------------
> Adding zlib module to python25.dll
> ----------------------------------
>
> Thomas Heller and Martin Heller had been discussing whether the zlib

Should be 'Martin v. L?wis', as we all know.

> module should become builtin, at least on Windows (i.e. part of
> python25.dll). This would simplify both the build process and py2exe,
> the latter could then bootstrap extraction from the compressed file
> just with pythonxy.dll, but this is currently not done, because the
> pythoncore.vcproj would then not be buildable anymore unless you also
> have the right version of zlib on disk.  To solve this problem, Thomas
> proposed that the Python release could incorporate a copy of zlib,
> primarily for use on Windows (with the project files appropriately
> adjusted).  This change was later made.
>
> Contributing thread:
>
>  - `Incorporation of zlib sources into Python subversion
> <http://mail.python.org/pipermail/python-dev/2005-December/058873.html>`__


From ianb at colorstudy.com  Fri Jan 13 08:05:44 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Fri, 13 Jan 2006 01:05:44 -0600
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
References: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>	<-8397309676539309595@unknownmsgid>
	<bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
Message-ID: <43C75148.1060003@colorstudy.com>

Brett Cannon wrote:
> Hmm.  It's an idea.  I also thought of Python -> JavaScript compiler
> since JavaScript is not fun and getting to test using Python might be
> cool.  But not sure how useful that would be.  Plus I bet someone has
> does this with Java or something.

There's a Python project that compiles to Javascript: 
http://www.aminus.org/blogs/index.php/phunt/2005/10/06/subway_s_new_ajax_framework

However, I think this is a bad idea.  The world doesn't need another 
compiled language with leaky abstractions that is next to impossible to 
debug.

Anyway, to chime in about something I'd really like to see, that maybe 
fits in somewhere between graphics and language implementation, is the 
ideas of Boxer (http://dewey.soe.berkeley.edu/boxer.html/) implemented 
for Python.  Basically meaning that some of what we represent with 
indentation in Python would be represented graphically (Boxer uses, 
unsurprisingly, boxes).  Also a kind of window in the runtime of the 
system -- variables in Boxer aren't names, but visual elements you can 
see as the program runs.  From what I've seen of HyperCard -- or at 
least what I choose to pay attention to ;) -- I think this kind of 
concrete runtime is part of its appeal.  I have a lot of more specific 
ideas related to this, if the idea interests you.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From mal at egenix.com  Fri Jan 13 10:59:49 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri, 13 Jan 2006 10:59:49 +0100
Subject: [Python-Dev] Py_ssize_t output parameters
 (Was:	[Python-checkins] r41971 ...)
In-Reply-To: <43C6D9D1.7040101@v.loewis.de>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>	<43C2DD44.2040908@v.loewis.de>	<43C37C92.3060302@egenix.com>	<43C4083F.3070001@v.loewis.de>	<43C421A3.2030806@egenix.com>	<43C425DE.2050001@v.loewis.de>
	<43C63458.10804@egenix.com> <43C6D9D1.7040101@v.loewis.de>
Message-ID: <43C77A15.5050803@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>> What if x64 has a 64-bit value ? How do you catch
>> and process the truncation error ?
> 
> We were *both* discussing a scenario where no sizes
> exceed 2**31, right?

Right, but I don't see the point of each and every
extension having to go through these hoops when you
could add support for these checks (including error
reporting) to the few APIs in question, in particular
the PyArg_ParseTuple() API.

> Under such a scenario, this just won't happen.
> 
> OTOH, if you were discussing a scenario where sizes
> might exceed 2**31, then I don't see why you are worried
> about Py_ssize_t* parameters alone: Even
> 
>   PyString_Size()
> 
> might (of course!) return a value > 2**31 - so it
> is not just output parameters, but also return values.

Indeed - which is why I'm going to convert our tools to
Py_ssize_t throughout.

I don't expect this to happen any time soon for the ten or
twenty other 3rd party extensions we regularly use and this
would prevent an upgrade to Python 2.5.

> For more information, please read "Conversion guidelines"
> in
> 
> http://www.python.org/peps/pep-0353.html

BTW, the open issue should be reworded:

In particular, functions that currently take int* output
parameters should continue to do so. New functions should be
revised to enable Py_ssize_t* output arguments and preseve
backwards compatibility.

(This also includes the strategy to be used, so you can
remove the note on strategy)

Please also add a comment on the fact that extensions
which haven't been recompiled for the Python 2.5 API
will not get imported (the API_LEVEL check should consistently
fail for these instead of just issuing a warning).

>>> That is not necessary. Can you give in example of a module
>>> where you think it is necessary?
>>
>> If you want to port the extension to Py_ssize_t, this
>> is essentially the case.
>>
>> You might want to take the unicodeobject.c file as
>> example.
> 
> unicodeobject.c is not an extension. We were discussing
> existing extension modules.
> 
>> We could use the type flags for these. much like we
>> do for the new style numbers.
> 
> Would you like to write a specification for that?

Sure, if there's hope to get this into the code.

>> If you don't like the macro approach, why not simply
>> leave the two separate sets of APIs visible.
> 
> To simplify the porting.

Sorry, I lost you there:

I'm saying that two sets of APIs (one for int*, one
for Py_ssize_t*) will make porting easier since
updated code will only have to rename the APIs
used. Furthermore, it will allow to write code
that easily works in Python 2.1-2.4 as well as
Python 2.5.

>> All Py_ssize_t aware and compatible extensions
>> would use the new APIs instead. The old-style APIs
>> should then take care of the proper down-casting and
>> error reporting.
> 
> That is not possible. Some API does not support
> error reporting (like PyString_Size()). So callers
> don't expect it to fail.

I'm talking about the few APIs that use output parameters.
Those do support error reporting.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 13 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ncoghlan at gmail.com  Fri Jan 13 10:59:58 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jan 2006 19:59:58 +1000
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dq6s50$mc4$1@sea.gmane.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>	<20060111195900.GB17269@activestate.com>
	<dq6s50$mc4$1@sea.gmane.org>
Message-ID: <43C77A1E.7010808@gmail.com>

Scott David Daniels wrote:
> OK I carried the code I offered earlier in this whole thread (tweaked in
> reaction to some comments) over to comp.lang.python, gathered some
> feedback, and put up a recipe on the cookbook.  After a week or so for
> more comment, I'll be happy to submit a patch to include the broken_test
> decorator function in unittest.
> 
> Here is where the recipe is, for those who want to comment further:
>      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288

Looks pretty handy to me - I've certainly experienced the "I know what the 
problem is and exactly how to provoke it, I just don't have time to fix it 
right now" problem myself.

I added a minor comment about testing for whether there were user supplied 
exceptions at definition time, rather than run time.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fredrik at pythonware.com  Fri Jan 13 11:04:35 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 13 Jan 2006 11:04:35 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2005-12-16
	through2005-12-31
References: <6c63de570601121446p390477a7q97a1486bada53d0d@mail.gmail.com>
Message-ID: <dq7tvl$m4m$1@sea.gmane.org>

Tony Meyer wrote:

> Fredrik Lundh kicked off a surprisingly long thread when he proposed
> that typing "quit" or "exit" in the interactive prompt actually exit
> (i.e. raises SystemError)

SystemExit

>     >>> def exit():
>     ...      print "bye"
>
>    >>> # what is it?
>     >>> exit
>
>     $ oops!
>
> (However, one could simply type "repr(exit)" in this case).

too late:

    >>> exit
    $ repr(exit)
    bash: syntax error near unexpected token `repr(exit)'

</F>




From mwh at python.net  Fri Jan 13 11:07:53 2006
From: mwh at python.net (Michael Hudson)
Date: Fri, 13 Jan 2006 10:07:53 +0000
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <43C68823.3000105@corest.com> (Gabriel Becedillas's message of
	"Thu, 12 Jan 2006 13:47:31 -0300")
References: <43C68823.3000105@corest.com>
Message-ID: <2mwth4qw3q.fsf@starship.python.net>

Gabriel Becedillas <gabriel.becedillas at corest.com> writes:

> Hi,
> At the company I work for, we've embedded Python in C++ application we 
> develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting 
> Py_FatalError("Invalid thread state for this thread") when using debug 
> builds.
> We use both multiple interpreters and thread states.

I know you've said this to me in email already, but I have no idea how
you managed to get this to work with 2.4.1 :)

> I think the problem is that PyThreadState_Delete (which is used when I
> destroy thread states and interpreters) is not making the same clean up
> that PyThreadState_DeleteCurrent is doing (which is used when we create 
> threads from python code). If a thread id is reused (obviously not 
> between two running threads), and the previous thread used 
> PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old 
> and invalid value for autoTLSkey is still stored, and that is
> triggering the Py_FatalError when a call to PyThreadState_Swap is made.
> If I add this code at the end of PyThreadState_Delete:
>
> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
> 	PyThread_delete_key_value(autoTLSkey);
>
> then everything works fine.

I think I begin to understand the problem... but this is still
fragile: it relies on the fact that you delete a thread state from the
OS level thread that created it, but while a thread belonging to a
different *interpreter state* has the GIL (or at least: the
interpreter state of the thread state being deleted *doesn't* hold the
GIL).  Can you guarantee that?

It seems to me that to be robust, you need a way of saying "delete the
thread local value of autoTLSKey from thread $FOO".  But this is all
very confusing...

> Could you please confirm if this is a bug ?

Yes, I think it's a bug.

Cheers,
mwh

-- 
  <etrepum> Jokes around here tend to get followed by implementations.
                                                -- from Twisted.Quotes

From ncoghlan at gmail.com  Fri Jan 13 11:16:12 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 13 Jan 2006 20:16:12 +1000
Subject: [Python-Dev] PEP and stdlib
In-Reply-To: <43C691EC.3000503@evhr.net>
References: <43C691EC.3000503@evhr.net>
Message-ID: <43C77DEC.70404@gmail.com>

Fabien Schwob wrote:
> Hello,
> 
> I've often read new PEP that are published, and they are often about new 
> additions to the core language.  Why not using them with the standard
> library.

PEPs are used to make changes to the Standard Library - there's a bunch of 
them listed in PEP 0. However, a formal PEP is often not required when there 
is no significant dissent on c.l.p or python-dev that requires a BDFL 
pronouncement - in such cases, the relevant module maintainer (or python-dev 
collectively) may just accept the change and check it in. If there's no 
controversy about a change, the overhead involved in making a PEP just isn't 
worth it.

Making changes to the core language (especially syntax changes, but also 
adding or significantly altering builtins) always requires a BDFL 
pronouncement, is almost always controversial, and hence almost always 
involves a full PEP.

(The only syntax change I know of that Guido has approved without a PEP is to 
fix the minor irritation where keyword arguments can't come after a *argument. 
However, even that should probably be made into a full PEP so that some 
potential extensions to define keyword-only arguments and function default 
values can be debated).

> Guido often say that he don't want to include new module that aren't 
> widely used in the community. It's a good thing, but it lead to the 
> creation of a lot of API incompatible modules. Why not using PEP in 
> order to create standard API like the Java world do with JSRs (Java 
> Specification Requests) ?
> 
> What do you think about that ?

Some PEP's are already about exactly such issues - there are a few listed 
under "Other Informational PEP's" in PEP 0.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fredrik at pythonware.com  Fri Jan 13 11:32:47 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri, 13 Jan 2006 11:32:47 +0100
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp><20060111195900.GB17269@activestate.com>
	<dq6s50$mc4$1@sea.gmane.org>
Message-ID: <dq7vki$rco$1@sea.gmane.org>

Scott David Daniels wrote:

> OK I carried the code I offered earlier in this whole thread (tweaked in
> reaction to some comments) over to comp.lang.python, gathered some
> feedback, and put up a recipe on the cookbook.  After a week or so for
> more comment, I'll be happy to submit a patch to include the broken_test
> decorator function in unittest.
>
> Here is where the recipe is, for those who want to comment further:
>      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288

my main nit is the name: the test isn't broken in itself, and doesn't need
to be fixed; it's just not expected to succeed at this time.

the usual term for this is "expected failure" (sometimes called XFAIL).

for the developer, this means that a failure is not a regression, and is pro-
bably not caused by something that the developer just did.

</F>




From steve at holdenweb.com  Fri Jan 13 14:16:01 2006
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 13 Jan 2006 13:16:01 +0000
Subject: [Python-Dev] TAR problems under Solaris
In-Reply-To: <1136409821.10363.59.camel@geddy.wooz.org>
References: <dphd31$8av$1@sea.gmane.org>
	<1136409821.10363.59.camel@geddy.wooz.org>
Message-ID: <dq896d$rsn$1@sea.gmane.org>

Barry Warsaw wrote:
> On Wed, 2006-01-04 at 22:01 +0100, Reinhold Birkenfeld wrote:
> 
>>Recently, someone on dclpy posted about an error he got
>>when he tried to unpack the Python distribution tarball
>>with Sparc Solaris 9's tar:
>>
>>tar: directory checksum error
>>
>>With GNU tar, it worked correctly.
>>
>>Is this a known issue, or is it irrelevant?
> 
> 
> Dunno, but I'm always having problems w/ Solaris tar, so I just use GNU
> tar on Solaris. ;)
> 
Yes. Back in the 1980's Sun's consulting group made good money selling 
an "extended" version of tar (that coped with longer paths than the 
standard version) to customers who could afford it. Guess they never fed 
the changes back into the trunk ...

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From kbk at shore.net  Fri Jan 13 22:24:11 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Fri, 13 Jan 2006 16:24:11 -0500 (EST)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200601132124.k0DLOBMh006701@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  384 open ( +2) /  3016 closed (+13) /  3400 total (+15)
Bugs    :  909 open ( +6) /  5500 closed (+21) /  6409 total (+27)
RFE     :  208 open ( +5) /   196 closed ( +1) /   404 total ( +6)

New / Reopened Patches
______________________

add support for thread function result storage  (2006-01-02)
       http://python.org/sf/1395552  opened by  tosi-in

Nit: incorrect subsection LaTeX label in cookielib docs  (2006-01-03)
CLOSED http://python.org/sf/1395715  opened by  John J Lee

Further .vcproj cleanups  (2006-01-03)
       http://python.org/sf/1396093  opened by  Adal Chiriliuc

FreeBSD is system scope threads capable  (2006-01-04)
       http://python.org/sf/1396919  opened by  Valts

Fix dict and set docs, re: immutability  (2006-01-05)
       http://python.org/sf/1397711  opened by  Collin Winter

dictnotes.txt  (2006-01-05)
       http://python.org/sf/1397848  opened by  Jim Jewett

File-iteration and read* method protection  (2006-01-05)
       http://python.org/sf/1397960  opened by  Thomas Wouters

ConfigParser to save with order  (2006-01-07)
       http://python.org/sf/1399309  opened by  Facundo Batista

enhance unittest to define tests that *should* fail  (2006-01-08)
       http://python.org/sf/1399935  opened by  Neal Norwitz

unicode formats floats according to locale  (2006-01-09)
CLOSED http://python.org/sf/1400181  opened by  Neal Norwitz

Need warning that `dl` module can cause segfaults/crashes  (2006-01-10)
       http://python.org/sf/1402224  opened by  Tim Delaney

Fix dictionary subclass semantics when used as global dicts.  (2006-01-10)
       http://python.org/sf/1402289  opened by  crutcher

Suggested patch for #1403410  (2006-01-13)
CLOSED http://python.org/sf/1404357  opened by  Peter van Kampen

Patch for #1394565  (2006-01-13)
CLOSED http://python.org/sf/1404374  opened by  Peter van Kampen

Patches Closed
______________

dict.merge  (2005-12-27)
       http://python.org/sf/1391204  closed by  gvanrossum

cookielib LWPCookieJar and MozillaCookieJar exceptions  (2005-02-06)
       http://python.org/sf/1117398  closed by  nnorwitz

Nit: incorrect subsection LaTeX label in cookielib docs  (2006-01-03)
       http://python.org/sf/1395715  closed by  birkenfeld

mingw compile  (2004-10-25)
       http://python.org/sf/1053879  closed by  loewis

skip winsound for Windows/IA64 build  (2005-03-09)
       http://python.org/sf/1160169  closed by  loewis

PCbuild vcproj project files need a cleanup  (2005-09-29)
       http://python.org/sf/1307806  closed by  loewis

look in libbsd.a also for openpty and forkpty  (2004-01-22)
       http://python.org/sf/881820  closed by  loewis

UTF-8-Sig codec  (2005-04-05)
       http://python.org/sf/1177307  closed by  loewis

fix for distutils "upload" command  (2005-09-23)
       http://python.org/sf/1299675  closed by  loewis

unicode formats floats according to locale  (2006-01-09)
       http://python.org/sf/1400181  closed by  nnorwitz

Python crashes in pyexpat.c if malformed XML is parsed  (2005-03-31)
       http://python.org/sf/1173998  closed by  nnorwitz

Suggested patch for #1403410  (2006-01-13)
       http://python.org/sf/1404357  closed by  birkenfeld

Patch for #1394565  (2006-01-13)
       http://python.org/sf/1404374  closed by  birkenfeld

New / Reopened Bugs
___________________

errata  (2006-01-01)
CLOSED http://python.org/sf/1394868  opened by  Chad Whitacre

os.remove should behave like rm, not unlink  (2006-01-02)
CLOSED http://python.org/sf/1395442  opened by  Chad Whitacre

Please document pyc format guarantees  (2006-01-02)
CLOSED http://python.org/sf/1395511  opened by  Joe Wreschnig

module os, very small doc inconsistency  (2006-01-02)
CLOSED http://python.org/sf/1395597  opened by  tiissa

Can't build Python on POSIX w/o $HOME  (2004-05-24)
       http://python.org/sf/959576  reopened by  birkenfeld

make fails trying to run svnversion  (2006-01-03)
CLOSED http://python.org/sf/1395926  opened by  M.-A. Lemburg

TimedRotatingFileHandler at midnight rolls over at 01:00  (2006-01-03)
       http://python.org/sf/1396030  opened by  Andrew Waters

TimedRotatingFileHandler does not recover from open error  (2006-01-03)
       http://python.org/sf/1396040  opened by  Andrew Waters

KeyboardInterrupt prevents return to Windows console  (2006-01-03)
       http://python.org/sf/1396258  opened by  Vernon Cole

file.tell() returns illegal value on Windows  (2006-01-03)
       http://python.org/sf/1396471  opened by  Tom Goddard

urlparse is confused by /  (2006-01-04)
       http://python.org/sf/1396543  opened by  John Hansen

TimedRotatingFileHandler midnight rollover time increases  (2006-01-04)
       http://python.org/sf/1396622  opened by  Andrew Waters

bsddb.__init__ causes error  (2006-01-04)
       http://python.org/sf/1396678  opened by  Fabian_M

%ehrntDRT support for time.strptime   (2006-01-04)
       http://python.org/sf/1396946  opened by  Johan Dahlin

no handler base classes in xml.sax  (2006-01-04)
CLOSED http://python.org/sf/1397205  opened by  Alan G

timeit execution enviroment  (2006-01-05)
       http://python.org/sf/1397474  opened by  rurpy

libpython2.4.so get not installed   (2006-01-05)
       http://python.org/sf/1397850  opened by  hajo

Example in section 5.3 "Pure Embedding" doesn't work.  (2006-01-06)
       http://python.org/sf/1398781  opened by  Bill Studenmund

i get a memory leak after using split() function on windows  (2006-01-07)
CLOSED http://python.org/sf/1399099  opened by  Leo Jay

segfault in curses.panel with userptr()  (2006-01-09)
CLOSED http://python.org/sf/1400115  opened by  Tom Quetchenbach

'is' does not work on methods  (2006-01-09)
CLOSED http://python.org/sf/1400227  opened by  Collin Winter

'is' does not work for methods  (2006-01-09)
CLOSED http://python.org/sf/1400235  opened by  Collin Winter

Extended version of _curses over{lay,write} does not work.  (2006-01-09)
CLOSED http://python.org/sf/1400822  opened by  J. Sipprell

Win32COM policy.py unicode exceptions  (2006-01-10)
CLOSED http://python.org/sf/1401642  opened by  rbell01824

null source chars handled oddly  (2005-01-20)
CLOSED http://python.org/sf/1105770  reopened by  birkenfeld

segfault when using mmap(-1,...) [+PATCH]  (2006-01-11)
       http://python.org/sf/1402308  opened by  Ralf Schmitt

cannot import extension module with Purify  (2006-01-11)
       http://python.org/sf/1403068  opened by  Amaury Forgeot d'Arc

2.3.5 source RPM install fails w/o tk-devel  (2006-01-11)
       http://python.org/sf/1403221  opened by  Nathan Zook

2.3.5 RPM provides incompatibility  (2006-01-11)
       http://python.org/sf/1403225  opened by  Nathan Zook

in email can't get attachments' filenames using get_filename  (2006-01-11)
       http://python.org/sf/1403349  opened by  Michal P.

TypeError unsubscriptable object caused by warnings.py  (2006-01-12)
CLOSED http://python.org/sf/1403410  opened by  FreyP

os.getlogin() goes **poof**  (2006-01-12)
CLOSED http://python.org/sf/1404213  opened by  Jim Wilson

subprocess.Popen inside thread locks the thread in some case  (2006-01-13)
       http://python.org/sf/1404925  opened by  Toon Verstraelen

Bugs Closed
___________

'Plus' filemode exposes uninitialized memory on win32  (2005-12-31)
       http://python.org/sf/1394612  closed by  tim_one

doc errata  (2006-01-01)
       http://python.org/sf/1394868  closed by  birkenfeld

os.remove should behave like rm, not unlink  (2006-01-02)
       http://python.org/sf/1395442  deleted by  whit537

Please document pyc format guarantees  (2006-01-02)
       http://python.org/sf/1395511  closed by  piman

module os, very small doc inconsistency  (2006-01-02)
       http://python.org/sf/1395597  closed by  birkenfeld

make fails trying to run svnversion  (2006-01-03)
       http://python.org/sf/1395926  closed by  bwarsaw

no handler base classes in xml.sax  (2006-01-04)
       http://python.org/sf/1397205  closed by  birkenfeld

SimpleHTTPServer doesn't understand query arguments  (2005-12-31)
       http://python.org/sf/1394565  closed by  birkenfeld

Float marshaling problems with test_colorsys  (2005-11-30)
       http://python.org/sf/1370322  closed by  birkenfeld

i get a memory leak after using split() function on windows  (2006-01-07)
       http://python.org/sf/1399099  closed by  perky

2.3.3 make fails build posix_openpty'  (2004-01-21)
       http://python.org/sf/880996  closed by  loewis

SegFault in interactive mode when cursor is over a non-ASCII  (2004-11-15)
       http://python.org/sf/1066545  closed by  nnorwitz

segfault in curses.panel with userptr()  (2006-01-08)
       http://python.org/sf/1400115  closed by  nnorwitz

2.4.1 build fails on OpenBSD 3.7  (2005-07-25)
       http://python.org/sf/1244610  closed by  nnorwitz

build fails on BSD 3.8  (2005-12-29)
       http://python.org/sf/1392915  closed by  nnorwitz

'is' does not work on methods  (2006-01-09)
       http://python.org/sf/1400227  closed by  mwh

'is' does not work for methods  (2006-01-09)
       http://python.org/sf/1400235  deleted by  collinwinter

Extended version of _curses over{lay,write} does not work.  (2006-01-09)
       http://python.org/sf/1400822  closed by  nnorwitz

Win32COM policy.py unicode exceptions  (2006-01-10)
       http://python.org/sf/1401642  closed by  birkenfeld

simple attribute access causing segfault in certain cases  (2004-07-01)
       http://python.org/sf/983311  closed by  birkenfeld

Another dealloc stack killer  (2002-08-26)
       http://python.org/sf/600007  closed by  birkenfeld

PyOS_snprintf segfaults on missing native snprintf  (2004-01-05)
       http://python.org/sf/871026  closed by  tim_one

null source chars handled oddly  (2005-01-20)
       http://python.org/sf/1105770  closed by  birkenfeld

TypeError unsubscriptable object caused by warnings.py  (2006-01-12)
       http://python.org/sf/1403410  closed by  birkenfeld

os.getlogin() goes **poof**  (2006-01-12)
       http://python.org/sf/1404213  closed by  birkenfeld

New / Reopened RFE
__________________

subprocess: wait for a period of time  (2006-01-04)
       http://python.org/sf/1396825  opened by  Ragnar Ouchterlony

python.org - add RFE tracker  (2006-01-05)
       http://python.org/sf/1397806  opened by  Jim Jewett

Add str.partition  (2006-01-08)
       http://python.org/sf/1399936  opened by  Georg Brandl

invoking an external Java program from Python script  (2006-01-09)
CLOSED http://python.org/sf/1400252  opened by  Laure

friendly quit object  (2006-01-13)
       http://python.org/sf/1404859  opened by  Jim Jewett

RFE Closed
__________

invoking an external Java program from Python script  (2006-01-09)
       http://python.org/sf/1400252  closed by  birkenfeld


From martin at v.loewis.de  Sat Jan 14 00:29:51 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 14 Jan 2006 00:29:51 +0100
Subject: [Python-Dev] Py_ssize_t output parameters
 (Was:	[Python-checkins] r41971 ...)
In-Reply-To: <43C77A15.5050803@egenix.com>
References: <20060108054820.15F321E4002@bag.python.org>	<43C25CE1.4070106@egenix.com>	<43C2DD44.2040908@v.loewis.de>	<43C37C92.3060302@egenix.com>	<43C4083F.3070001@v.loewis.de>	<43C421A3.2030806@egenix.com>	<43C425DE.2050001@v.loewis.de>
	<43C63458.10804@egenix.com> <43C6D9D1.7040101@v.loewis.de>
	<43C77A15.5050803@egenix.com>
Message-ID: <43C837EF.30903@v.loewis.de>

M.-A. Lemburg wrote:
> Right, but I don't see the point of each and every
> extension having to go through these hoops when you
> could add support for these checks (including error
> reporting) to the few APIs in question, in particular
> the PyArg_ParseTuple() API.

I don't want to rename the existing APIs, so porters
to ssize_t have an easier life (just fix the compiler
warnings).

As for PyArg_ParseTuple: this already works and requires
no changes either way.

> I don't expect this to happen any time soon for the ten or
> twenty other 3rd party extensions we regularly use and this
> would prevent an upgrade to Python 2.5.

It shouldn't. It could sensibly prevent an upgrade to 64-bit
machines, but there would be no reason not to upgrade on
32-bit machines.

> In particular, functions that currently take int* output
> parameters should continue to do so. New functions should be
> revised to enable Py_ssize_t* output arguments and preseve
> backwards compatibility.
> 
> (This also includes the strategy to be used, so you can
> remove the note on strategy)

Will do.

> Please also add a comment on the fact that extensions
> which haven't been recompiled for the Python 2.5 API
> will not get imported (the API_LEVEL check should consistently
> fail for these instead of just issuing a warning).

I will add it as an open issue. I disagree that this should
actually be done.

>>Would you like to write a specification for that?
> 
> 
> Sure, if there's hope to get this into the code.

I don't know. So far, you are the only one wanting this
kind of change. If the change actually existed, more
people might be willing to voice an opinion - whether
that will be in favour of more hackishness and backwards
compatibility, or against, I cannot guess.

I would personally want to restrict hackishness to places
that the compiler doesn't warn about.

>>To simplify the porting.
> 
> 
> Sorry, I lost you there:
> 
> I'm saying that two sets of APIs (one for int*, one
> for Py_ssize_t*) will make porting easier since
> updated code will only have to rename the APIs
> used. Furthermore, it will allow to write code
> that easily works in Python 2.1-2.4 as well as
> Python 2.5.

Yes, but users of the new API (who would conceptually
do the right thing) would need more effort, just to
protect users who do nothing (which conceptually do
the wrong thing). By just editing the APIs (instead
of also renaming them), I simplify the porting.

> I'm talking about the few APIs that use output parameters.
> Those do support error reporting.

Hmm. I think I could personally accept a change that uses
macro tricks to do all that, i.e. keep the output parameters
at int* if some macro (Py_SIZE_T_CLEAN) is not defined,
and use Py_ssize_t* if it is defined. People would then get
the right warnings still, but no changes would be necessary
to "old" code.

Regards,
Martin

From Scott.Daniels at Acm.Org  Sat Jan 14 05:22:49 2006
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Fri, 13 Jan 2006 20:22:49 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dq7vki$rco$1@sea.gmane.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp><20060111195900.GB17269@activestate.com>	<dq6s50$mc4$1@sea.gmane.org>
	<dq7vki$rco$1@sea.gmane.org>
Message-ID: <dq9ua7$vhh$1@sea.gmane.org>

Fredrik Lundh wrote:
> Scott David Daniels wrote:
>>      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288
> 
> my main nit is the name: the test isn't broken in itself, and doesn't need
> to be fixed; it's just not expected to succeed at this time.
> 
> the usual term for this is "expected failure" (sometimes called XFAIL).

Would "expect_fail", "expect_failure", "expected_fail", or 
"expected_failure",
work for you?

If so, could you rank them?  I don't get anything from "xfail", and I'm
not sure others will either.

--Scott David Daniels
Scott.Daniels at Acm.Org


From ncoghlan at gmail.com  Sat Jan 14 05:40:10 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 14 Jan 2006 14:40:10 +1000
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dq9ua7$vhh$1@sea.gmane.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp><20060111195900.GB17269@activestate.com>	<dq6s50$mc4$1@sea.gmane.org>	<dq7vki$rco$1@sea.gmane.org>
	<dq9ua7$vhh$1@sea.gmane.org>
Message-ID: <43C880AA.3060608@gmail.com>

Scott David Daniels wrote:
> Fredrik Lundh wrote:
>> Scott David Daniels wrote:
>>>      http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288
>> my main nit is the name: the test isn't broken in itself, and doesn't need
>> to be fixed; it's just not expected to succeed at this time.
>>
>> the usual term for this is "expected failure" (sometimes called XFAIL).
> 
> Would "expect_fail", "expect_failure", "expected_fail", or 
> "expected_failure",
> work for you?
> 
> If so, could you rank them?  I don't get anything from "xfail", and I'm
> not sure others will either.

I'd be happy with either "expect_fail" (as the shortest) or "expected_failure" 
(as the actual English term), with a slight preference for the former as being 
just as clear, and requiring slightly less typing.

There's also the fact that unittest has a large number of test case methods 
that start with "failIf" or "failUnless", so the "expect_fail" term aligns 
nicely with those.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fdrake at acm.org  Sat Jan 14 05:51:42 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri, 13 Jan 2006 23:51:42 -0500
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <43C880AA.3060608@gmail.com>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dq9ua7$vhh$1@sea.gmane.org> <43C880AA.3060608@gmail.com>
Message-ID: <200601132351.42410.fdrake@acm.org>

Scott David Daniels wrote:
 > Would "expect_fail", "expect_failure", "expected_fail", or
 > "expected_failure", work for you?

None of these use the same naming convention as the other unittest object 
attributes.  Perhaps something like failureExpected?

I'd definately prefer something that reads cleanly; mirroring the exact form 
of the word "fail" doesn't make sense; making it readable does.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From arigo at tunes.org  Sat Jan 14 11:14:54 2006
From: arigo at tunes.org (Armin Rigo)
Date: Sat, 14 Jan 2006 11:14:54 +0100
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
References: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>
	<-8397309676539309595@unknownmsgid>
	<bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
Message-ID: <20060114101454.GA29087@code1.codespeak.net>

Hi Brett,

If by any chance PyPy continues to be funded beyond 2006, we would
definitely welcome you around :-)  (If our funding model doesn't change,
it might be difficult for us to give you money oversea, though... just
asking, just in case, would you consider moving to a European
university?)

PyPy contains several open language research areas that you mentioned:
network-distributed language support, concurrent programming...  and we
even already have a Python -> JavaScript compiler :-)  Making it useful
is an open challange, though.


A bientot,

Armin

From Scott.Daniels at Acm.Org  Sat Jan 14 15:17:55 2006
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Sat, 14 Jan 2006 06:17:55 -0800
Subject: [Python-Dev] Checking in a broken test was: Re:
 [Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <200601132351.42410.fdrake@acm.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dq9ua7$vhh$1@sea.gmane.org>
	<43C880AA.3060608@gmail.com> <200601132351.42410.fdrake@acm.org>
Message-ID: <dqb162$s8q$1@sea.gmane.org>

Fred L. Drake, Jr. wrote:
> Scott David Daniels wrote:
>  > Would "expect_fail", "expect_failure", "expected_fail", or
>  > "expected_failure", work for you?
> 
> None of these use the same naming convention as the other unittest object 
> attributes.  Perhaps something like failureExpected?
> 
> I'd definately prefer something that reads cleanly; mirroring the exact form 
> of the word "fail" doesn't make sense; making it readable does.

Hmmm.... I'd like to avoid anything looking like "failIf" or whatever
(I'm afraid it will be attempted as a method), but the point about
matching styles does make sense.  I see my idea of XXX has inspired a
huge groundswell of "no comment" yet.  How about:

     @expectFailure('some reason for the failure')
     def test_whatever(self): ...

Nick Coghlan (on the recipe) wrote:
 > While embedding the 'or' in the except clause header is cute, its also
 > a little subtle and obscure, and causes the check to be executed every
 > time the test is run, rather than when the test is defined.

I wasn't striving for cute or clever there.  I did want the pick up of
TestCase.failureException to occur at test time, however.  My reasoning
is that failureException became a class variable for a reason, and test
running frameworks other than unittest.main might well be stomping their
own exception in there.  Certainly the "failIf" methods all raise
whatever exception is stored as a class variable of TestCase.


Do we want to include the equivalent of find_broken_tests(module)
in unittest or is leaving it in the recipe for report writers better?
I suppose it wants a better name as well.  findExpectedFailures(module)?
I'm off to see family, I'll check back end of next week.

--Scott David Daniels
Scott.Daniels at Acm.Org


From g.brandl-nospam at gmx.net  Sun Jan 15 00:19:30 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sun, 15 Jan 2006 00:19:30 +0100
Subject: [Python-Dev] Python icon
Message-ID: <dqc0u2$ul0$1@sea.gmane.org>

Hi,

does Python have an official icon? Not py.ico from PC/, that's a bit
ugly and does not scale. Has no designerhead ever done such a thing?

Georg


From jason.orendorff at gmail.com  Sun Jan 15 01:17:19 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Sat, 14 Jan 2006 19:17:19 -0500
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>
References: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>
Message-ID: <bb8868b90601141617y60df5b00ma5100e42ab546f98@mail.gmail.com>

Brett,

You could create a downloadable corpus of Python source code, and
maybe a web site through which people can easily browse/search it,
contribute to it, and maintain it.  The point would be to support
language designers, tool developers, and researchers.  Several
python-dev folks have their own corpuses; I think other people would
be happy to use a free one if it were out there.

Of course there's no need to limit it to Python...

Creating a really *good* corpus is maybe not super-easy; I imagine
there are myriad linguistics papers explaining the nuances.  Hey,
cross-discipline research--cool points!

Once this exists, there's no shortage of research questions you can
quickly and easily answer with it.  What percentage of Python programs
use functional programming techniques?  How often are list
comprehensions used?  What do people use generators for?

And if you do something web-based, you can certainly work XML in there
somewhere.  :)

-j

From python-dev at zesty.ca  Sun Jan 15 01:36:31 2006
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Sat, 14 Jan 2006 18:36:31 -0600 (CST)
Subject: [Python-Dev] Python icon
In-Reply-To: <dqc0u2$ul0$1@sea.gmane.org>
References: <dqc0u2$ul0$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>

On Sun, 15 Jan 2006, Georg Brandl wrote:
> does Python have an official icon? Not py.ico from PC/, that's a bit
> ugly and does not scale. Has no designerhead ever done such a thing?

There have been a couple of proposed logos -- i found some images
at http://www.pythonology.com/logos -- but i don't know what their
official status is.


-- ?!ng

From tismer at stackless.com  Sun Jan 15 02:37:47 2006
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 15 Jan 2006 02:37:47 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
Message-ID: <43C9A76B.8070801@stackless.com>

Hi Python developers,

today I got a complaint from the python.de IRC channel
about os.path.getmtime and time zone.

How to produce the weird behavior:

1. create a file
2. get it's os.path.getmtime()
3. change your time zone
4. get os.path.getmtime again

compare - the time stamps are different.
Change the time zone back, and they are identical, again.

I was not ableto produce an identity, neither by time.gmtime
nor by time.localtime, so I'm a bit confused.

I checked the sources, and this is probably not a Python
problem. It uses the suggested win32 function properly.
But the win32 documentation seems to have no hints about this.

I assumend the value would be in UTC, but it is obviously not.

Is there a way to circumvent this problem, or am I missing something?
If this is not the expected behavior, then it might make sense
to find a patch.

thanks -- chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From brett at python.org  Sun Jan 15 02:51:25 2006
From: brett at python.org (Brett Cannon)
Date: Sat, 14 Jan 2006 17:51:25 -0800
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <20060114101454.GA29087@code1.codespeak.net>
References: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>
	<-8397309676539309595@unknownmsgid>
	<bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
	<20060114101454.GA29087@code1.codespeak.net>
Message-ID: <bbaeab100601141751g5b345918q71e7787c5233f162@mail.gmail.com>

On 1/14/06, Armin Rigo <arigo at tunes.org> wrote:
> Hi Brett,
>
> If by any chance PyPy continues to be funded beyond 2006, we would
> definitely welcome you around :-)  (If our funding model doesn't change,
> it might be difficult for us to give you money oversea, though... just
> asking, just in case, would you consider moving to a European
> university?)
>

That would be cool!  I definitely would not mind working on PyPy. 
Unfortunately I would not consider changing universities; I really
like it here.

> PyPy contains several open language research areas that you mentioned:
> network-distributed language support, concurrent programming...  and we
> even already have a Python -> JavaScript compiler :-)  Making it useful
> is an open challange, though.
>

Well, if PyPy picked up the grants tab I would work on whatever you
guys wanted that could be turned into a dissertation topic.  =)

-Brett

From pje at telecommunity.com  Sun Jan 15 03:41:46 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat, 14 Jan 2006 21:41:46 -0500
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43C9A76B.8070801@stackless.com>
Message-ID: <5.1.1.6.0.20060114214008.022459d8@mail.telecommunity.com>

At 02:37 AM 1/15/2006 +0100, Christian Tismer wrote:
>I assumend the value would be in UTC, but it is obviously not.
>
>Is there a way to circumvent this problem, or am I missing something?
>If this is not the expected behavior, then it might make sense
>to find a patch.

Windows doesn't store UTC timestamps, at least not on older FAT filesystems 
and maybe not even on NTFS.  Changing Python won't help.  :)


From tim.peters at gmail.com  Sun Jan 15 04:31:36 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sat, 14 Jan 2006 22:31:36 -0500
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43C9A76B.8070801@stackless.com>
References: <43C9A76B.8070801@stackless.com>
Message-ID: <1f7befae0601141931u5dbd38e5nd2e24338f50776f1@mail.gmail.com>

[Christian Tismer]
> Hi Python developers,
>
> today I got a complaint from the python.de IRC channel
> about os.path.getmtime and time zone.
>
> How to produce the weird behavior:
>
> 1. create a file
> 2. get it's os.path.getmtime()
> 3. change your time zone
> 4. get os.path.getmtime again
>
> compare - the time stamps are different.
> Change the time zone back, and they are identical, again.
>
> I was not ableto produce an identity, neither by time.gmtime
> nor by time.localtime, so I'm a bit confused.
>
> I checked the sources, and this is probably not a Python
> problem. It uses the suggested win32 function properly.
> But the win32 documentation seems to have no hints about this.
>
> I assumend the value would be in UTC, but it is obviously not.

See:

    http://www.codeproject.com/datetime/dstbugs.asp

and just be grateful you didn't bump into the additional mysteries MS
added around "daylight time" switches.

> Is there a way to circumvent this problem, or am I missing something?
> If this is not the expected behavior, then it might make sense
> to find a patch.

While I love the article linked to above, I've never had the will to
force myself to read it all the way to the end ;-)

From Martin.vonLoewis at hpi.uni-potsdam.de  Sun Jan 15 08:43:02 2006
From: Martin.vonLoewis at hpi.uni-potsdam.de (=?iso-8859-1?Q?Martin_v._L=F6wis?=)
Date: Sun, 15 Jan 2006 08:43:02 +0100
Subject: [Python-Dev] Installing PEPs
Message-ID: <43C9FD06.6@hpi.uni-potsdam.de>

I finally arranged for an automatic installation of PEPs:
they will now get published in the subversion post-commit.

Regards,
Martin


From martin at v.loewis.de  Sun Jan 15 09:21:47 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jan 2006 09:21:47 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <5.1.1.6.0.20060114214008.022459d8@mail.telecommunity.com>
References: <5.1.1.6.0.20060114214008.022459d8@mail.telecommunity.com>
Message-ID: <43CA061B.3080604@v.loewis.de>

Phillip J. Eby wrote:
> Windows doesn't store UTC timestamps, at least not on older FAT filesystems 
> and maybe not even on NTFS.  Changing Python won't help.  :)

Windows definitely stores UTC timestamps on NTFS, in units of 100ns
since Jan 1, 1601.

Regards,
Martin

From martin at v.loewis.de  Sun Jan 15 09:38:25 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jan 2006 09:38:25 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43C9A76B.8070801@stackless.com>
References: <43C9A76B.8070801@stackless.com>
Message-ID: <43CA0A01.4020406@v.loewis.de>

Christian Tismer wrote:
> Is there a way to circumvent this problem, or am I missing something?
> If this is not the expected behavior, then it might make sense
> to find a patch.

I have meant to work on a patch for several years now. I would like to
drop usage of msvcrt's stat(3), and instead implement os.stat in terms
of Windows API directly. That would also have the advantage that
subsecond time-stamps can be exposed.

There are several issues involved in implementing such a patch, though.
One is that you need to do it twice: once for Win9x, and once for
NT+, because you have to use Unicode file names on one system, and
ANSI file names on the other.

Regards,
Martin

From collinw at gmail.com  Sun Jan 15 12:00:53 2006
From: collinw at gmail.com (Collin Winter)
Date: Sun, 15 Jan 2006 12:00:53 +0100
Subject: [Python-Dev] Checking in a broken test was: Re:
	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <dq7vki$rco$1@sea.gmane.org>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>
	<dpr983$95r$1@sea.gmane.org>
	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>
	<20060111195900.GB17269@activestate.com> <dq6s50$mc4$1@sea.gmane.org>
	<dq7vki$rco$1@sea.gmane.org>
Message-ID: <43aa6ff70601150300sacfa02fvc6709964061c19af@mail.gmail.com>

On 1/13/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> my main nit is the name: the test isn't broken in itself, and doesn't need
> to be fixed; it's just not expected to succeed at this time.
>
> the usual term for this is "expected failure" (sometimes called XFAIL).
>
> for the developer, this means that a failure is not a regression, and is pro-
> bably not caused by something that the developer just did.

When I've implemented this kind of thing in the past, I've generally
called the decorator/marker/whatever "TODO" (or some variation of
caps/lowercase).

</bikeshed>

Thanks,
Collin Winter

From skip at pobox.com  Sun Jan 15 14:41:06 2006
From: skip at pobox.com (skip at pobox.com)
Date: Sun, 15 Jan 2006 07:41:06 -0600
Subject: [Python-Dev] Python icon
In-Reply-To: <Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
References: <dqc0u2$ul0$1@sea.gmane.org>
	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
Message-ID: <17354.20722.560136.55434@montanaro.dyndns.org>


    >> does Python have an official icon?

    Ping> i found some images at http://www.pythonology.com/logos...

It appears the yin/yang Python's on that page are being used in the new site
(beta.python.org).  I don't know if that makes it official or not though.

Skip

From tismer at stackless.com  Sun Jan 15 15:13:21 2006
From: tismer at stackless.com (Christian Tismer)
Date: Sun, 15 Jan 2006 15:13:21 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43CA0A01.4020406@v.loewis.de>
References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de>
Message-ID: <43CA5881.3020305@stackless.com>

Hi Martin,

>> Is there a way to circumvent this problem, or am I missing something?
>> If this is not the expected behavior, then it might make sense
>> to find a patch.
> 
> I have meant to work on a patch for several years now. I would like to
> drop usage of msvcrt's stat(3), and instead implement os.stat in terms
> of Windows API directly. That would also have the advantage that
> subsecond time-stamps can be exposed.

I see! Still trying to understand the story. I'm working through
the article Tim pointed us at.
http://www.codeproject.com/datetime/dstbugs.asp

Does it mean that msvcrt does extra magic to modify the existing
correct UTC entries? And would usage of the Windows API heal this
immediately, or are extra steps involved?
As I understand the article, things are different when a file is
stored in a FAT or NTFS drive.

Do you think the provided solution is worthwhile to be adapted
for Python?
http://www.codeproject.com/datetime/DstBugs/DstBugs.zip

> There are several issues involved in implementing such a patch, though.
> One is that you need to do it twice: once for Win9x, and once for
> NT+, because you have to use Unicode file names on one system, and
> ANSI file names on the other.

Correcting it just for NT/XP would make the majority of people
happy, IMHO.

cheers - chris
-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/

From gabriel.becedillas at corest.com  Fri Jan 13 15:37:19 2006
From: gabriel.becedillas at corest.com (Gabriel Becedillas)
Date: Fri, 13 Jan 2006 11:37:19 -0300
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <2mwth4qw3q.fsf@starship.python.net>
References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net>
Message-ID: <43C7BB1F.3000904@corest.com>

Michael Hudson wrote:
> Gabriel Becedillas <gabriel.becedillas at corest.com> writes:
> 
> 
>>Hi,
>>At the company I work for, we've embedded Python in C++ application we 
>>develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started hitting 
>>Py_FatalError("Invalid thread state for this thread") when using debug 
>>builds.
>>We use both multiple interpreters and thread states.
> 
> 
> I know you've said this to me in email already, but I have no idea how
> you managed to get this to work with 2.4.1 :)
> 
> 
>>I think the problem is that PyThreadState_Delete (which is used when I
>>destroy thread states and interpreters) is not making the same clean up
>>that PyThreadState_DeleteCurrent is doing (which is used when we create 
>>threads from python code). If a thread id is reused (obviously not 
>>between two running threads), and the previous thread used 
>>PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an old 
>>and invalid value for autoTLSkey is still stored, and that is
>>triggering the Py_FatalError when a call to PyThreadState_Swap is made.
>>If I add this code at the end of PyThreadState_Delete:
>>
>>if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
>>	PyThread_delete_key_value(autoTLSkey);
>>
>>then everything works fine.
> 
> 
> I think I begin to understand the problem... but this is still
> fragile: it relies on the fact that you delete a thread state from the
> OS level thread that created it, but while a thread belonging to a
> different *interpreter state* has the GIL (or at least: the
> interpreter state of the thread state being deleted *doesn't* hold the
> GIL).  Can you guarantee that?

Mmm... it doesn't have to do with a race condition or something. Let me 
try to show you with an example (keep in mind that this relies on the 
fact that at some moment the operating system gives you a thread with 
the same id of another thread that allready finished executing):

// Initialize python.
Py_Initialize();
PyEval_InitThreads();
PyThreadState* p_main =  PyThreadState_Swap(NULL);
PyEval_ReleaseLock();

/*
Asume this block is executed in a separate thread, and that has been 
asigned thread id 10.
It doesn't matter what the main thread (the thread that initialized 
Python) is doing. Lets assume its waiting for this one to finish.
*/
{
	PyEval_AcquireLock();
	PyThreadState_Swap(p_main);
	PyThreadState* p_child = PyThreadState_New(p_main->interp);

	PyThreadState_Swap(p_child);
	PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'");

	PyThreadState_Clear(p_child);
	PyThreadState_Swap(NULL);
	PyThreadState_Delete(p_child);
	PyEval_ReleaseLock();
	// The os level thread exits here.
}
/*
When this thread state gets deleted (using PyThreadState_Delete), the 
autoTLSkey stored for thread id number 10 is not removed, because 
PyThreadState_Delete doesn't have the necesary cleanup code (check 
pystate.c):

if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
	PyThread_delete_key_value(autoTLSkey);

PyThreadState_DeleteCurrent behaves correctly because it does have this 
code.
I think that this code should be added to tstate_delete_common (I've 
done this and everything worked just fine).
If a new thread executes the same code, and that thread is assigned the 
same thread id, you get the Py_FatalError I'm talking about.
A workaround for this would be to use PyThreadState_DeleteCurrent 
instead of PyThreadState_Delete.

If you use the following code instead of the one above, the you have no 
way out to the Py_FatalError:
*/
{
	PyEval_AcquireLock();
	PyThreadState* ret = Py_NewInterpreter();

	PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'");

	Py_EndInterpreter(ret);
	PyThreadState_Swap(NULL);
	PyEval_ReleaseLock();
}
/*
When this interpreter gets deleted (using Py_EndInterpreter), its thread 
state gets deleted using PyThreadState_Delete, and you are back to the 
beginning of the problem.
*/

I hope this helps to clarify the problem.
Thanks a lot and have a nice day.

> It seems to me that to be robust, you need a way of saying "delete the
> thread local value of autoTLSKey from thread $FOO".  But this is all
> very confusing...
> 
> 
>>Could you please confirm if this is a bug ?
> 
> 
> Yes, I think it's a bug.
> 
> Cheers,
> mwh
> 


-- 


Gabriel Becedillas
Developer
CORE SECURITY TECHNOLOGIES

Florida 141 - 2? cuerpo - 7? piso
C1005AAC Buenos Aires - Argentina
Tel/Fax: (54 11) 5032-CORE (2673)
http://www.corest.com

From martin at v.loewis.de  Sun Jan 15 20:23:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jan 2006 20:23:39 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43CA5881.3020305@stackless.com>
References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de>
	<43CA5881.3020305@stackless.com>
Message-ID: <43CAA13B.2050902@v.loewis.de>

Christian Tismer wrote:
> Does it mean that msvcrt does extra magic to modify the existing
> correct UTC entries? 

Mostly, yes. For FAT, the system does also some conversion.
Those conversions I don't fully understand,

http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sysinfo/base/file_times.asp

suggests that there are "cached UTC times" on FAT somewhere.

> And would usage of the Windows API heal this
> immediately, or are extra steps involved?

If we use GetFileTime, apparently yes. FindFirstFile apparently also
suffers from daylight conversion bugs when reading time stamps from FAT.

> As I understand the article, things are different when a file is
> stored in a FAT or NTFS drive.

Correct.

> Do you think the provided solution is worthwhile to be adapted
> for Python?
> http://www.codeproject.com/datetime/DstBugs/DstBugs.zip

No. If anything is done about this, it should be to drop msvcrt.

>> There are several issues involved in implementing such a patch, though.
>> One is that you need to do it twice: once for Win9x, and once for
>> NT+, because you have to use Unicode file names on one system, and
>> ANSI file names on the other.
> 
> 
> Correcting it just for NT/XP would make the majority of people
> happy, IMHO.

Right - the question would be whether completely breaking W9x support
in the process would be acceptable. We use the very same binaries
for W9x and NT.

Regards,
Martin

From martin at v.loewis.de  Sun Jan 15 20:45:32 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jan 2006 20:45:32 +0100
Subject: [Python-Dev] Checking in a broken test was:
 Re:	[Python-checkins]r41940 - python/trunk/Lib/test/test_compiler.py
In-Reply-To: <43aa6ff70601150300sacfa02fvc6709964061c19af@mail.gmail.com>
References: <ee2a432c0601071245t31160b4as380766c3be1093ba@mail.gmail.com>	<dpr983$95r$1@sea.gmane.org>	<87zmm6d7jp.fsf@tleepslib.sk.tsukuba.ac.jp>	<20060111195900.GB17269@activestate.com>
	<dq6s50$mc4$1@sea.gmane.org>	<dq7vki$rco$1@sea.gmane.org>
	<43aa6ff70601150300sacfa02fvc6709964061c19af@mail.gmail.com>
Message-ID: <43CAA65C.8060005@v.loewis.de>

Collin Winter wrote:
> When I've implemented this kind of thing in the past, I've generally
> called the decorator/marker/whatever "TODO" (or some variation of
> caps/lowercase).

I usually call things TODO if they need to be done. The test case is
not "TODO", since it is already done. "TODO" would be the place in
the code that needs to change; for an expected failure, you might not
even know where that code is.

Regards,
Martin

From 2005a at usenet.alexanderweb.de  Sun Jan 15 22:18:02 2006
From: 2005a at usenet.alexanderweb.de (Alexander Schremmer)
Date: Sun, 15 Jan 2006 22:18:02 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
References: <43C9A76B.8070801@stackless.com> <43CA0A01.4020406@v.loewis.de>
	<43CA5881.3020305@stackless.com> <43CAA13B.2050902@v.loewis.de>
Message-ID: <gi49ixye313w.dlg@usenet.alexanderweb.de>

On Sun, 15 Jan 2006 20:23:39 +0100, "Martin v. L?wis" wrote:

>>> There are several issues involved in implementing such a patch, though.
>>> One is that you need to do it twice: once for Win9x, and once for
>>> NT+, because you have to use Unicode file names on one system, and
>>> ANSI file names on the other.

> Right - the question would be whether completely breaking W9x support
> in the process would be acceptable. We use the very same binaries
> for W9x and NT.

How about accessing/calling the wide char versions just if the OS is NT,
i.e. compiling both versions into the dll and just using one?

Looking at the file object, the open function uses "_wfopen" which needs
Windows NT according to the MSDN lib. So, how is 9x compat ensured here?

Kind regards,
Alexander


From martin at v.loewis.de  Sun Jan 15 22:32:01 2006
From: martin at v.loewis.de (=?ISO-8859-15?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 15 Jan 2006 22:32:01 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <gi49ixye313w.dlg@usenet.alexanderweb.de>
References: <43C9A76B.8070801@stackless.com>
	<43CA0A01.4020406@v.loewis.de>	<43CA5881.3020305@stackless.com>
	<43CAA13B.2050902@v.loewis.de>
	<gi49ixye313w.dlg@usenet.alexanderweb.de>
Message-ID: <43CABF51.4020108@v.loewis.de>

Alexander Schremmer wrote:
>>>>There are several issues involved in implementing such a patch, though.
>>>>One is that you need to do it twice: once for Win9x, and once for
>>>>NT+, because you have to use Unicode file names on one system, and
>>>>ANSI file names on the other.
> 
> 
>>Right - the question would be whether completely breaking W9x support
>>in the process would be acceptable. We use the very same binaries
>>for W9x and NT.
> 
> 
> How about accessing/calling the wide char versions just if the OS is NT,
> i.e. compiling both versions into the dll and just using one?

Right. That's what I meant when I said: "you need to do it twice".

> Looking at the file object, the open function uses "_wfopen" which needs
> Windows NT according to the MSDN lib. So, how is 9x compat ensured here?

There is a GetVersion check in there; not sure how effective it is,
though. However, if you pass byte strings, as the file name, _wfopen
is not used.

Regards,
Martin

From martin at v.loewis.de  Mon Jan 16 01:06:16 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Jan 2006 01:06:16 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43C9A76B.8070801@stackless.com>
References: <43C9A76B.8070801@stackless.com>
Message-ID: <43CAE378.6050901@v.loewis.de>

Christian Tismer wrote:
> 1. create a file
> 2. get it's os.path.getmtime()
> 3. change your time zone
> 4. get os.path.getmtime again
> 
> compare - the time stamps are different.
> Change the time zone back, and they are identical, again.

Just to add an important detail here: I assume
you did not exit Python between step 2 and step 4, right?
(please confirm whether this is the case).

If so, it has nothing to do with daylight saving time.

If I start a new Python interpreter and fetch the mtime
again, it will report the value I got at step 2.

This is on NTFS, so the time stamps the system reports
(in FindFirstFile) are true UTC.

What happens is apparently this: msvcrt converts the UTC
time to local time, using FileTimeToLocalFileTime; this
gets the new time zone offset from the system. It then
converts it back through __loctotime_t. This invokes
__tzset, however, __tzset caches the _timezone offset,
so it is unaffected by the time zone change.

So *this* specific problem goes away as soon as we
start dropping msvcrt.

Regards,
Martin

P.S. The other problem (FindFirstFile gives bad UTC time
stamps on FAT, for files created in a different DST period)
has no real solution, AFAICT. There is no portable way to
determine whether the underlying file system stores time
stamps in local time or in UTC, and even if you know that
file stamps were in local time, you still couldn't reliably
convert that to UTC (because of the repeated hour, and
because of potential time zone changes).

So I would rather return to the user what the system gives
me, knowing that
a) the problem exists on FAT only, and people should use
   NTFS if they care about time stamps
b) it is better to document the limitation instead of
   trying to work around it: msvcrt shows that attempts
   to work around it make the problem worse, not better.
   So I would like to use FindFirstFile for stat(),
   and GetFileType+GetFileInformationByHandle for fstat().

From tim.peters at gmail.com  Mon Jan 16 03:46:19 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 15 Jan 2006 21:46:19 -0500
Subject: [Python-Dev] Birkenfeld's gone
In-Reply-To: <dpqp2g$u8f$1@sea.gmane.org>
References: <dpqp2g$u8f$1@sea.gmane.org>
Message-ID: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>

[Georg Brandl -- or so he claimed on January 8]
> today, when two Python developers here had approached me about the PSF, I
> realized that it is time to correct a mistake which I had made over three years
> ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time).
>
> I then assumed a different name, partly to anonymize myself as others had advised.
> When I came across Python, I instantly liked it, and since I had interest in
> Open Source development in general, I wanted to try and contribute to the
> development.
>
> And, as a matter of course, I used my different name for that. When I realized that
> I liked the community and the development of Python very much, I decided to
> "unveil" myself, but I could never overcome myself -- till now.
>
> I'm really sorry, and I hope you could at least partly understand what caused
> me to act so irrational.

It doesn't really matter, Georg:  by default, Python compares by
object identity, not name.  If _you_ haven't changed, your name
doesn't matter.

Still, it takes real courage to reveal one's true identity, and you're
to be commended for it.  If it will make you feel better, "Aahz" was
born Bartholomew Fortescue Chillington the Third, and I was christened
Griselda Dolores Bignose.  Curiously, I'm not sure Guido even
remembers his true name (but, obviously, nobody would name their kid
"Guido").

I hope that revealing my true name too, and outing Aahz, will inspire
everyone to reveal their secret powers and home planets too.  I won't
be the first, though -- as you've discovered, when you're the first
you get ridiculed ;-).

From aleaxit at gmail.com  Mon Jan 16 05:20:10 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Sun, 15 Jan 2006 20:20:10 -0800
Subject: [Python-Dev] Birkenfeld's gone
In-Reply-To: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
References: <dpqp2g$u8f$1@sea.gmane.org>
	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
Message-ID: <5C666F20-EB46-401F-B685-299BE5FD9ED7@gmail.com>


On Jan 15, 2006, at 6:46 PM, Tim Peters wrote:
     ...
> I hope that revealing my true name too, and outing Aahz, will inspire
> everyone to reveal their secret powers and home planets too.  I won't

OK, there I go -- "Alessandro Ferruccio Raffaele Martelli-Fortini".   
Good thing it's spelled in full only in one obscure church register  
and an even obscurer "Italian Register of Gentry", because I even  
feel it an imposition to have to spell out my legal first name  
"Alessandro" in some documents (it's that way in my passport, alas!)  
rather than the "Alex" I prefer... besides, no American ever spelled  
"Alessandro" right, I think they'd lose US citizenship if they did,  
or something, so I wanna stick with "Alex", which even THEY hardly  
ever manage to mis-spell (so they take their revenge by spelling my  
surname "Martinelli", like a popular local brand of [NON-alcoholic,  
yeck] cider... can't win, can't break even...).

Unfortunately none of my secret powers has to do with forcing  
Americans to spell correctly (indeed, that's rather my secret  
_weakness_ -- just seeing "its" and "it's" freely exchanged for each  
other instantly saps my will to live!)...


Alex


From allison at shasta.stanford.edu  Mon Jan 16 05:29:12 2006
From: allison at shasta.stanford.edu (Dennis Allison)
Date: Sun, 15 Jan 2006 20:29:12 -0800 (PST)
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>
Message-ID: <Pine.LNX.4.44.0601132104430.5489-100000@shasta.stanford.edu>

On Thu, 12 Jan 2006, Brett Cannon wrote:

> It is time once again in my educational career to come to python-dev
> for help for major project ideas.  This time, though, it is for my
> Ph.D. dissertation (and thus can have larger scope than my masters
> thesis) but there are funding restrictions (and thus only certain
> areas I have possible funding for).
> 
> First off, there are two areas where I have possible funding: XML
> integration into programming languages and game scripting support (I
> have a third, AOP for anything, but  I don't think AOP is a good match
> for Python and thus not considering it for Python work).  The XML
> integration I have no direct ideas since I don't use XML.  There is
> the possibility of doing something like the LINQ project
> (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp)
> or something, but I don't know how useful that could be.  It would
> have to be just as generic, though, as the LINQ project in terms of
> working with intefaces to be Pythonic so it might turn out to be more
> about language support for querying data.
> 
> For the game scripting, trying out the active objects for Python is a
> possibility (or something else that would help with concurrency). 
> Since that could allow for agent objects to be more concurrent it can
> be argued that it would help with game scripting.  I don't have any
> other ideas for the area of game scripting support.

Brett--

Financial considerations aside, neither of the ideas you mention see ready
to be a dissertation topic.  I'd encourge you to look to other topics.

You might research interpreter structures for languages like Python which
do not use a global interpreter lock (GIL).  There's been discussion about
this from time to time on this list, even some suggestions as to what
might be done.  Given the trend toward multi-core processors, Python would
benefit if all processors did not need to block on the GIL. Guido told me
once that someone had build a Python interpreter with multiple locks, but
that the peformance was not very good and that the interpreter was
fragile.  This is, of course, a hard problem.

In 1974, Dijkstra wrote a fascinating paper on self-stabalizing 
algorithms; since that time a considerable literature has grown up.

Managing a Python system these days is a management problem when the
standard system is extended with libraries from many different sources,
each with different release schedules, dependencies, compatibilities, and 
documentation.  Automating the management of Python and its libraries 
might make a good dissertation topic, particualarly if there is some 
advice giving system that helps find an appropriate library for a 
particular problem.

Testing, optimization, refactopring tools, etc.

Dave Altel, following others, suggested in a provacatively title talks--

 




From brett at python.org  Mon Jan 16 05:54:44 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 15 Jan 2006 20:54:44 -0800
Subject: [Python-Dev] Ph.D. dissertation ideas?
In-Reply-To: <Pine.LNX.4.44.0601132104430.5489-100000@shasta.stanford.edu>
References: <bbaeab100601121436t77ba9916ybd2de81191b0c109@mail.gmail.com>
	<Pine.LNX.4.44.0601132104430.5489-100000@shasta.stanford.edu>
Message-ID: <bbaeab100601152054q7cb16e7aqd836aa08dd7cb8b5@mail.gmail.com>

On 1/15/06, Dennis Allison <allison at shasta.stanford.edu> wrote:
> On Thu, 12 Jan 2006, Brett Cannon wrote:
>
> > It is time once again in my educational career to come to python-dev
> > for help for major project ideas.  This time, though, it is for my
> > Ph.D. dissertation (and thus can have larger scope than my masters
> > thesis) but there are funding restrictions (and thus only certain
> > areas I have possible funding for).
> >
> > First off, there are two areas where I have possible funding: XML
> > integration into programming languages and game scripting support (I
> > have a third, AOP for anything, but  I don't think AOP is a good match
> > for Python and thus not considering it for Python work).  The XML
> > integration I have no direct ideas since I don't use XML.  There is
> > the possibility of doing something like the LINQ project
> > (http://msdn.microsoft.com/netframework/future/linq/default.aspx?pull=/library/en-us/dndotnet/html/linqprojectovw.asp)
> > or something, but I don't know how useful that could be.  It would
> > have to be just as generic, though, as the LINQ project in terms of
> > working with intefaces to be Pythonic so it might turn out to be more
> > about language support for querying data.
> >
> > For the game scripting, trying out the active objects for Python is a
> > possibility (or something else that would help with concurrency).
> > Since that could allow for agent objects to be more concurrent it can
> > be argued that it would help with game scripting.  I don't have any
> > other ideas for the area of game scripting support.
>
> Brett--
>
> Financial considerations aside, neither of the ideas you mention see ready
> to be a dissertation topic.  I'd encourge you to look to other topics.
>

I'm open to other topics and in no way settled or locked into anything
mentioned so far.  Just need to be able to feed myself.  =)

> You might research interpreter structures for languages like Python which
> do not use a global interpreter lock (GIL).  There's been discussion about
> this from time to time on this list, even some suggestions as to what
> might be done.  Given the trend toward multi-core processors, Python would
> benefit if all processors did not need to block on the GIL. Guido told me
> once that someone had build a Python interpreter with multiple locks, but
> that the peformance was not very good and that the interpreter was
> fragile.  This is, of course, a hard problem.
>

It was Greg Stein who worked on it while at Microsoft.  He didn't
start to approach breaking even until two processors (if I remember
correctly).  None of the code was ever released.

This could fit under the concurrency or gaming funding if there is
some way to make threading just work better for Python.  So it's a
possibility.

> In 1974, Dijkstra wrote a fascinating paper on self-stabalizing
> algorithms; since that time a considerable literature has grown up.
>
> Managing a Python system these days is a management problem when the
> standard system is extended with libraries from many different sources,
> each with different release schedules, dependencies, compatibilities, and
> documentation.  Automating the management of Python and its libraries
> might make a good dissertation topic, particualarly if there is some
> advice giving system that helps find an appropriate library for a
> particular problem.
>

Possibilty.  Does tie into my lab's area of focus.

> Testing, optimization, refactopring tools, etc.
>

All possible general areas.

> Dave Altel, following others, suggested in a provacatively title talks--
>

Was there more to this line?  Seems like the email was cut off.

Thanks Dennis (and everyone, for that matter) for the continued help
and support!

-Brett

From tinuviel at sparcs.kaist.ac.kr  Mon Jan 16 05:37:18 2006
From: tinuviel at sparcs.kaist.ac.kr (Seo Sanghyeon)
Date: Mon, 16 Jan 2006 13:37:18 +0900
Subject: [Python-Dev] PEP 247 and hashlib
Message-ID: <20060116043718.GA1129@sparcs.kaist.ac.kr>

"PEP 247 -- API for Cryptographic Hash Functions" specifies a standard
API for hashing modules.

new([string])
... the optional 'string' parameter, if supplied, will be immediately
hashed into the object's starting state, as if obj.update(string) was
called.

But hashlib.new() takes the algorithm name... Does PEP need an update?

By the way, I am thinking about mapping .NET's
System.Security.Cryptography.HashAlgorithm.Create and created object's
ComputeHash method to Python's hash API for IronPython. For that, I'd
like to see a clarification.

Seo Sanghyeon

From aleaxit at gmail.com  Mon Jan 16 06:59:20 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Sun, 15 Jan 2006 21:59:20 -0800
Subject: [Python-Dev] basenumber redux
Message-ID: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>

For the last 2+ years I've been occasionally arguing for the  
introduction of a basenumber (and ideally a baseinteger, but that, to  
me, is a slightly lesser issue) analogous to basestring.  Google  
search fo [basenumber site:python.org] for several messages on the  
subject, by me and others; it will also find the recent thread about  
more general abstract baseclasses, which seems to have bogged down on  
such issues as whether sets are mappings.

Now, today, I have _again_ been bit by the lack of basenumber (by a  
bug of mine, fixed by adding decimal.Decimal to a long tuple of  
classes to be passed to an isinstance call -- I hadn't run that  
particular numeric code of mine since the time of Python 2.3,  
apparently), so I'm back to pining for it.  The previous discussion  
was short but pretty exhaustive, so I'd ask further discussants to  
refer back to it, rather than repeating it; no blocking issue appears  
to have emerged at that time, plenty of use cases were pointed out,  
etc.  Can we PLEASE have basenumber (and maybe baseinteger, so  
sequences can typecheck against that for their indices -- that's the  
key usecase of baseinteger) rather than have them "hijacked" by wider  
consideration of basesequence, basemapping, and so on...?  Pretty  
please....?  Let's be pragmatic: basenumber isn't at all complicated  
nor controversial, baseinteger hardly at all, so let's accept them  
while pondering on other potential base* classes for as long as it  
takes for the dust to settle....

I'll be happy to draft a PEP if needed (and just as happy to  
eventually provide an implementation patch if the PEP's accepted),  
but wanted to doublecheck on the general issue first!


Alex


From martin at v.loewis.de  Mon Jan 16 07:39:41 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Jan 2006 07:39:41 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
Message-ID: <43CB3FAD.2090602@v.loewis.de>

Alex Martelli wrote:
> I'll be happy to draft a PEP if needed (and just as happy to  
> eventually provide an implementation patch if the PEP's accepted),  
> but wanted to doublecheck on the general issue first!

Please do so. I've browsed somewhat through past discussions,
but wasn't able to find a proposed specification of basenumber.
Also, I only found half of a rationale for it: it is meant to
be used along with isinstance, but I couldn't find out why
you wanted to do that. In

http://mail.python.org/pipermail/python-dev/2003-November/039916.html

you explain that you wanted to multiply all items of self with
other if other is a number; why couldn't this be written as

  def __mul__(self, other):
    try:
       return self.__class__([ x*other for x in self.items ])
    except TypeError:
       # various other multiplication cases

You give performance as the rationale; this is unconvincing as
it would rather indicate that performance of exceptions should
be improved (also, I think it is unpythonic to change the
language for performance reasons, except in really common
cases).

Also, in that example, I wonder if the use of multiplication
is flawed. If you have so many multiplication cases, perhaps
you abuse the notion of multiplication? Users will need to
understand the different cases, as well, and they will be
surprised when it works in one case, but not in a (seemingly
similar) othercase.

Regards,
Martin

From ncoghlan at gmail.com  Mon Jan 16 10:05:24 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Mon, 16 Jan 2006 19:05:24 +1000
Subject: [Python-Dev] basenumber redux
In-Reply-To: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
Message-ID: <43CB61D4.4000908@gmail.com>

Alex Martelli wrote:
> I'll be happy to draft a PEP if needed (and just as happy to  
> eventually provide an implementation patch if the PEP's accepted),  
> but wanted to doublecheck on the general issue first!

I haven't really followed the earlier basenumber discussions (aside from the 
sidetrack into the nature of mappings), but why would we want this ability as 
a typecheck and not some form of interface check?

For example, what's wrong with "hasattr(x, __int__)"? That works for all the 
builtin types, and, IMO, anyone defining a direct conversion to an integer for 
a non-numeric type deserves whatever happens to them.

Something like:

   def is_number(x):
       return hasattr(x, '__int__')

   def is_integer(x):
       return x == int(x)

Requiring inheritance from "basenumber" in order to make something behave like 
a real number seems antithetical to both duck-typing and the adaptation PEP.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From anthony at interlink.com.au  Mon Jan 16 10:53:06 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon, 16 Jan 2006 20:53:06 +1100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <43CB61D4.4000908@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
	<43CB61D4.4000908@gmail.com>
Message-ID: <200601162053.08253.anthony@interlink.com.au>

On Monday 16 January 2006 20:05, Nick Coghlan wrote:
> For example, what's wrong with "hasattr(x, __int__)"? That works
> for all the builtin types, and, IMO, anyone defining a direct
> conversion to an integer for a non-numeric type deserves whatever
> happens to them.

What about something that's got something like:

  def __int__(self):
      raise TypeError("This type is not a number!")

I don't see a problem with defining basenumber. For the use cases, 
pretty much the same set as basesstring.



-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From theller at python.net  Mon Jan 16 10:53:35 2006
From: theller at python.net (Thomas Heller)
Date: Mon, 16 Jan 2006 10:53:35 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de>
Message-ID: <psms7b34.fsf@python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Guido van Rossum wrote:
>> On the other hand it breaks one of the most fundamental Python
>> guidelines: if you get a core dump (segfault etc.) it's a bug in
>> Python or in a 3rd party extension, not in *your* Python code. An
>> exception would have to be made for any code that uses ctypes, as it
>> is usually trivial to cause core dumps with ctypes (I'd venture it's
>> hard to avoid them ;-).
>> 
>> I don't expect this to count against including ctypes; but I do want
>> it to be dealt with somehow!
>
> I think the strongest point *for* ctypes is the inclusion of dl
> in the source distribution, which has the very same flaws as
> ctypes in terms of crashability.
>
> So as for dealing with it "somehow": I would make ctypes a dynamically
> loaded module (ctypes.pyd), so administrators could remove it, and
> I could also make it a separate option in the Windows installer,
> so administrators could reject to install it.

It looks like we need a pronouncement now.

Thomas


From jim at zope.com  Mon Jan 16 12:51:27 2006
From: jim at zope.com (Jim Fulton)
Date: Mon, 16 Jan 2006 06:51:27 -0500
Subject: [Python-Dev] Names matter.
In-Reply-To: <1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
References: <dpqp2g$u8f$1@sea.gmane.org>
	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
Message-ID: <43CB88BF.4080501@zope.com>

Tim Peters wrote:
> [Georg Brandl -- or so he claimed on January 8]
> 
>>today, when two Python developers here had approached me about the PSF, I
>>realized that it is time to correct a mistake which I had made over three years
>>ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time).
>>
>>I then assumed a different name, partly to anonymize myself as others had advised.
>>When I came across Python, I instantly liked it, and since I had interest in
>>Open Source development in general, I wanted to try and contribute to the
>>development.
>>
>>And, as a matter of course, I used my different name for that. When I realized that
>>I liked the community and the development of Python very much, I decided to
>>"unveil" myself, but I could never overcome myself -- till now.
>>
>>I'm really sorry, and I hope you could at least partly understand what caused
>>me to act so irrational.
> 
> 
> It doesn't really matter, Georg:  by default, Python compares by
> object identity, not name.  If _you_ haven't changed, your name
> doesn't matter.

I realize that folks are, rightly, trying to encourage Georg.  I think
protection of the identity of a minor on the Internet is understandable
and justifiable.

In general though, for adults, truthfulness and non-anonymity *do*
matter.  At least they matter to me, a *lot*.  I don't think members
of the PSF should be allowed to hide their identity and certainly,
it should not be acceptable to contribute to Python under a false name.

Jim

-- 
Jim Fulton           mailto:jim at zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org

From rganesan at myrealbox.com  Mon Jan 16 12:59:50 2006
From: rganesan at myrealbox.com (Ganesan Rajagopal)
Date: Mon, 16 Jan 2006 17:29:50 +0530
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <psms7b34.fsf@python.net>
Message-ID: <xkvh64okids9.fsf@grajagop-lnx.cisco.com>

>>>>> Thomas Heller <theller at python.net> writes:

> It looks like we need a pronouncement now.

+1

I am a lurker in this list. I maintain ctypes for Debian and I would love to
see it in core python. Debian now includes ctypes for 10 Linux architectures
and kFreeBSD. The only missing significant architecture is ARM because of
lack of FFI support.

Ganesan

-- 
Ganesan Rajagopal (rganesan at debian.org) | GPG Key: 1024D/5D8C12EA
Web: http://employees.org/~rganesan        | http://rganesan.blogspot.com



From connellybarnes at yahoo.com  Mon Jan 16 06:27:23 2006
From: connellybarnes at yahoo.com (Connelly Barnes)
Date: Sun, 15 Jan 2006 21:27:23 -0800 (PST)
Subject: [Python-Dev] timeit module
Message-ID: <20060116052723.31287.qmail@web54314.mail.yahoo.com>


Hi,

Perhaps I am the only one bothered by the timeit
module, but it seems poorly designed to me.

First of all, it should use a geometric series with a
timeout value to detect how many iterations it should
perform.  Currently, the user is required to manually
specify the number of iterations (the default is 1
million).  If the user optimizes his or her code, then
the number of iterations must be changed.  If the user
moves to a slower or faster computer, then the number
of iterations must be changed again.  This is
annoying.

Secondly, there should be a way to time a callable
directly.  That is, without finding the string name of
the callable and using a string "import X" statement. 
These contortions violate rules #1 and #3 of the Zen
of Python.

Yes, there is function call overhead in timing a
callable as opposed to a fragment of code.  However,
when I'm benchmarking code I am often timing
functions, so it seems logical to make this easy to
do.

I have three (mutually exclusive) ideas for improving
the current situation:

 * Add a time_callable() or time_func() function to
   the timeit module.  These would take the callable,
   args, kwargs, and maximum time* to spend timing the
   function as arguments.
 * Add a FuncTimer class (function/callable timer) to
the
   existing timeit module.  The constructor takes a
callable,
   args, kwargs as arguments.  The class has instance
methods  
   timeit(max_time=4.0) and repeat(repeat=3,
max_time=4.0).
 * Create a new timeit module (e.g. timeit2) which
uses
   geometric series to implement both code fragment
timing
   (which timeit does) and callable timing.

I have a simple implementation of geometric series
timing routines for callables and code fragments. 
This is recipe
440657.

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/440657

This recipe leaves the GC enabled.

Clearly, this is simple to implement.  If python-dev
likes the
idea of improving Python's timeit functionality, I
volunteer to
implement whichever interface is decided upon.

Opinions?

Thanks,
Connelly Barnes

* The maximum time will be exceeded only if a single
call to
  the callable takes more than max_time seconds.


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

From tismer at stackless.com  Mon Jan 16 13:56:13 2006
From: tismer at stackless.com (Christian Tismer)
Date: Mon, 16 Jan 2006 13:56:13 +0100
Subject: [Python-Dev] os.path.getmtime on Windows
In-Reply-To: <43CAE378.6050901@v.loewis.de>
References: <43C9A76B.8070801@stackless.com> <43CAE378.6050901@v.loewis.de>
Message-ID: <43CB97ED.4020307@stackless.com>

Martin v. L?wis wrote:
> Christian Tismer wrote:
>> 1. create a file
>> 2. get it's os.path.getmtime()
>> 3. change your time zone
>> 4. get os.path.getmtime again
>>
>> compare - the time stamps are different.
>> Change the time zone back, and they are identical, again.
> 
> Just to add an important detail here: I assume
> you did not exit Python between step 2 and step 4, right?
> (please confirm whether this is the case).

No, I did not exit Python.
The problem only appeared when I changed time zone.
Changing the date had no effect.

-- 
Christian Tismer             :^)   <mailto:tismer at stackless.com>
tismerysoft GmbH             :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9A     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 802 86 56  mobile +49 173 24 18 776  fax +49 30 80 90 57 05
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From steve at holdenweb.com  Mon Jan 16 13:57:51 2006
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 16 Jan 2006 12:57:51 +0000
Subject: [Python-Dev] Names matter.
In-Reply-To: <43CB88BF.4080501@zope.com>
References: <dpqp2g$u8f$1@sea.gmane.org>	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
	<43CB88BF.4080501@zope.com>
Message-ID: <dqg58f$73c$1@sea.gmane.org>

Jim Fulton wrote:
> Tim Peters wrote:
> 
>>[Georg Brandl -- or so he claimed on January 8]
>>
>>
>>>today, when two Python developers here had approached me about the PSF, I
>>>realized that it is time to correct a mistake which I had made over three years
>>>ago, when I discovered Linux, free software, Usenet etc (I was sixteen at that time).
>>>
>>>I then assumed a different name, partly to anonymize myself as others had advised.
>>>When I came across Python, I instantly liked it, and since I had interest in
>>>Open Source development in general, I wanted to try and contribute to the
>>>development.
>>>
>>>And, as a matter of course, I used my different name for that. When I realized that
>>>I liked the community and the development of Python very much, I decided to
>>>"unveil" myself, but I could never overcome myself -- till now.
>>>
>>>I'm really sorry, and I hope you could at least partly understand what caused
>>>me to act so irrational.
>>
>>
>>It doesn't really matter, Georg:  by default, Python compares by
>>object identity, not name.  If _you_ haven't changed, your name
>>doesn't matter.
> 
> 
> I realize that folks are, rightly, trying to encourage Georg.  I think
> protection of the identity of a minor on the Internet is understandable
> and justifiable.
> 
> In general though, for adults, truthfulness and non-anonymity *do*
> matter.  At least they matter to me, a *lot*.  I don't think members
> of the PSF should be allowed to hide their identity and certainly,
> it should not be acceptable to contribute to Python under a false name.
> 
In principle I think you are correct.

In practice it's difficult to see how to nsure this without insistence 
on some digital certificate from an acceptable CA - by which I mean one 
that actually checks a driving license or passport before asserting a 
user's identity. Both CACert and Thawte have web-of-trust-like schemes 
to support this.

In the meantine what do we do? Ask for a signed statement that 
contributors are contributing under their own identities?

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From steve at holdenweb.com  Mon Jan 16 14:05:18 2006
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 16 Jan 2006 13:05:18 +0000
Subject: [Python-Dev] timeit module
In-Reply-To: <20060116052723.31287.qmail@web54314.mail.yahoo.com>
References: <20060116052723.31287.qmail@web54314.mail.yahoo.com>
Message-ID: <dqg5mf$8mr$1@sea.gmane.org>

Connelly Barnes wrote:
> Hi,
> 
> Perhaps I am the only one bothered by the timeit
> module, but it seems poorly designed to me.
> 
> First of all, it should use a geometric series with a
> timeout value to detect how many iterations it should
> perform.  Currently, the user is required to manually
> specify the number of iterations (the default is 1

The provision of a default value generally seems to chop the legs from 
your argument that the number of iterations is required.

> million).  If the user optimizes his or her code, then
> the number of iterations must be changed.  If the user
> moves to a slower or faster computer, then the number
> of iterations must be changed again.  This is
> annoying.
> 
What? The purpose of timeit is to give an approximation to the length of 
time to run a specific piece ofcode.

Why must the number of iterations be changed when moving to a slower or 
faster computer?

> Secondly, there should be a way to time a callable
> directly.  That is, without finding the string name of
> the callable and using a string "import X" statement. 
> These contortions violate rules #1 and #3 of the Zen
> of Python.
> 
Presumably for benchmarking purposes the function call overhead would be 
present for all compaered techniques. Do you mean rules #0 and #2?

 > [...]
regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From amk at amk.ca  Mon Jan 16 15:32:29 2006
From: amk at amk.ca (A.M. Kuchling)
Date: Mon, 16 Jan 2006 09:32:29 -0500
Subject: [Python-Dev] PEP 247 and hashlib
In-Reply-To: <20060116043718.GA1129@sparcs.kaist.ac.kr>
References: <20060116043718.GA1129@sparcs.kaist.ac.kr>
Message-ID: <20060116143229.GB12888@rogue.amk.ca>

On Mon, Jan 16, 2006 at 01:37:18PM +0900, Seo Sanghyeon wrote:
> But hashlib.new() takes the algorithm name... Does PEP need an update?

The new() function in hashlib's interface doesn't conform to PEP 247;
that's all.  I don't think the PEP needs to be updated.

--amk

From guido at python.org  Mon Jan 16 16:45:55 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Jan 2006 07:45:55 -0800
Subject: [Python-Dev] Names matter.
In-Reply-To: <dqg58f$73c$1@sea.gmane.org>
References: <dpqp2g$u8f$1@sea.gmane.org>
	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
	<43CB88BF.4080501@zope.com> <dqg58f$73c$1@sea.gmane.org>
Message-ID: <ca471dc20601160745s3dd524bfk98830c6600310679@mail.gmail.com>

On 1/16/06, Steve Holden <steve at holdenweb.com> wrote:
> Jim Fulton wrote:
> > In general though, for adults, truthfulness and non-anonymity *do*
> > matter.  At least they matter to me, a *lot*.  I don't think members
> > of the PSF should be allowed to hide their identity and certainly,
> > it should not be acceptable to contribute to Python under a false name.

Agreed. Also, I find it difficult to exchange emails with someone who
is known by an alias only.

> In principle I think you are correct.
>
> In practice it's difficult to see how to nsure this without insistence
> on some digital certificate from an acceptable CA - by which I mean one
> that actually checks a driving license or passport before asserting a
> user's identity. Both CACert and Thawte have web-of-trust-like schemes
> to support this.
>
> In the meantine what do we do? Ask for a signed statement that
> contributors are contributing under their own identities?

I think that's unnecessary, and not any more effective than what we
already do -- ask for a signed contributor form. Someone who signs
that form "Tim Peters" won't be stopped by a clause asking them to use
their real name, *really*.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Jan 16 16:48:34 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Jan 2006 07:48:34 -0800
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <psms7b34.fsf@python.net>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <psms7b34.fsf@python.net>
Message-ID: <ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>

On 1/16/06, Thomas Heller <theller at python.net> wrote:
> It looks like we need a pronouncement now.

Sorry. It appeared to me that there was general agreement to using a
strongly worded warning in the docs, so I tuned out of the rest of the
discussion. So for the record, I'm fine with that.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Jan 16 16:53:20 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Jan 2006 07:53:20 -0800
Subject: [Python-Dev] basenumber redux
In-Reply-To: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
Message-ID: <ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>

On 1/15/06, Alex Martelli <aleaxit at gmail.com> wrote:
> Now, today, I have _again_ been bit by the lack of basenumber (by a
> bug of mine, fixed by adding decimal.Decimal to a long tuple of
> classes to be passed to an isinstance call -- I hadn't run that
> particular numeric code of mine since the time of Python 2.3,
> apparently), so I'm back to pining for it.

As you already suspected, I think a PEP is needed. The intent of
basestring was to *only* be used as the base class for *built-in*
string types. Clearly what you're proposing is different (Decimal is
not built-in -- not yet anyway).

Like other posters, I suspect that the best way of detecting numbers
might be some other kind of test, not necessarily a call to
isinstance().

It would also help to explain the user case more. ("I've been bitten"
doesn't convey a lot of information. :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.peters at gmail.com  Mon Jan 16 17:09:20 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 16 Jan 2006 11:09:20 -0500
Subject: [Python-Dev] [Python-checkins] r42064 -
	python/trunk/Lib/logging/handlers.py
In-Reply-To: <17355.49089.866569.489467@montanaro.dyndns.org>
References: <20060116090810.7D2581E4002@bag.python.org>
	<17355.49089.866569.489467@montanaro.dyndns.org>
Message-ID: <1f7befae0601160809n7c7888bg2534402750d03cf6@mail.gmail.com>

[vinay]
>> Log:
>>   Fixed bug in time-to-midnight calculation.

[Skip]
> Is there some reason not to use datetime to do this?

PEP 291 says logging intends to be compatible with Python 1.5.2, which
leaves datetime out.

Vinay, rather than:

    if (currentMinute == 0) and (currentSecond == 0):
        r = (24 - currentHour) * 60 * 60 # number of hours in seconds
    else:
        r = (23 - currentHour) * 60 * 60
        r = r + (59 - currentMinute) * 60 # plus the number of minutes (in secs)
        r = r + (60 - currentSecond) # pl

for clarity I suggest this instead:

# A module-level constant
_MIDNIGHT = 24 * 60 * 60  # number of seconds in a day

    r = _MIDNIGHT - ((currentHour * 60 + currentMinute) * 60 + currentSecond)

That's "obviously correct" instead of "huh?" <wink>.

From lists at core-sdi.com  Mon Jan 16 17:27:11 2006
From: lists at core-sdi.com (Gabriel Becedillas)
Date: Mon, 16 Jan 2006 13:27:11 -0300
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <43C7BB1F.3000904@corest.com>
References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net>
	<43C7BB1F.3000904@corest.com>
Message-ID: <43CBC95F.3020907@core-sdi.com>

Gabriel Becedillas wrote:
> Michael Hudson wrote:
>> Gabriel Becedillas <gabriel.becedillas at corest.com> writes:
>>
>>
>>> Hi,
>>> At the company I work for, we've embedded Python in C++ application 
>>> we develop. Since our upgrade to Python 2.4.2 from 2.4.1 we started 
>>> hitting Py_FatalError("Invalid thread state for this thread") when 
>>> using debug builds.
>>> We use both multiple interpreters and thread states.
>>
>>
>> I know you've said this to me in email already, but I have no idea how
>> you managed to get this to work with 2.4.1 :)
>>
>>
>>> I think the problem is that PyThreadState_Delete (which is used when I
>>> destroy thread states and interpreters) is not making the same clean up
>>> that PyThreadState_DeleteCurrent is doing (which is used when we 
>>> create threads from python code). If a thread id is reused (obviously 
>>> not between two running threads), and the previous thread used 
>>> PyThreadState_Delete instead of PyThreadState_DeleteCurrent, then an 
>>> old and invalid value for autoTLSkey is still stored, and that is
>>> triggering the Py_FatalError when a call to PyThreadState_Swap is made.
>>> If I add this code at the end of PyThreadState_Delete:
>>>
>>> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
>>>     PyThread_delete_key_value(autoTLSkey);
>>>
>>> then everything works fine.
>>
>>
>> I think I begin to understand the problem... but this is still
>> fragile: it relies on the fact that you delete a thread state from the
>> OS level thread that created it, but while a thread belonging to a
>> different *interpreter state* has the GIL (or at least: the
>> interpreter state of the thread state being deleted *doesn't* hold the
>> GIL).  Can you guarantee that?
> 
> Mmm... it doesn't have to do with a race condition or something. Let me 
> try to show you with an example (keep in mind that this relies on the 
> fact that at some moment the operating system gives you a thread with 
> the same id of another thread that allready finished executing):
> 
> // Initialize python.
> Py_Initialize();
> PyEval_InitThreads();
> PyThreadState* p_main =  PyThreadState_Swap(NULL);
> PyEval_ReleaseLock();
> 
> /*
> Asume this block is executed in a separate thread, and that has been 
> asigned thread id 10.
> It doesn't matter what the main thread (the thread that initialized 
> Python) is doing. Lets assume its waiting for this one to finish.
> */
> {
>     PyEval_AcquireLock();
>     PyThreadState_Swap(p_main);
>     PyThreadState* p_child = PyThreadState_New(p_main->interp);
> 
>     PyThreadState_Swap(p_child);
>     PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'");
> 
>     PyThreadState_Clear(p_child);
>     PyThreadState_Swap(NULL);
>     PyThreadState_Delete(p_child);
>     PyEval_ReleaseLock();
>     // The os level thread exits here.
> }
> /*
> When this thread state gets deleted (using PyThreadState_Delete), the 
> autoTLSkey stored for thread id number 10 is not removed, because 
> PyThreadState_Delete doesn't have the necesary cleanup code (check 
> pystate.c):
> 
> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
>     PyThread_delete_key_value(autoTLSkey);
> 
> PyThreadState_DeleteCurrent behaves correctly because it does have this 
> code.
> I think that this code should be added to tstate_delete_common (I've 
> done this and everything worked just fine).
> If a new thread executes the same code, and that thread is assigned the 
> same thread id, you get the Py_FatalError I'm talking about.
> A workaround for this would be to use PyThreadState_DeleteCurrent 
> instead of PyThreadState_Delete.
> 
> If you use the following code instead of the one above, the you have no 
> way out to the Py_FatalError:
> */
> {
>     PyEval_AcquireLock();
>     PyThreadState* ret = Py_NewInterpreter();
> 
>     PyRun_SimpleString("print 'mi vieja mula ya no es lo que era'");
> 
>     Py_EndInterpreter(ret);
>     PyThreadState_Swap(NULL);
>     PyEval_ReleaseLock();
> }
> /*
> When this interpreter gets deleted (using Py_EndInterpreter), its thread 
> state gets deleted using PyThreadState_Delete, and you are back to the 
> beginning of the problem.
> */
> 
> I hope this helps to clarify the problem.
> Thanks a lot and have a nice day.
> 
>> It seems to me that to be robust, you need a way of saying "delete the
>> thread local value of autoTLSKey from thread $FOO".  But this is all
>> very confusing...
>>
>>
>>> Could you please confirm if this is a bug ?
>>
>>
>> Yes, I think it's a bug.
>>
>> Cheers,
>> mwh
>>
> 
> 

Can anybody tell me if the patch I suggested is ok ?
That will be to add the following code at the end of PyThreadState_Delete:

if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
     PyThread_delete_key_value(autoTLSkey);

Thank you very much.

-- 


Gabriel Becedillas
Developer
CORE SECURITY TECHNOLOGIES

Florida 141 - 2? cuerpo - 7? piso
C1005AAC Buenos Aires - Argentina
Tel/Fax: (54 11) 5032-CORE (2673)
http://www.corest.com

From aleaxit at gmail.com  Mon Jan 16 18:27:31 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 16 Jan 2006 09:27:31 -0800
Subject: [Python-Dev] basenumber redux
In-Reply-To: <ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
Message-ID: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>


On Jan 16, 2006, at 7:53 AM, Guido van Rossum wrote:

> On 1/15/06, Alex Martelli <aleaxit at gmail.com> wrote:
>> Now, today, I have _again_ been bit by the lack of basenumber (by a
>> bug of mine, fixed by adding decimal.Decimal to a long tuple of
>> classes to be passed to an isinstance call -- I hadn't run that
>> particular numeric code of mine since the time of Python 2.3,
>> apparently), so I'm back to pining for it.
>
> As you already suspected, I think a PEP is needed. The intent of

I'll be happy to write it, if it stands any chance.

> basestring was to *only* be used as the base class for *built-in*
> string types. Clearly what you're proposing is different (Decimal is
> not built-in -- not yet anyway).

I can't find a PEP describing this restriction of basestring, and I  
don't see why a coder who needs to implement another kind of  
character string shouldn't subclass basestring, so that those  
instances pass an isinstance test on basestring which is quite likely  
to be found e.g. in the standard library.

Implementing different kinds of numbers is more likely than  
implementing different kinds of strings, of course.

> Like other posters, I suspect that the best way of detecting numbers
> might be some other kind of test, not necessarily a call to
> isinstance().

I've tended to use a try/except around x+0 to detect if "x is a  
number".  But that's NOT how the standard library does it -- rather,  
it has isinstance tests (often forgetting long in the tuple of  
types), as I pointed out in my mails on the subject back in 2003  
(google for [basenumber site:python.org], there aren't many).  I will  
reproduce those in the PEP, of course, if I do write one.

The x+0 test has been criticized in the past because x COULD be an  
instance of a type which defines an __add__ which has side effects,  
or a very inclusive __add__ which happens to accept an int argument  
even though the type is NOT meant to be a number.  These could be  
seen as design defects of x's type, of course.

A second argument against the x+0 test is performance.

A third argument against it is asymmetry: why should I use completely  
different approaches to check if x is "some kind of string", vs  
checking if x is "some kind of number"?  Once upon a time I used x 
+'' (with try/except around it) to check for stringness, but the  
introduction of basestring strongly signaled that this was not the  
"one obvious way" any more.

>
> It would also help to explain the user case more. ("I've been bitten"
> doesn't convey a lot of information. :-)

isinstance with a tuple of number types, where the tuple did not  
include Decimal (because when I developed and tested that module,  
Decimal wasn't around yet).

That's the problem of using isinstance without a "flag class" like  
basestring: one is hardwiring a specific tuple of types as being  
"singled out" for different treatment.  If a new type comes up (be it  
for the standard library or some extension) there's no way to respect  
the "open-closed principle", leaving the affected module "closed to  
changes" yet "open for extension".  In this way, using isinstance  
with a "hardwired" tuple of types is open to the same objections as  
"type-switching": it produces code that is not extensible to new  
types beyond those specific ones it had considered at coding time.

I have the same issue in the C-coded extension gmpy: I want (e.g.) a  
gmpy.mpq to be able to be constructed by passing any number as the  
argument, but I have no good way to say "what's a number", so I use  
rather dirty tricks -- in particular, I've had to tweak things in a  
weird direction in the latest gmpy to accomodate Python 2.4  
(specifically Decimal).

Since "being a number" is a protocol (albeit, like "being a string",  
a rather peculiar one -- "the verb TO BE" is always fraught;-), other  
traditional possibilities for supporting it are introducing a special  
method or flag attribute such as "__isanumber__" (either callable, or  
flagging numberhood just by its presence).  But introducing flag- 
abstract-class basenumber is more consistent with what was done for  
strings and affords a simpler test via isinstance.  Of course _if_  
PEP 246 was accepted, anybody could add more types to the set of  
those which "can be adapted to being numbers", but an object to  
denote that protocol should still be there (since the standard  
library does need to test for numberhood in a few places).

If I do write the PEP, should it be just about basenumber, or should  
it include baseinteger as well?  The case for the latter is IMHO  
still good but a bit weaker; it would be nice to be able to code 'xy'  
* Z without having str.__rmul__  perform a hardcoded test on Z being  
specifically an int or a long, making "other kinds of integers" (e.g.  
gmpy.mpz instances) smoothly substitutable for ints everywhere  
(similarly for somelist[Z], of course).  Right now I have to pepper  
my code with int(Z) casts when Z is a gmpy.mpz and I need to use it  
to index or multiply a sequence, which isn't nice for either  
readability or performance, but (in my experience so far, at least)  
these aren't _frequent_ needs, just occasional ones.


Thanks,

Alex


From fredrik at pythonware.com  Mon Jan 16 19:03:34 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 16 Jan 2006 19:03:34 +0100
Subject: [Python-Dev] basenumber redux
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com><ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
Message-ID: <dqgn5o$fmc$1@sea.gmane.org>

Alex Martelli wrote:

> > As you already suspected, I think a PEP is needed. The intent of
>
> I'll be happy to write it, if it stands any chance.
>
> > basestring was to *only* be used as the base class for *built-in*
> > string types. Clearly what you're proposing is different (Decimal is
> > not built-in -- not yet anyway).
>
> I can't find a PEP describing this restriction of basestring

that's how it's documented, at least:

    http://effbot.org/lib/builtin.basestring

> Implementing different kinds of numbers is more likely than
> implementing different kinds of strings, of course.

indeed.  implementing a new string type is not a trivial task in today's
CPython.

</F>




From steve at holdenweb.com  Mon Jan 16 19:48:23 2006
From: steve at holdenweb.com (Steve Holden)
Date: Mon, 16 Jan 2006 18:48:23 +0000
Subject: [Python-Dev] Names matter.
In-Reply-To: <ca471dc20601160745s3dd524bfk98830c6600310679@mail.gmail.com>
References: <dpqp2g$u8f$1@sea.gmane.org>	
	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>	
	<43CB88BF.4080501@zope.com> <dqg58f$73c$1@sea.gmane.org>
	<ca471dc20601160745s3dd524bfk98830c6600310679@mail.gmail.com>
Message-ID: <43CBEA77.5080105@holdenweb.com>

Guido van Rossum wrote:
[...]
> 
> I think that's unnecessary, and not any more effective than what we
> already do -- ask for a signed contributor form. Someone who signs
> that form "Tim Peters" won't be stopped by a clause asking them to use
> their real name, *really*.
> 
Sorry. My previous message suffered from a smiley-deficiency. Clearly 
there is little point in a signed piece of paper saying "I am me, signed 
'me'".

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/

From martin at v.loewis.de  Mon Jan 16 23:01:29 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Jan 2006 23:01:29 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
Message-ID: <43CC17B9.3060200@v.loewis.de>

Alex Martelli wrote:
> I can't find a PEP describing this restriction of basestring, and I  
> don't see why a coder who needs to implement another kind of  
> character string shouldn't subclass basestring, so that those  
> instances pass an isinstance test on basestring which is quite likely  
> to be found e.g. in the standard library.

People could do that, but they would be on their own. basestring
could be interpreted as "immutable sequence of characterish things",
but it was really meant to be "union of str and unicode". There
is no PEP because it was introduced by BDFL pronouncement.

That it is not a generic base class for stringish types can be
seen by looking at UserString.UserString, which doesn't inherit
from basestring.

For most practical purposes, those two definitions actually
define the same thing - there simply aren't any stringish data
types in praxis: you always have Unicode, and if you don't,
you have bytes.

> Implementing different kinds of numbers is more likely than  
> implementing different kinds of strings, of course.

Right. That's why a PEP is needed here, but not for basestring.

> A third argument against it is asymmetry: why should I use completely  
> different approaches to check if x is "some kind of string", vs  
> checking if x is "some kind of number"?

I guess that's for practicality which beats purity. People often
support interfaces that either accept both an individual string
and a list of strings, and they need the test in that case.
It would be most natural to look for whether it is a sequence;
unfortunately, strings are also sequences.

> isinstance with a tuple of number types, where the tuple did not  
> include Decimal (because when I developed and tested that module,  
> Decimal wasn't around yet).

As I suggested in a different message: Why are you doing that
in the first place?

> I have the same issue in the C-coded extension gmpy: I want (e.g.) a  
> gmpy.mpq to be able to be constructed by passing any number as the  
> argument, but I have no good way to say "what's a number", so I use  
> rather dirty tricks -- in particular, I've had to tweak things in a  
> weird direction in the latest gmpy to accomodate Python 2.4  
> (specifically Decimal).

Not sure what a gmpy.mpq is, but I would expect that can only work
if the  parameter belongs to some algebraic ring homomorphic
with the real numbers, or some such. Are complex numbers also numbers?
Is it meaningful to construct gmpy.mpqs out of them? What about
Z/nZ?

> If I do write the PEP, should it be just about basenumber, or should  
> it include baseinteger as well?

I think it should only do the case you care about. If others have other
use cases, they might get integrated, or they might have to write
another PEP.

Regards,
Martin

From martin at v.loewis.de  Mon Jan 16 23:02:35 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Jan 2006 23:02:35 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>
References: <u0cbyfa1.fsf@python.net>	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>	<43C58200.20803@v.loewis.de>
	<psms7b34.fsf@python.net>
	<ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>
Message-ID: <43CC17FB.5020904@v.loewis.de>

Guido van Rossum wrote:
> On 1/16/06, Thomas Heller <theller at python.net> wrote:
> 
>>It looks like we need a pronouncement now.
> 
> 
> Sorry. It appeared to me that there was general agreement to using a
> strongly worded warning in the docs, so I tuned out of the rest of the
> discussion. So for the record, I'm fine with that.

Ok. If Thomas checks in the code and the documentation, I'll do the
Windows/setup part of it.

Regards,
Martin

From martin at v.loewis.de  Mon Jan 16 23:07:15 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Mon, 16 Jan 2006 23:07:15 +0100
Subject: [Python-Dev] Names matter.
In-Reply-To: <43CB88BF.4080501@zope.com>
References: <dpqp2g$u8f$1@sea.gmane.org>	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
	<43CB88BF.4080501@zope.com>
Message-ID: <43CC1913.1050306@v.loewis.de>

Jim Fulton wrote:
> In general though, for adults, truthfulness and non-anonymity *do*
> matter.  At least they matter to me, a *lot*.  I don't think members
> of the PSF should be allowed to hide their identity and certainly,
> it should not be acceptable to contribute to Python under a false name.

Absolutely right. The encouragement is not for the past pseudonym, but
for actually revealing his true identity when it mattered (i.e. when
asked to "officially join"). For contributors, we should really track
copyright forms more thoroughly (and indeed, Georg said he sent one
right after I asked him).

Regards,
Martin

From t-meyer at ihug.co.nz  Mon Jan 16 23:10:54 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue, 17 Jan 2006 11:10:54 +1300
Subject: [Python-Dev] Names matter.
In-Reply-To: <dqg58f$73c$1@sea.gmane.org>
References: <dpqp2g$u8f$1@sea.gmane.org>	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
	<43CB88BF.4080501@zope.com> <dqg58f$73c$1@sea.gmane.org>
Message-ID: <12C84BD6-0F38-4D02-9807-47A4BDFAFFC8@ihug.co.nz>

[Jim Fulton]
> certainly, it should not be acceptable to contribute to Python  
> under a false name.

What do you mean "contribute to Python"?  Do you mean become one of  
the developers listed on sourceforge?  Contribute any patches, even  
simple documentation ones?  Review, comment and test patches?  Work  
on pydotorg?  Write python-dev summaries?  Give money to the PSF?   
(These, and many other things, are all contributing to Python, IMO).

I can understand that for legal purposes having some sort legally  
valid name might be required in some cases (e.g. contributing  
significant amounts of code), but wouldn't having that name known to  
the PSF suffice?  I don't see how a pseudonym hurts when dealing with  
the "public".

=Tony.Meyer

From aahz at pythoncraft.com  Tue Jan 17 01:43:34 2006
From: aahz at pythoncraft.com (Aahz)
Date: Mon, 16 Jan 2006 16:43:34 -0800
Subject: [Python-Dev] Names matter.
In-Reply-To: <43CB88BF.4080501@zope.com>
References: <dpqp2g$u8f$1@sea.gmane.org>
	<1f7befae0601151846h57bbdfa0s2609bf0d2bf06f44@mail.gmail.com>
	<43CB88BF.4080501@zope.com>
Message-ID: <20060117004334.GC12718@panix.com>

On Mon, Jan 16, 2006, Jim Fulton wrote:
>
> In general though, for adults, truthfulness and non-anonymity *do*
> matter.  At least they matter to me, a *lot*.  I don't think members
> of the PSF should be allowed to hide their identity and certainly, it
> should not be acceptable to contribute to Python under a false name.

And to me it's important for both political and personal reasons that
people be accepted -- nay, encouraged -- to use pseudonyms as they
please (or even choose to be anonymous).  I won't write a long screed on
this subject because it's off-topic here, but I will oppose any attempt
to limit access to the Python community from people who do not use their
legal names.  (See Alex's excellent post for more detail.)

One point where I do agree with you: I have little taste for morphing
aliases that refer to a single person.  If someone chooses to be
declaratively anonymous, that's fine; if someone chooses to use a stable
pseudonym (or a "real name" that happens to not be zir legal name),
that's fine, too.  But constantly changing identity does strike me as
abusing community.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From aleaxit at gmail.com  Tue Jan 17 03:18:10 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 16 Jan 2006 18:18:10 -0800
Subject: [Python-Dev] basenumber redux
In-Reply-To: <43CC17B9.3060200@v.loewis.de>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
	<43CC17B9.3060200@v.loewis.de>
Message-ID: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>


On Jan 16, 2006, at 2:01 PM, Martin v. L?wis wrote:

> Alex Martelli wrote:
>> I can't find a PEP describing this restriction of basestring, and I
>> don't see why a coder who needs to implement another kind of
>> character string shouldn't subclass basestring, so that those
>> instances pass an isinstance test on basestring which is quite likely
>> to be found e.g. in the standard library.
>
> People could do that, but they would be on their own. basestring
> could be interpreted as "immutable sequence of characterish things",
> but it was really meant to be "union of str and unicode". There
> is no PEP because it was introduced by BDFL pronouncement.

Unfortunately the lack of a PEP leaves this a bit underdocumented.

> That it is not a generic base class for stringish types can be
> seen by looking at UserString.UserString, which doesn't inherit
> from basestring.

Raymond Hettinger's reason for not wanting to add basestring as a  
base for UserString was: UserString is a legacy class, since today  
people can inherit from str directly, and it cannot be changed from a  
classic class to a new-style one without breaking backwards  
compatibility, which for a legacy class would be a big booboo.   
Nothing was said about "different design intent for basestring", as I  
recall (that discussion comes up among the few hits for [basestring  
site:python.org] if you want to check the details).

> For most practical purposes, those two definitions actually
> define the same thing - there simply aren't any stringish data
> types in praxis: you always have Unicode, and if you don't,
> you have bytes.

But not necessarily in one big blob that's consecutive (==compact) in  
memory.  mmap instances are "almost" strings and could easily be made  
into a closer match, at least for the immutable variants, for  
example; other implementations such as SGI STL's "Rope" also come to  
mind.

In the context of a current struggle (a different and long story)  
between Python builds with 2-bytes Unicode and ones with 4-bytes  
Unicode, I've sometimes found myself dreaming of a string type that's  
GUARANTEED to be 2-bytes, say, and against which extension modules  
could be written that don't need recompilation to move among such  
different builds, for example.  It's not (yet) hurting enough to make  
me hunker down and write such an extension (presumably mostly by copy- 
past-edit from Python's own sources;-), but if somebody did it would  
sure be nice if they could have that type "assert it's a string" by  
inheriting from basestring, no?

>> Implementing different kinds of numbers is more likely than
>> implementing different kinds of strings, of course.
>
> Right. That's why a PEP is needed here, but not for basestring.

OK, I've mailed requesting a number.

>
>> A third argument against it is asymmetry: why should I use completely
>> different approaches to check if x is "some kind of string", vs
>> checking if x is "some kind of number"?
>
> I guess that's for practicality which beats purity. People often
> support interfaces that either accept both an individual string
> and a list of strings, and they need the test in that case.
> It would be most natural to look for whether it is a sequence;
> unfortunately, strings are also sequences.

Sure, isinstance-tests with basestring are a fast and handy way to  
typetest that.

But so would similar tests with basenumber be.

>
>> isinstance with a tuple of number types, where the tuple did not
>> include Decimal (because when I developed and tested that module,
>> Decimal wasn't around yet).
>
> As I suggested in a different message: Why are you doing that
> in the first place?

Because isinstance is faster and handier than testing with try/except  
around (say) "x+0".

As to why I want to distinguish numbers from non-numbers, let me  
quote from a message I sent in 2003 (one of the few you'll find by  
searching for [basestring site:python.org] as I have repeatedly  
recommended, but apparently there's no way to avoid just massively  
copying and pasting...):

"""
def __mul__(self, other):
     if isinstance(other, self.KnownNumberTypes):
         return self.__class__([ x*other for x in self.items ])
     else:
         # etc etc, various other multiplication cases

right now, that (class, actually) attribute KnownNumberTypes starts out
"knowing" about int, long, float, gmpy.mpz, etc, and may require user
customization (e.g by subclassing) if any other "kind of (scalar)  
number"
needs to be supported; besides, the isinstance check must walk linearly
down the tuple of known number types each time.  (I originally had
quite a different test structure:
     try: other + 0
     except TypeError:  # other is not a number
         # various other multiplication cases
     else:
         # other is a number, so...
         return self.__class__([ x*other for x in self.items ])
but the performance for typical benchmarks improved with the isinstance
test, so, reluctantly, that's what I changed to).  If an abstract  
basetype
'basenumber' caught many useful cases, I'd put it right at the start of
the KnownNumberTypes tuple, omit all subclasses thereof from it, get
better performance, AND be able to document very simply what the user
must do to ensure his own custom type is known to me as "a number".
"""

Other use cases, still quoted from the very same message:

"""
in  Python/bltinmodule.c , function builtin_sum uses C-coded  
typechecking
to single out strings as an error case:

		/* reject string values for 'start' parameter */
		if (PyObject_TypeCheck(result, &PyBaseString_Type)) {
			PyErr_SetString(PyExc_TypeError,
				"sum() can't sum strings [use ''.join(seq) instea

[etc].  Now, what builtin_sum really "wants" to do is to accept numbers,
only -- it's _documented_ as being meant for "numbers": it uses +, NOT
+=, so its performance on sequences, matrix and array-ish things, etc,
is not going to be good.  But -- it can't easily _test_ whether  
something
"is a number".  If we had a PyBaseNumber_Type to use here, it would
be smooth, easy, and fast to check for it.
"""

and yet another, which is the directly gmpy-related one:

"""
I see a few other cases in the standard library which want to treat  
"numbers"
in some specific way different from other types (often forgetting  
longs:-),
e.g. Lib/plat-mac/plistlib.py has one.  In gmpy, I would often like some
operations to be able to accept "a number", perhaps by letting it try to
transform itself into a float as a worst case (so complex numbers  
would fail
there), but I definitely do NOT want to accept non-number objects which
"happen to be able to return a value from float(x)", such as  
strings.  In all
such cases of wanting to check if something "is a number", an abstract
basetype might be handy, smooth, fast.
"""

>
>> I have the same issue in the C-coded extension gmpy: I want (e.g.) a
>> gmpy.mpq to be able to be constructed by passing any number as the
>> argument, but I have no good way to say "what's a number", so I use
>> rather dirty tricks -- in particular, I've had to tweak things in a
>> weird direction in the latest gmpy to accomodate Python 2.4
>> (specifically Decimal).
>
> Not sure what a gmpy.mpq is, but I would expect that can only work

A fast rational number type, see http://gmpy.sourceforge.net for  
details (gmpy wraps LGPL'd library GMP, and gets a lot of speed and  
functionality thereby).

> if the  parameter belongs to some algebraic ring homomorphic
> with the real numbers, or some such. Are complex numbers also numbers?
> Is it meaningful to construct gmpy.mpqs out of them? What about
> Z/nZ?

If I could easily detect "this is a number" about an argument x, I'd  
then ask x to change itself into a float, so complex would be easily  
rejected (while decimals would mostly work fine, although a bit  
slowly without some specialcasing, due to the Stern-Brocot-tree  
algorithm I use to build gmpy.mpq's from floats).  I can't JUST ask x  
to "make itself into a float" (without checking for x's "being a  
number") because that would wrongfully succeed for many cases such as  
strings.


>> If I do write the PEP, should it be just about basenumber, or should
>> it include baseinteger as well?
>
> I think it should only do the case you care about. If others have  
> other
> use cases, they might get integrated, or they might have to write
> another PEP.

Good idea.  I'll check around -- if anybody feels strongly about  
baseinteger they may choose to co-author with me (and the PEP will  
propose both), otherwise we'll hassle on basenumber only;-)


Alex


From guido at python.org  Tue Jan 17 04:00:23 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Jan 2006 19:00:23 -0800
Subject: [Python-Dev] basenumber redux
In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
	<43CC17B9.3060200@v.loewis.de>
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
Message-ID: <ca471dc20601161900x2e14951br6767eadd556a86c6@mail.gmail.com>

On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
> Nothing was said about "different design intent for basestring", as I
> recall (that discussion comes up among the few hits for [basestring
> site:python.org] if you want to check the details).

Please. How many times do I have to say this. *I added this with the
stated intent.* Maybe I didn't communicate that intent clearly. But my
intent was exactly as it is documented -- basestring == (str, unicode)
and nothing else.

Why this intent? Because I felt the need to be able to distinguish the
*built-in* string types from *anything* else; they have many special
properties and purposes.

Also note that basestring was introduced in 2.3, a whole release
*after* inheritance from str was made possible.
--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aleaxit at gmail.com  Tue Jan 17 04:44:44 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 16 Jan 2006 19:44:44 -0800
Subject: [Python-Dev] str with base
Message-ID: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>

Is it finally time in Python 2.5 to allow the "obvious" use of, say,  
str(5,2) to give '101', just the converse of the way int('101',1)  
gives 5?  I'm not sure why str has never allowed this obvious use --  
any bright beginner assumes it's there and it's awkward to explain  
why it's not!-).  I'll be happy to propose a patch if the BDFL  
blesses this, but I don't even think it's worth a PEP... it's an  
inexplicable though long-standing omission (given the argumentative  
nature of this crowd I know I'll get pushback, but I still hope the  
BDFL can Pronounce about it anyway;-).


Alex


From python at rcn.com  Tue Jan 17 04:37:19 2006
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 16 Jan 2006 22:37:19 -0500
Subject: [Python-Dev] basenumber redux
In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
Message-ID: <001901c61b17$52c0c280$e221cb97@oemcomputer>

> Because isinstance is faster and handier than testing with try/except
> around (say) "x+0".

Are you sure?  The former has two builtin lookups (which also entail two
failed global lookups), a function call, and a test/jump for the result.
The latter approach has no lookups (just a load constant), a try-block
setup, an add operation (optimized for integers, a fast slot lookup
otherwise), and a block end.

Even if there were a performance edge, I suppose that the type checking
is the time critical part of most scripts.



> If an abstract
> basetype
> 'basenumber' caught many useful cases, I'd put it right at the start
of
> the KnownNumberTypes tuple, omit all subclasses thereof from it, get
> better performance, AND be able to document very simply what the user
> must do to ensure his own custom type is known to me as "a number".

So, this would essentially become a required API?  It would no longer be
enough to duck-type a number, you would also have to subclass from
basenumber?

Wouldn't this preclude custom number types that also need to derive from
another builtin such as str?  For instance, somewhere I have
Gram-Schmidt orthogonalization transformation code for computing
orthonormal bases and the only requirement for the input basis is that
it be an inner-product space -- accordingly, the inputs can be
functions, symbols (a subclass of str), or vectors (pretty-much anything
supporting multiplication and subtraction).

Similar conflicts arise for any pure computation function -- I should be
able to supply either numbers or symbolic inputs (a subclass of str)
that act like numbers.



> in  Python/bltinmodule.c , function builtin_sum uses C-coded
> typechecking
> to single out strings as an error case:
> 
> 		/* reject string values for 'start' parameter */
> 		if (PyObject_TypeCheck(result, &PyBaseString_Type)) {
> 			PyErr_SetString(PyExc_TypeError,
> 				"sum() can't sum strings [use
''.join(seq) instea
> 
> [etc].  Now, what builtin_sum really "wants" to do is to accept
numbers,
> only -- it's _documented_ as being meant for "numbers": it uses +, NOT
> +=, so its performance on sequences, matrix and array-ish things, etc,
> is not going to be good.  But -- it can't easily _test_ whether
> something
> "is a number".  If we had a PyBaseNumber_Type to use here, it would
> be smooth, easy, and fast to check for it.
> """

I think this a really a counter-example.  We wanted to preclude
sequences so that sum() didn't become a low performance synonym for
''.join().  However, there's no reason to preclude user-defined matrix
or vector classes.

And, if you're suggesting that the type-check should have been written
with as instance(result, basenumber), one consequence would be that
existing number-like classes would start to fail unless retro-fitted
with a basenumber superclass.  Besides being a PITA, retro-fitting
existing, working code could also entail changing from an old to
new-style class (possibly changing the semantics of the class).


> A fast rational number type, see http://gmpy.sourceforge.net for
> details (gmpy wraps LGPL'd library GMP, and gets a lot of speed and
> functionality thereby).
> 
> > if the  parameter belongs to some algebraic ring homomorphic
> > with the real numbers, or some such. Are complex numbers also
numbers?
> > Is it meaningful to construct gmpy.mpqs out of them? What about
> > Z/nZ?
> 
> If I could easily detect "this is a number" about an argument x, I'd
> then ask x to change itself into a float, so complex would be easily
> rejected (while decimals would mostly work fine, although a bit
> slowly without some specialcasing, due to the Stern-Brocot-tree
> algorithm I use to build gmpy.mpq's from floats).  I can't JUST ask x
> to "make itself into a float" (without checking for x's "being a
> number") because that would wrongfully succeed for many cases such as
> strings.

Part of the rationale for basestring was convenience of writing
isinstance(x,basestring) instead of isinstance(x,(str,unicode)).  No
existing code needed to be changed for this to work.

This proposal, on the other hand, is different.  To get the purported
benefits, everyone has to play along.  All existing number-look-a-like
classes would have to be changed in order to work with functions testing
for basenumber.

That would be too bad for existing, work code.  Likewise, it would be
impossible for symbolic code which already subclassed from str (as
discussed above).


> >> If I do write the PEP, should it be just about basenumber, or
should
> >> it include baseinteger as well?


Introducing a baseinteger type is likely to create confusion because all
integers are real, but baseintegers are not a subclass of floats.

I don't see a way around creating an integer recognition tool that
doesn't conflate its terminology with broadly-held, pre-existing math
knowledge: complex is a superset of reals, reals include rationals and
irrationals some of which are trancendental, and rationals include
integers which are an infinite superset of non-negative integers, whole
numbers, negative numbers, etc.

The decimal class only makes this more complicated.  All binary floats
can be translated exactly to decimal but not vice-versa.  I'm not sure
where they would fit into a inheritance hierarchy.  

On the one hand, decimals are very much like floats.  On the other hand,
they can be used as integers and should not be rejected by some code
testing for baseinteger unless the value actually has a fractional part.
The latter isn't a nit in the same league as wanting float('2.0') to be
acceptable as an integer argument; rather, the decimal class was
specifically designed to usable in integer-only situations (the spec
holds itself out as encompassing integer math, fixed-point, and
floating-point models).



Raymond
(my real name is unpronounceable by humans)

From brett at python.org  Tue Jan 17 04:50:55 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Jan 2006 19:50:55 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <bbaeab100601161950m6de8a1dfl3e977d3549a46045@mail.gmail.com>

On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> str(5,2) to give '101', just the converse of the way int('101',1)

I think you mean ``int('101' 2)``.  =)

> gives 5?  I'm not sure why str has never allowed this obvious use --
> any bright beginner assumes it's there and it's awkward to explain
> why it's not!-).  I'll be happy to propose a patch if the BDFL
> blesses this, but I don't even think it's worth a PEP... it's an
> inexplicable though long-standing omission (given the argumentative
> nature of this crowd I know I'll get pushback, but I still hope the
> BDFL can Pronounce about it anyway;-).
>

I'm +0.  Not a big thing for me, but having the symmetry seems reasonable.

-Brett

From python at rcn.com  Tue Jan 17 04:54:47 2006
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 16 Jan 2006 22:54:47 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <002d01c61b19$c2cec160$e221cb97@oemcomputer>

> Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> str(5,2) to give '101', just the converse of the way int('101',2)
> gives 5? 

+1

The only downside is that like list.reverse(), it will deprive students
of being assigned to write the roll-your-own version.  


Raymond

From python at rcn.com  Tue Jan 17 04:57:30 2006
From: python at rcn.com (Raymond Hettinger)
Date: Mon, 16 Jan 2006 22:57:30 -0500
Subject: [Python-Dev] basenumber redux
In-Reply-To: <001901c61b17$52c0c280$e221cb97@oemcomputer>
Message-ID: <002e01c61b1a$24014340$e221cb97@oemcomputer>

[Me]
> Even if there were a performance edge, I suppose that the type
checking
> is the time critical part of most scripts.

That should be:  RARELY the time critical part of most scripts.

From jeremy at alum.mit.edu  Tue Jan 17 05:03:17 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 16 Jan 2006 23:03:17 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <e8bf7a530601162003i10c51dd4n7631047b423d0ca6@mail.gmail.com>

It never occured to me that str() would behave like int() for this
case.  It makes complete sense to me that a factory for numbers would
ask about the base of the number.  What would the base of a string be,
except in a few limited cases? str([1, 2], 4) doesn't make any sense. 
You might argue that I wasn't all that bright as a beginner <0.5
wink>.

I think it shouldn't be changed, because the second positional
argument only works for a small number of the panoply types that can
be passed to str().  It would be fine to have a function for this
hiding somewhere, perhaps even as a method on numbers, but str() is
too generic.

Jeremy

On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> str(5,2) to give '101', just the converse of the way int('101',1)
> gives 5?  I'm not sure why str has never allowed this obvious use --
> any bright beginner assumes it's there and it's awkward to explain
> why it's not!-).  I'll be happy to propose a patch if the BDFL
> blesses this, but I don't even think it's worth a PEP... it's an
> inexplicable though long-standing omission (given the argumentative
> nature of this crowd I know I'll get pushback, but I still hope the
> BDFL can Pronounce about it anyway;-).
>
>
> Alex
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From guido at python.org  Tue Jan 17 05:04:27 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 16 Jan 2006 20:04:27 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>

On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> str(5,2) to give '101', just the converse of the way int('101',1)
[I'm sure you meant int('101', 2) here]
> gives 5?  I'm not sure why str has never allowed this obvious use --
> any bright beginner assumes it's there and it's awkward to explain
> why it's not!-).  I'll be happy to propose a patch if the BDFL
> blesses this, but I don't even think it's worth a PEP... it's an
> inexplicable though long-standing omission (given the argumentative
> nature of this crowd I know I'll get pushback, but I still hope the
> BDFL can Pronounce about it anyway;-).

I wish you had an argument better than "every bright beginner assumes
it exists". :-)

But (unlike for some other things that bright beginners might assume)
I don't think there's a deep reason why this couldn't exist.

The only reasons I can come up with is "because input and output are
notoriously asymmetric in Python" and "because nobody submitted a
patch". :-)

There are some corner cases to consider though.

- Should repr() support the same convention? I think not.
- Should str(3.1, n) be allowed? I think not.
- Should str(x, n) call x.__str__(n)? Neither.
- Should bases other than 2..36 be considered? I don't think so.

Unfortunately this doesn't obsolete oct() and hex() -- oct(10) returns
'012', while str(10, 8) soulc return '12'; hex(10) returns '0xa' while
str(10, 16) would return 'a'.

I do think that before we add this end-user nicety, it's more
important to implement __index__() in Python 2.5. This behaves like
__int__() for integral types, but is not defined for float or Decimal.
Operations that intrinsically require an integral argument, like
indexing and slicing, should call __index__() on their argument rather
than __int__(), so as to support non-built-in integral arguments while
still complaining about float arguments. This is currently implemented
by explicitly checking for float in a few places, which I find
repulsing. __index__() won't be requested by bright beginners, but it
is important e.g. to the Numeric Python folks, who like to implement
their own integral types but are suffering from that their integers
aren't usable everywhere.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From andrew-pythondev at puzzling.org  Tue Jan 17 05:08:02 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue, 17 Jan 2006 15:08:02 +1100
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <20060117040802.GG7771@home.puzzling.org>

On Mon, Jan 16, 2006 at 07:44:44PM -0800, Alex Martelli wrote:
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,  
> str(5,2) to give '101',

My reaction having read this far was "huh?".  It took some time (several
seconds) before it occurred to me what you wanted str(5,2) to mean, and why it
should give '101'.

If you'd proposed, say (5).as_binary() == '101', or "5".encode("base2"), I
wouldn't have been as baffled.  Or perhaps even str(5, base=2), but frankly the
idea of the string type doing numeric base conversions seems weird to me, rather
than symmetric.

I wouldn't mind seeing arbitrary base encoding of integers included somewhere,
but as a method of str -- let alone the constructor! -- it feels quite wrong.

-Andrew.


From raymond.hettinger at verizon.net  Tue Jan 17 05:13:27 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon, 16 Jan 2006 23:13:27 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
Message-ID: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>

> I wish you had an argument better than "every bright beginner assumes
> it exists". :-)
> 
> But (unlike for some other things that bright beginners might assume)
> I don't think there's a deep reason why this couldn't exist.
> 
> The only reasons I can come up with is "because input and output are
> notoriously asymmetric in Python" and "because nobody submitted a
> patch". :-)

My reason is that I've rolled-my-own more times than I can count but
infrequently enough to where it was easier to re-write than to search
for the previous use.

Another quick thought:  I presume that only the str() builtin would
change and that the underlying __str__ slot would continue to be
hard-wired to the (reprfunc) signature.


Raymond


From exarkun at divmod.com  Tue Jan 17 05:16:37 2006
From: exarkun at divmod.com (Jean-Paul Calderone)
Date: Mon, 16 Jan 2006 23:16:37 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <20060117041637.26200.1650489643.divmod.quotient.1321@ohm>

On Mon, 16 Jan 2006 19:44:44 -0800, Alex Martelli <aleaxit at gmail.com> wrote:
>Is it finally time in Python 2.5 to allow the "obvious" use of, say,
>str(5,2) to give '101', just the converse of the way int('101',1)
>gives 5?  I'm not sure why str has never allowed this obvious use --
>any bright beginner assumes it's there and it's awkward to explain
>why it's not!-).  I'll be happy to propose a patch if the BDFL
>blesses this, but I don't even think it's worth a PEP... it's an
>inexplicable though long-standing omission (given the argumentative
>nature of this crowd I know I'll get pushback, but I still hope the
>BDFL can Pronounce about it anyway;-).
>

-1.  Confusing and non-obvious.  The functionality may be valuable but 
it is mis-placed as a feature of str() or a method of the str type.  I 
work with a lot of Python beginners too, and while they occassionally 
ask for this functionality, I've never heard anyone wonder why str() 
didn't provide it or suggest that it should.

Jean-Paul

From barry at python.org  Tue Jan 17 05:18:11 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 16 Jan 2006 23:18:11 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <20060117040802.GG7771@home.puzzling.org>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<20060117040802.GG7771@home.puzzling.org>
Message-ID: <1137471491.5352.87.camel@geddy.wooz.org>

On Tue, 2006-01-17 at 15:08 +1100, Andrew Bennetts wrote:

> My reaction having read this far was "huh?".  It took some time (several
> seconds) before it occurred to me what you wanted str(5,2) to mean, and why it
> should give '101'.
> 
> If you'd proposed, say (5).as_binary() == '101', or "5".encode("base2"), I
> wouldn't have been as baffled.  Or perhaps even str(5, base=2), but frankly the
> idea of the string type doing numeric base conversions seems weird to me, rather
> than symmetric.
> 
> I wouldn't mind seeing arbitrary base encoding of integers included somewhere,
> but as a method of str -- let alone the constructor! -- it feels quite wrong.

Hear, hear.  I was similarly perplexed when I first read that!

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060116/ce26803a/attachment.pgp 

From aleaxit at gmail.com  Tue Jan 17 05:25:54 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Mon, 16 Jan 2006 20:25:54 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <e8bf7a530601162003i10c51dd4n7631047b423d0ca6@mail.gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<e8bf7a530601162003i10c51dd4n7631047b423d0ca6@mail.gmail.com>
Message-ID: <E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>


On Jan 16, 2006, at 8:03 PM, Jeremy Hylton wrote:

> It never occured to me that str() would behave like int() for this
> case.  It makes complete sense to me that a factory for numbers would
> ask about the base of the number.  What would the base of a string be,
> except in a few limited cases? str([1, 2], 4) doesn't make any sense.
> You might argue that I wasn't all that bright as a beginner <0.5
> wink>.
>
> I think it shouldn't be changed, because the second positional
> argument only works for a small number of the panoply types that can
> be passed to str().

Identically the same situation as for int: the base argument is only  
accepted if the first argument is a str (not a float, etc).  Just the  
same way, the base argument to str will only be accepted if the first  
argument is an int (not a float, etc).


Alex


From jeremy at alum.mit.edu  Tue Jan 17 05:47:50 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon, 16 Jan 2006 23:47:50 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<e8bf7a530601162003i10c51dd4n7631047b423d0ca6@mail.gmail.com>
	<E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>
Message-ID: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>

On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
>
> On Jan 16, 2006, at 8:03 PM, Jeremy Hylton wrote:
> > I think it shouldn't be changed, because the second positional
> > argument only works for a small number of the panoply types that can
> > be passed to str().
>
> Identically the same situation as for int: the base argument is only
> accepted if the first argument is a str (not a float, etc).  Just the
> same way, the base argument to str will only be accepted if the first
> argument is an int (not a float, etc).

The concept of base is closely related to ints, and the base argument
is useful for a large percentage of the types that int accepts.  It is
not related to strings, in general, and applies to only one of the
types it accepts.  In one case "not a float, etc." applies to a very
limited set of types, in the other case it applies to every
conceivable type (except int).

If str() were to take two argument, the analogy with int suggests it
should be an encoding, where the second argument describes how to
interpret the representation of the first (it's base 7 or it's utf-8).

Jeremy

From bob at redivi.com  Tue Jan 17 05:49:26 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 16 Jan 2006 20:49:26 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <1137471491.5352.87.camel@geddy.wooz.org>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<20060117040802.GG7771@home.puzzling.org>
	<1137471491.5352.87.camel@geddy.wooz.org>
Message-ID: <501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com>


On Jan 16, 2006, at 8:18 PM, Barry Warsaw wrote:

> On Tue, 2006-01-17 at 15:08 +1100, Andrew Bennetts wrote:
>
>> My reaction having read this far was "huh?".  It took some time  
>> (several
>> seconds) before it occurred to me what you wanted str(5,2) to  
>> mean, and why it
>> should give '101'.
>>
>> If you'd proposed, say (5).as_binary() == '101', or "5".encode 
>> ("base2"), I
>> wouldn't have been as baffled.  Or perhaps even str(5, base=2),  
>> but frankly the
>> idea of the string type doing numeric base conversions seems weird  
>> to me, rather
>> than symmetric.
>>
>> I wouldn't mind seeing arbitrary base encoding of integers  
>> included somewhere,
>> but as a method of str -- let alone the constructor! -- it feels  
>> quite wrong.
>
> Hear, hear.  I was similarly perplexed when I first read that!

The only bases I've ever really had a good use for are 2, 8, 10, and  
16.  There are currently formatting codes for 8 (o), 10 (d, u), and  
16 (x, X).  Why not just add a string format code for unsigned  
binary?  The obvious choice is probably "b".

For example:

 >>> '%08b' % (12)
'00001100'
 >>> '%b' % (12)
'1100'

I'd probably expect "5".encode("base2") to return '00110101', because  
"5".encode("hex") returns '35'

-bob


From raymond.hettinger at verizon.net  Tue Jan 17 05:54:05 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Mon, 16 Jan 2006 23:54:05 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>
Message-ID: <003b01c61b22$0bbfe040$e221cb97@oemcomputer>

[Jeremy Hylton]
> The concept of base is closely related to ints, and the base argument
> is useful for a large percentage of the types that int accepts.  It is
> not related to strings, in general, and applies to only one of the
> types it accepts.  In one case "not a float, etc." applies to a very
> limited set of types, in the other case it applies to every
> conceivable type (except int).


That suggests that it would be better to simply add an int method:

     x.convert_to_base(7)


Raymond


From andrew-pythondev at puzzling.org  Tue Jan 17 06:12:53 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue, 17 Jan 2006 16:12:53 +1100
Subject: [Python-Dev] str with base
In-Reply-To: <003b01c61b22$0bbfe040$e221cb97@oemcomputer>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>
	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>
Message-ID: <20060117051253.GH7771@home.puzzling.org>

On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote:
[...]
> That suggests that it would be better to simply add an int method:
> 
>      x.convert_to_base(7)

This seems clear and simple to me.  I like it.  I strongly suspect the "bright
beginners" Alex is interested in would have no trouble using it or finding it.

-Andrew.


From bob at redivi.com  Tue Jan 17 06:28:10 2006
From: bob at redivi.com (Bob Ippolito)
Date: Mon, 16 Jan 2006 21:28:10 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <20060117051253.GH7771@home.puzzling.org>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>
	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>
	<20060117051253.GH7771@home.puzzling.org>
Message-ID: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>


On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote:

> On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote:
> [...]
>> That suggests that it would be better to simply add an int method:
>>
>>      x.convert_to_base(7)
>
> This seems clear and simple to me.  I like it.  I strongly suspect  
> the "bright
> beginners" Alex is interested in would have no trouble using it or  
> finding it.

I don't know about that, all of the methods that int and long  
currently have are __special__.  They'd really need to start with  
Python 2.5 (assuming int/long grow "public methods" in 2.5) to even  
think to look there.  A format code or a built-in would be more  
likely to be found, since that's how you convert integers to hex and  
oct string representations with current Python.

 >>> [name for name in dir(0)+dir(0L) if not name.startswith('__')]
[]

-bob


From andrew-pythondev at puzzling.org  Tue Jan 17 06:34:32 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue, 17 Jan 2006 16:34:32 +1100
Subject: [Python-Dev] str with base
In-Reply-To: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>
	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>
	<20060117051253.GH7771@home.puzzling.org>
	<771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
Message-ID: <20060117053432.GI7771@home.puzzling.org>

On Mon, Jan 16, 2006 at 09:28:10PM -0800, Bob Ippolito wrote:
> On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote:
[...]
> >>     x.convert_to_base(7)
> >
> >This seems clear and simple to me.  I like it.  I strongly suspect  
> >the "bright
> >beginners" Alex is interested in would have no trouble using it or  
> >finding it.
> 
> I don't know about that, all of the methods that int and long  
> currently have are __special__.  They'd really need to start with  
> Python 2.5 (assuming int/long grow "public methods" in 2.5) to even  
> think to look there.  A format code or a built-in would be more  
> likely to be found, since that's how you convert integers to hex and  
> oct string representations with current Python.
> 
> >>> [name for name in dir(0)+dir(0L) if not name.startswith('__')]
> []

I should have said, I'm equally happy with the format code as well (although it
doesn't allow arbitary base conversions, I've never had need for that, so I'm
not too worried about that case).  Either option is better than making the str
constructor do relatively rarely used mathematics!

-Andrew.


From barry at python.org  Tue Jan 17 07:03:32 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Jan 2006 01:03:32 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<20060117040802.GG7771@home.puzzling.org>
	<1137471491.5352.87.camel@geddy.wooz.org>
	<501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com>
Message-ID: <1137477812.5351.91.camel@geddy.wooz.org>

On Mon, 2006-01-16 at 20:49 -0800, Bob Ippolito wrote:

> The only bases I've ever really had a good use for are 2, 8, 10, and  
> 16.  There are currently formatting codes for 8 (o), 10 (d, u), and  
> 16 (x, X).  Why not just add a string format code for unsigned  
> binary?  The obvious choice is probably "b".
> 
> For example:
> 
>  >>> '%08b' % (12)
> '00001100'
>  >>> '%b' % (12)
> '1100'

+1

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060117/14da01fc/attachment.pgp 

From brett at python.org  Tue Jan 17 07:41:06 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Jan 2006 22:41:06 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
Message-ID: <bbaeab100601162241i71478668t3da06fff46e01ab5@mail.gmail.com>

On 1/16/06, Guido van Rossum <guido at python.org> wrote:
> On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
> > Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> > str(5,2) to give '101', just the converse of the way int('101',1)
> [I'm sure you meant int('101', 2) here]
> > gives 5?  I'm not sure why str has never allowed this obvious use --
> > any bright beginner assumes it's there and it's awkward to explain
> > why it's not!-).  I'll be happy to propose a patch if the BDFL
> > blesses this, but I don't even think it's worth a PEP... it's an
> > inexplicable though long-standing omission (given the argumentative
> > nature of this crowd I know I'll get pushback, but I still hope the
> > BDFL can Pronounce about it anyway;-).
>
> I wish you had an argument better than "every bright beginner assumes
> it exists". :-)
>
> But (unlike for some other things that bright beginners might assume)
> I don't think there's a deep reason why this couldn't exist.
>
> The only reasons I can come up with is "because input and output are
> notoriously asymmetric in Python" and "because nobody submitted a
> patch". :-)
>
> There are some corner cases to consider though.
>
> - Should repr() support the same convention? I think not.
> - Should str(3.1, n) be allowed? I think not.
> - Should str(x, n) call x.__str__(n)? Neither.
> - Should bases other than 2..36 be considered? I don't think so.
>
> Unfortunately this doesn't obsolete oct() and hex() -- oct(10) returns
> '012', while str(10, 8) soulc return '12'; hex(10) returns '0xa' while
> str(10, 16) would return 'a'.
>
> I do think that before we add this end-user nicety, it's more
> important to implement __index__() in Python 2.5. This behaves like
> __int__() for integral types, but is not defined for float or Decimal.
> Operations that intrinsically require an integral argument, like
> indexing and slicing, should call __index__() on their argument rather
> than __int__(), so as to support non-built-in integral arguments while
> still complaining about float arguments. This is currently implemented
> by explicitly checking for float in a few places, which I find
> repulsing. __index__() won't be requested by bright beginners, but it
> is important e.g. to the Numeric Python folks, who like to implement
> their own integral types but are suffering from that their integers
> aren't usable everywhere.
>

+1 from me (feel like this has come up before, but not totally sure). 
Be nice to add an abstraction for indexing.

Added to the PyCon wiki as a possible sprint topic.

-Brett

From martin at v.loewis.de  Tue Jan 17 07:41:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Jan 2006 07:41:13 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
	<43CC17B9.3060200@v.loewis.de>
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
Message-ID: <43CC9189.2060206@v.loewis.de>

Alex Martelli wrote:
>> As I suggested in a different message: Why are you doing that
>> in the first place?
> 
> 
> Because isinstance is faster and handier than testing with try/except 
> around (say) "x+0".

Nit: I think you should not test. Instead, you should starting you mean
to do if the test passes, and then expect TypeErrors from doing so.

> As to why I want to distinguish numbers from non-numbers, let me  quote
> from a message I sent in 2003 (one of the few you'll find by  searching
> for [basestring site:python.org] as I have repeatedly  recommended, but
> apparently there's no way to avoid just massively  copying and pasting...):

As I said in that other message, I found this one, and still don't
understand what the use case is, because the example you give still
reads hypothetical (I'm sure there is an actual example behind it, but
you don't say what that is).

> """
> def __mul__(self, other):
>     if isinstance(other, self.KnownNumberTypes):
>         return self.__class__([ x*other for x in self.items ])
>     else:
>         # etc etc, various other multiplication cases

So what *are* the "various other multiplication cases"?

You just shouldn't be doing that: multiplication shouldn't mean
"item multiplication" sometimes, and various other things if it
can't mean item multiplication.

> in  Python/bltinmodule.c , function builtin_sum uses C-coded  typechecking
> to single out strings as an error case:
> 
>         /* reject string values for 'start' parameter */
>         if (PyObject_TypeCheck(result, &PyBaseString_Type)) {
>             PyErr_SetString(PyExc_TypeError,
>                 "sum() can't sum strings [use ''.join(seq) instea

This is indeed confusing: why is it again that sum refuses to sum
up strings?

> [etc].  Now, what builtin_sum really "wants" to do is to accept numbers,
> only -- it's _documented_ as being meant for "numbers": it uses +, NOT
> +=, so its performance on sequences, matrix and array-ish things, etc,
> is not going to be good.  But -- it can't easily _test_ whether  something
> "is a number".  If we had a PyBaseNumber_Type to use here, it would
> be smooth, easy, and fast to check for it.

There shouldn't be a check at all. It should just start doing the
summing, and it will "work" if PyNumber_Add works for the individual
items. Of course, there is an education value of ruling out string
addition, since there is a better way to do that, and there should
be only one obvious way.

I see nothing wrong in summing up sequences, matrices, and arrayish
things, using sum.

> A fast rational number type, see http://gmpy.sourceforge.net for 
> details (gmpy wraps LGPL'd library GMP, and gets a lot of speed and 
> functionality thereby).

Ok, so mpq are rational numbers.

>> if the  parameter belongs to some algebraic ring homomorphic
>> with the real numbers, or some such. Are complex numbers also numbers?
>> Is it meaningful to construct gmpy.mpqs out of them? What about
>> Z/nZ?
> 
> 
> If I could easily detect "this is a number" about an argument x, I'd 
> then ask x to change itself into a float, so complex would be easily 
> rejected (while decimals would mostly work fine, although a bit  slowly
> without some specialcasing, due to the Stern-Brocot-tree  algorithm I
> use to build gmpy.mpq's from floats).  I can't JUST ask x  to "make
> itself into a float" (without checking for x's "being a  number")
> because that would wrongfully succeed for many cases such as  strings.

Hmm. If you want to do the generic conversion from numbers to rationals
by going through float, then this is what you should do. Convert to
float, and don't bother with checking whether it will succeed. Instead,
if the type conversion gives an error, just return that to the caller.

However, it also sounds odd that you are trying to do the
to-rational-with-arbitrary-precision conversion by going through
floats. Instead, if the argument is decimal, you really should
do the division by the approprate base of 10, no?

Regards,
Martin

From brett at python.org  Tue Jan 17 07:45:23 2006
From: brett at python.org (Brett Cannon)
Date: Mon, 16 Jan 2006 22:45:23 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>
	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>
	<20060117051253.GH7771@home.puzzling.org>
	<771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
Message-ID: <bbaeab100601162245i7b93e0efq3150b2998b0e0106@mail.gmail.com>

On 1/16/06, Bob Ippolito <bob at redivi.com> wrote:
>
> On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote:
>
> > On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote:
> > [...]
> >> That suggests that it would be better to simply add an int method:
> >>
> >>      x.convert_to_base(7)
> >
> > This seems clear and simple to me.  I like it.  I strongly suspect
> > the "bright
> > beginners" Alex is interested in would have no trouble using it or
> > finding it.
>
> I don't know about that, all of the methods that int and long
> currently have are __special__.  They'd really need to start with
> Python 2.5 (assuming int/long grow "public methods" in 2.5) to even
> think to look there.  A format code or a built-in would be more
> likely to be found, since that's how you convert integers to hex and
> oct string representations with current Python.
>
>  >>> [name for name in dir(0)+dir(0L) if not name.startswith('__')]

If a method is the best solution, then fine, 2.5 is the beginning of
methods on int/long.  We could do a static method like
int.from_str("101", 2) and str.from_int(5, 2) if people don't like the
overloading of the constructors.  Otherwise add methods like
'101'.to_int(2) or 5 .to_str(2) .

-Brett

From martin at v.loewis.de  Tue Jan 17 07:46:17 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Jan 2006 07:46:17 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <001901c61b17$52c0c280$e221cb97@oemcomputer>
References: <001901c61b17$52c0c280$e221cb97@oemcomputer>
Message-ID: <43CC92B9.2030709@v.loewis.de>

Raymond Hettinger wrote:
> Are you sure?  The former has two builtin lookups (which also entail two
> failed global lookups), a function call, and a test/jump for the result.
> The latter approach has no lookups (just a load constant), a try-block
> setup, an add operation (optimized for integers, a fast slot lookup
> otherwise), and a block end.
> 
> Even if there were a performance edge, I suppose that the type checking
> is the time critical part of most scripts.

My guess is that it depends on the common case. If the common case is
that the type test fails (i.e. element-wise operations are the
exception), then you also have the exception creation and storing
in the exception approach, compared to returning only existing objects
in the type test case. If the common case is that x+0 succeeds, x+0
may or may not create new objects.

However, I have long ago learned not to guess about performance, so
I won't do further guessing until I see the actual code.

Regards,
Martin

From martin at v.loewis.de  Tue Jan 17 09:21:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 17 Jan 2006 09:21:59 +0100
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <43CCA927.8030600@v.loewis.de>

Alex Martelli wrote:
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,  
> str(5,2) to give '101', just the converse of the way int('101',1)  
> gives 5?  I'm not sure why str has never allowed this obvious use --  
> any bright beginner assumes it's there and it's awkward to explain  
> why it's not!-). 

My main concern is what the impact on __str__ would be. It seems
"obvious" that

  def str(obj, *args):
    return obj.__str__(*args)

because it is ultimately int's responsibility to interpret the base
argument, not str's.

People would then come up with use cases like

  class Color:
    msg = {'en':['red', 'green', 'blue'], 'de':['rot','gr?n','blau']}
    def __str__(self, language='en'):
      return self.msg[language][self.value]

  red = Color(0)

so you could say

  print str(red, 'de')

I don't think I like this direction.

Regards,
Martin

From abo at minkirri.apana.org.au  Tue Jan 17 09:52:55 2006
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Tue, 17 Jan 2006 08:52:55 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <1137477812.5351.91.camel@geddy.wooz.org>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<20060117040802.GG7771@home.puzzling.org>
	<1137471491.5352.87.camel@geddy.wooz.org>
	<501A02B9-738E-4334-8621-50E0E1AB3736@redivi.com>
	<1137477812.5351.91.camel@geddy.wooz.org>
Message-ID: <1137487975.4342.0.camel@warna.dub.corp.google.com>

On Tue, 2006-01-17 at 01:03 -0500, Barry Warsaw wrote:
> On Mon, 2006-01-16 at 20:49 -0800, Bob Ippolito wrote:
> 
> > The only bases I've ever really had a good use for are 2, 8, 10, and  
> > 16.  There are currently formatting codes for 8 (o), 10 (d, u), and  
> > 16 (x, X).  Why not just add a string format code for unsigned  
> > binary?  The obvious choice is probably "b".
> > 
> > For example:
> > 
> >  >>> '%08b' % (12)
> > '00001100'
> >  >>> '%b' % (12)
> > '1100'
> 
> +1

+1 me too.

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From mal at egenix.com  Tue Jan 17 11:02:22 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 17 Jan 2006 11:02:22 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>	<43CC17B9.3060200@v.loewis.de>
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
Message-ID: <43CCC0AE.5070800@egenix.com>

Alex, I think you're missing a point here: what you are looking
for is an interface, not a base class - simply because the
assumptions you make when finding a "KnownNumberTypes" instance
are only related to an interface you expect them to provide.

A common case class won't really help all that much with this,
since the implementations of the different types will vary a
lot (unlike, for example, strings and Unicode, which implement
a very common interface) and not necessarily provide a common
interface.

If you look at the Python C API, you'll find that "a number"
is actually never tested. The tests always ask for either
integers or floats.

The addition of a basenumber base class won't make these any
simpler.

Here's a snippet which probably does what you're looking for
using Python's natural way of hooking up to an implicit
interface:

import UserString

STRING_TYPES = (basestring, UserString.UserString)

def floatnumber(obj):
    if isinstance(obj, STRING_TYPES):
        raise TypeError('strings are not numbers')

    # Convert to a float
    try:
        return float(obj)
    except (AttributeError, TypeError, ValueError):
        raise TypeError('%r is not a float' % obj)

def intnumber(obj):
    if isinstance(obj, STRING_TYPES):
        raise TypeError('strings are not numbers')

    # Convert to an integer
    try:
        value = int(obj)
    except (AttributeError, TypeError, ValueError):
        raise TypeError('%r is not an integer' % obj)

    # Double check so that we don't lose precision
    try:
        floatvalue = floatnumber(obj)
    except TypeError:
        return value
    if floatvalue != value:
        raise TypeError('%r is not an integer' % obj)

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 17 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From nick at craig-wood.com  Tue Jan 17 11:05:44 2006
From: nick at craig-wood.com (Nick Craig-Wood)
Date: Tue, 17 Jan 2006 10:05:44 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
Message-ID: <20060117100544.GC1419@craig-wood.com>

On Mon, Jan 16, 2006 at 11:13:27PM -0500, Raymond Hettinger wrote:
> My reason is that I've rolled-my-own more times than I can count but
> infrequently enough to where it was easier to re-write than to search
> for the previous use.

Me too!  The assymetry is annoying.  Its easy to consume base 2..36
integers, but its hard to generate them.

However str() seems far too important to monkey with to me.

I like a method on int that would be great.  That keeps all the base
conversions in int (either in __init__ or in as_yet_unnamed_method()).

Another suggestion would be to give hex() and oct() another parameter,
base, so you'd do hex(123123123, 2). Perhaps a little
counter-intuitive, but if you were looking for base conversion
functions you'd find hex() pretty quickly and the documentation would
mention the other parameter.

-- 
Nick Craig-Wood <nick at craig-wood.com> -- http://www.craig-wood.com/nick

From mal at egenix.com  Tue Jan 17 11:30:37 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Tue, 17 Jan 2006 11:30:37 +0100
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <43CCC74D.80105@egenix.com>

Alex Martelli wrote:
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,  
> str(5,2) to give '101', just the converse of the way int('101',1)  
> gives 5?  I'm not sure why str has never allowed this obvious use --  
> any bright beginner assumes it's there and it's awkward to explain  
> why it's not!-).  I'll be happy to propose a patch if the BDFL  
> blesses this, but I don't even think it's worth a PEP... it's an  
> inexplicable though long-standing omission (given the argumentative  
> nature of this crowd I know I'll get pushback, but I still hope the  
> BDFL can Pronounce about it anyway;-).

Hmm, how about this:

str(obj, ifisunicode_decode_using_encoding='ascii',
         ifisinteger_use_base=10,
         ifisfile_open_and_read_it=False,
         isdecimal_use_precision=10,
         ismoney_use_currency='EUR',
         isdatetime_use_format='%c')

and so on ?!

Or even better:

str(obj, **kws) and then call obj.__str__(**kws) instead of
just obj.__str__() ?!


Seriously, shouldn't these more specific "convert to a string"
functions be left to specific object methods or helper
functions ?

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 17 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From ianb at colorstudy.com  Tue Jan 17 11:36:54 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 17 Jan 2006 04:36:54 -0600
Subject: [Python-Dev] str with base
In-Reply-To: <771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>	<20060117051253.GH7771@home.puzzling.org>
	<771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
Message-ID: <43CCC8C6.8060503@colorstudy.com>

Bob Ippolito wrote:
> On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote:
> 
> 
>>On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote:
>>[...]
>>
>>>That suggests that it would be better to simply add an int method:
>>>
>>>     x.convert_to_base(7)
>>
>>This seems clear and simple to me.  I like it.  I strongly suspect  
>>the "bright
>>beginners" Alex is interested in would have no trouble using it or  
>>finding it.
> 
> 
> I don't know about that, all of the methods that int and long  
> currently have are __special__.  They'd really need to start with  
> Python 2.5 (assuming int/long grow "public methods" in 2.5) to even  
> think to look there.  A format code or a built-in would be more  
> likely to be found, since that's how you convert integers to hex and  
> oct string representations with current Python.

How about just stuffing some function in the math module?  Everything in 
that module works on floats, but it seems incidental to me; I'm pretty 
sure I've even looked in that module for such a function before.  But 
it's such an obscure need that only comes up in the kind of algorithms 
students write, that it just seems odd and unnecessary to put it in 
str() which is *so* much more general than int() is.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From bob at redivi.com  Tue Jan 17 11:44:05 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 02:44:05 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <43CCC8C6.8060503@colorstudy.com>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>	<20060117051253.GH7771@home.puzzling.org>
	<771D4186-062D-446A-9621-95FD6048FCEB@redivi.com>
	<43CCC8C6.8060503@colorstudy.com>
Message-ID: <A2B164BA-C80A-41C2-BB6D-9D5E0C304C47@redivi.com>


On Jan 17, 2006, at 2:36 AM, Ian Bicking wrote:

> Bob Ippolito wrote:
>> On Jan 16, 2006, at 9:12 PM, Andrew Bennetts wrote:
>>> On Mon, Jan 16, 2006 at 11:54:05PM -0500, Raymond Hettinger wrote:
>>> [...]
>>>
>>>> That suggests that it would be better to simply add an int method:
>>>>
>>>>     x.convert_to_base(7)
>>>
>>> This seems clear and simple to me.  I like it.  I strongly  
>>> suspect  the "bright
>>> beginners" Alex is interested in would have no trouble using it  
>>> or  finding it.
>> I don't know about that, all of the methods that int and long   
>> currently have are __special__.  They'd really need to start with   
>> Python 2.5 (assuming int/long grow "public methods" in 2.5) to  
>> even  think to look there.  A format code or a built-in would be  
>> more  likely to be found, since that's how you convert integers to  
>> hex and  oct string representations with current Python.
>
> How about just stuffing some function in the math module?   
> Everything in that module works on floats, but it seems incidental  
> to me; I'm pretty sure I've even looked in that module for such a  
> function before.  But it's such an obscure need that only comes up  
> in the kind of algorithms students write, that it just seems odd  
> and unnecessary to put it in str() which is *so* much more general  
> than int() is.

I want binary all the time when I'm dealing with bitflags and such.   
Of course, I'm trained to be able to read bits in hex format, but it  
would be nicer to see the flags as-is.  Even worse when you have to  
deal with some kind of file format where fields are N bits long,  
where N is not a multiple of 8.

-bob


From fredrik at pythonware.com  Tue Jan 17 11:48:04 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 17 Jan 2006 11:48:04 +0100
Subject: [Python-Dev] str with base
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>	<20060117051253.GH7771@home.puzzling.org><771D4186-062D-446A-9621-95FD6048FCEB@redivi.com><43CCC8C6.8060503@colorstudy.com>
	<A2B164BA-C80A-41C2-BB6D-9D5E0C304C47@redivi.com>
Message-ID: <dqii14$kab$1@sea.gmane.org>

Bob Ippolito wrote:

> I want binary all the time when I'm dealing with bitflags and such.
> Of course, I'm trained to be able to read bits in hex format, but it
> would be nicer to see the flags as-is.  Even worse when you have to
> deal with some kind of file format where fields are N bits long,
> where N is not a multiple of 8.

so you want flags for bit order and fill order too, I assume ?

</F> 




From arigo at tunes.org  Tue Jan 17 11:14:02 2006
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 17 Jan 2006 11:14:02 +0100
Subject: [Python-Dev] Summer of PyPy
In-Reply-To: <bbaeab100601141751g5b345918q71e7787c5233f162@mail.gmail.com>
References: <bbaeab100601121658r533c69e2q35aa441d1671bd89@mail.gmail.com>
	<-8397309676539309595@unknownmsgid>
	<bbaeab100601121929ic27211dk4585e83f1138ad54@mail.gmail.com>
	<20060114101454.GA29087@code1.codespeak.net>
	<bbaeab100601141751g5b345918q71e7787c5233f162@mail.gmail.com>
Message-ID: <20060117101402.GA6458@code1.codespeak.net>

Hi Brett, hi all,

On Sat, Jan 14, 2006 at 05:51:25PM -0800, Brett Cannon wrote:
> That would be cool!  I definitely would not mind working on PyPy. 
> Unfortunately I would not consider changing universities; I really
> like it here.

We are looking at the possibility to do a "Summer of PyPy" in the same
style as last year's Google's Summer of Code.  It might be a way for you
(or anybody else interested!) to get to work a bit on PyPy first :-)

  http://codespeak.net/pipermail/pypy-dev/2006q1/002721.html


A bientot,

Armin

From bob at redivi.com  Tue Jan 17 12:12:17 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 03:12:17 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <dqii14$kab$1@sea.gmane.org>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>	<20060117051253.GH7771@home.puzzling.org><771D4186-062D-446A-9621-95FD6048FCEB@redivi.com><43CCC8C6.8060503@colorstudy.com>
	<A2B164BA-C80A-41C2-BB6D-9D5E0C304C47@redivi.com>
	<dqii14$kab$1@sea.gmane.org>
Message-ID: <A150A403-4B0E-4E83-89B1-EB07DA5BADB5@redivi.com>


On Jan 17, 2006, at 2:48 AM, Fredrik Lundh wrote:

> Bob Ippolito wrote:
>
>> I want binary all the time when I'm dealing with bitflags and such.
>> Of course, I'm trained to be able to read bits in hex format, but it
>> would be nicer to see the flags as-is.  Even worse when you have to
>> deal with some kind of file format where fields are N bits long,
>> where N is not a multiple of 8.
>
> so you want flags for bit order and fill order too, I assume ?

Not really, big endian covers almost everything I need.. and when it  
doesn't, then I can just flip and/or pad the string accordingly.

-bob


From abo at minkirri.apana.org.au  Tue Jan 17 12:54:58 2006
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Tue, 17 Jan 2006 11:54:58 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <20060117100544.GC1419@craig-wood.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
Message-ID: <1137498898.4342.68.camel@warna.dub.corp.google.com>

On Tue, 2006-01-17 at 10:05 +0000, Nick Craig-Wood wrote:
> On Mon, Jan 16, 2006 at 11:13:27PM -0500, Raymond Hettinger wrote:
[...]
> Another suggestion would be to give hex() and oct() another parameter,
> base, so you'd do hex(123123123, 2). Perhaps a little
> counter-intuitive, but if you were looking for base conversion
> functions you'd find hex() pretty quickly and the documentation would
> mention the other parameter.

Ugh!

I still favour extending % format strings. I really like '%b' for
binary, but if arbitary bases are really wanted, then perhaps also
leverage off the "precision" value for %d to indicate base such that '%
3.3d' % 5 = " 12"

If people think that using "." is for "precision" and is too ambiguous
for "base", you could do something like extend the whole conversion
specifier to (in EBNF)

conversion=%[mapping][flags][width][.precision][@base][modifier]type

which would allow for weird things like "%8.4 at 3f" % 5.5 == " 12.1111"

Note: it is possible for floats to be represented in non-decimal number
systems, its just extremely rare for anyone to do it. I have in my
distant past used base 16 float notation for fixed-point numbers.

I personally think %b would be adding enough. The other suggestions are
just me being silly :-)

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From mwh at python.net  Tue Jan 17 13:00:52 2006
From: mwh at python.net (Michael Hudson)
Date: Tue, 17 Jan 2006 12:00:52 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <1137498898.4342.68.camel@warna.dub.corp.google.com> (Donovan
	Baarda's message of "Tue, 17 Jan 2006 11:54:58 +0000")
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
Message-ID: <2mek37oyh7.fsf@starship.python.net>

Donovan Baarda <abo at minkirri.apana.org.au> writes:

> I personally think %b would be adding enough. The other suggestions are
> just me being silly :-)

Yeah, the whole area is just crying out for the simplicity and
restraint that is common lisp's #'format function :)

Cheers,
mwh

-- 
  <exarkun> INEFFICIENT CAPITALIST YOUR OPULENT 
            TOILET WILL BE YOUR UNDOING         -- from Twisted.Quotes

From collinw at gmail.com  Tue Jan 17 14:12:53 2006
From: collinw at gmail.com (Collin Winter)
Date: Tue, 17 Jan 2006 14:12:53 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <001901c61b17$52c0c280$e221cb97@oemcomputer>
References: <37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
	<001901c61b17$52c0c280$e221cb97@oemcomputer>
Message-ID: <43aa6ff70601170512y30dfe3d0ib420827ef5d9df5@mail.gmail.com>

On 1/17/06, Raymond Hettinger <python at rcn.com> wrote:
[snip]
> I don't see a way around creating an integer recognition tool that
> doesn't conflate its terminology with broadly-held, pre-existing math
> knowledge: complex is a superset of reals, reals include rationals and
> irrationals some of which are trancendental, and rationals include
> integers which are an infinite superset of non-negative integers, whole
> numbers, negative numbers, etc.
>
> The decimal class only makes this more complicated.  All binary floats
> can be translated exactly to decimal but not vice-versa.  I'm not sure
> where they would fit into a inheritance hierarchy.

To repeat a popular suggestion these days, python might borrow a page
from Haskell. Haskell's Prelude_ defines a number (pardon the pun) of
numeric typeclasses, each of which requires certain members. The
inheritance graph shapes up roughly like this:

Num - the ur-base class for all numbers
Real - inherits from Num
Integral - inherits from Real. Integral numbers support integer division
Fractional - inherits from Num. Fractionals support true division
Floating - inherits from Fractional. Floating-class objects support
trigonometric and hyperbolic functions and related functions.
RealFrac - inherits from Real and Fractional. This is used to operate
on the components of fractions.
RealFloat - inherits from RealFrac and Floating, providing efficient,
machine-independent access to the components of a floating-point
number.

While it may not be necessary to concoct that many base classes for
python, having a solid selection of such classes to subclass would
reduce the need for heuristics like attribute testing. Moreover,
subclassing a (or several) numeric type classes makes your intentions
explicit, rather than relying on testing for an "implicit interface".

Given the impact this would have on legacy code, and the need to refit
the built-in types and standard library, such a chance might be better
put off until Python 3k.

_Prelude - http://www.haskell.org/ghc/docs/latest/html/libraries/base/Prelude.html

Thanks,
Collin Winter

From jason.orendorff at gmail.com  Tue Jan 17 15:23:29 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Tue, 17 Jan 2006 09:23:29 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <2mek37oyh7.fsf@starship.python.net>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
Message-ID: <bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>

It seems dumb to support *parsing* integers in weird bases, but not
*formatting* them in weird bases.  Not a big deal, but if you're going
to give me a toy, at least give me the whole toy!

The %b idea is a little disappointing in two ways.  Even with %b,
Python is still dumb by the above criterion.  And, I suspect users
that don't know about %b are unlikely to find it when they want it.  I
know I've never looked for it there.

I think a method 5664400.to_base(13) sounds nice.

-j

From andrew-pythondev at puzzling.org  Tue Jan 17 15:30:48 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed, 18 Jan 2006 01:30:48 +1100
Subject: [Python-Dev] str with base
In-Reply-To: <bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
Message-ID: <20060117143048.GO7771@home.puzzling.org>

On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote:
> It seems dumb to support *parsing* integers in weird bases, but not
> *formatting* them in weird bases.  Not a big deal, but if you're going
> to give me a toy, at least give me the whole toy!
> 
> The %b idea is a little disappointing in two ways.  Even with %b,
> Python is still dumb by the above criterion.  And, I suspect users
> that don't know about %b are unlikely to find it when they want it.  I
> know I've never looked for it there.
> 
> I think a method 5664400.to_base(13) sounds nice.

It's also a SyntaxError.  With the current syntax, it would need to be
"(5664400).to_base(13)" or "5664400 .to_base(13)".

Why not just make the %b take a base, e.g.: 
    '%13b' % 5664400

<wink>

-Andrew.


From jimjjewett at gmail.com  Tue Jan 17 15:49:46 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 17 Jan 2006 09:49:46 -0500
Subject: [Python-Dev] index (was str with base)
Message-ID: <fb6fbf560601170649p402e09dfvec7c79ec4681d979@mail.gmail.com>

Guido wrote:

> more important to implement __index__() in Python 2.5.
> This behaves like __int__() for integral types, but is not
> defined for float or Decimal.

Why not for Decimal, or even float?  I would not be surprised
if 10.798 failed, but I would expect 1000D to work.

If indexing worked with more arbitrary extension integers,
then I would expect it work with Decimal, and possibly float,
when the number == a whole number, and to raise ValueError
otherwise.

-jJ

From jimjjewett at gmail.com  Tue Jan 17 15:51:58 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Tue, 17 Jan 2006 09:51:58 -0500
Subject: [Python-Dev] str with base
Message-ID: <fb6fbf560601170651i10ffe15wba55d3e7be75b3b@mail.gmail.com>

Raymond wrote:

> I presume that only the str() builtin would
> change and that the underlying __str__ slot
> would continue to be hard-wired to the
> (reprfunc) signature.

Are you saying that subclasses of str should
behave differently from the builtin str, and not
in the slots that were added or changed?

-jJ

From skip at pobox.com  Tue Jan 17 16:17:36 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 17 Jan 2006 09:17:36 -0600
Subject: [Python-Dev] str with base
In-Reply-To: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
Message-ID: <17357.2704.438248.183461@montanaro.dyndns.org>


    Raymond> My reason is that I've rolled-my-own more times than I can
    Raymond> count but infrequently enough to where it was easier to
    Raymond> re-write than to search for the previous use.

Maybe a bin() builtin would be better.  Even better it seems to me would be
to add a method to ints and longs that returns a string formatted in a base
between 2 and 36 (then deprecate hex and oct).  Like Jeremy, I wonder what
str([1,2], 4) means.

Skip

From skip at pobox.com  Tue Jan 17 16:19:35 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 17 Jan 2006 09:19:35 -0600
Subject: [Python-Dev] str with base
In-Reply-To: <E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<e8bf7a530601162003i10c51dd4n7631047b423d0ca6@mail.gmail.com>
	<E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>
Message-ID: <17357.2823.765518.514875@montanaro.dyndns.org>


    Alex> Identically the same situation as for int: the base argument is
    Alex> only accepted if the first argument is a str (not a float, etc).
    Alex> Just the same way, the base argument to str will only be accepted
    Alex> if the first argument is an int (not a float, etc).

A shortcoming in int() hardly seems like a good reason to mess with str().

Skip

From gmccaughan at synaptics-uk.com  Tue Jan 17 16:22:37 2006
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Tue, 17 Jan 2006 15:22:37 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <17357.2823.765518.514875@montanaro.dyndns.org>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>
	<17357.2823.765518.514875@montanaro.dyndns.org>
Message-ID: <200601171522.38184.gmccaughan@synaptics-uk.com>

On Tuesday 2006-01-17 15:19, skip at pobox.com wrote:

>     Alex> Identically the same situation as for int: the base argument is
>     Alex> only accepted if the first argument is a str (not a float, etc).
>     Alex> Just the same way, the base argument to str will only be accepted
>     Alex> if the first argument is an int (not a float, etc).
> 
> A shortcoming in int() hardly seems like a good reason to mess with str().

How's it a shortcoming in int() that it doesn't do anything with,
say, int(2.345,19)? (What would you like it to do?) Or are you
saying that the fact that int(<a string>) lets you specify a base
to interpret the string in is itself a shortcoming, and if so
why?

-- 
g


From guido at python.org  Tue Jan 17 16:26:57 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jan 2006 07:26:57 -0800
Subject: [Python-Dev] index (was str with base)
In-Reply-To: <fb6fbf560601170649p402e09dfvec7c79ec4681d979@mail.gmail.com>
References: <fb6fbf560601170649p402e09dfvec7c79ec4681d979@mail.gmail.com>
Message-ID: <ca471dc20601170726qf4c8097sde16fa1f87ed8b2c@mail.gmail.com>

On 1/17/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> Guido wrote:
>
> > more important to implement __index__() in Python 2.5.
> > This behaves like __int__() for integral types, but is not
> > defined for float or Decimal.
>
> Why not for Decimal, or even float?  I would not be surprised
> if 10.798 failed, but I would expect 1000D to work.
>
> If indexing worked with more arbitrary extension integers,
> then I would expect it work with Decimal, and possibly float,
> when the number == a whole number, and to raise ValueError
> otherwise.

Sorry, I forgot to explain this. There's a very good reason why this
should not be allowed.

You don't know if the 1000D was the result of an exact or of an
approximate calculation. It could be a pure coincidence that it's
1000D instead of 999.9D; and we certainly don't want the latter to be
allowed for indexing. Floating point (including Decimal floating
point) calculations are fraught with uncertainties about the precision
/ accuracy of the outcome. Even if when you test your algorithm you
always get exact outcomes, there's no guarantee that with real input
data the same will hold.

Requiring the index to be an int by *type* solves this by forcing you
to think about rounding/truncating. I talked to some Numeric folks and
they understand and agree.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Tue Jan 17 16:38:42 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 17 Jan 2006 09:38:42 -0600
Subject: [Python-Dev] str with base
In-Reply-To: <200601171522.38184.gmccaughan@synaptics-uk.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com>
	<17357.2823.765518.514875@montanaro.dyndns.org>
	<200601171522.38184.gmccaughan@synaptics-uk.com>
Message-ID: <17357.3970.537715.827114@montanaro.dyndns.org>


    Alex> Identically the same situation as for int: the base argument is
    Alex> only accepted if the first argument is a str (not a float, etc).
    Alex> Just the same way, the base argument to str will only be accepted
    Alex> if the first argument is an int (not a float, etc).
    >> 
    Skip> A shortcoming in int() hardly seems like a good reason to mess
    Skip> with str().

    Gareth> How's it a shortcoming in int() that it doesn't do anything
    Gareth> with, say, int(2.345,19)?

My reasoning was that just because int() was written to ignore the second
arg depending on type (the "shortcoming") doesn't mean that str() should as
well.

Skip

From fredrik at pythonware.com  Tue Jan 17 17:27:38 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 17 Jan 2006 17:27:38 +0100
Subject: [Python-Dev] str with base
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com><E4A3C19E-B62A-4040-9C05-8791FFC22A73@gmail.com><17357.2823.765518.514875@montanaro.dyndns.org><200601171522.38184.gmccaughan@synaptics-uk.com>
	<17357.3970.537715.827114@montanaro.dyndns.org>
Message-ID: <dqj5tq$ts$1@sea.gmane.org>

skip at pobox.com wrote:

>    Skip> A shortcoming in int() hardly seems like a good reason to mess
>    Skip> with str().
>
>    Gareth> How's it a shortcoming in int() that it doesn't do anything
>    Gareth> with, say, int(2.345,19)?
>
> My reasoning was that just because int() was written to ignore the second
> arg depending on type (the "shortcoming") doesn't mean that str() should as
> well.

"ignore" is perhaps the wrong word:

>>> int(1.0, 1)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
TypeError: int() can't convert non-string with explicit base

</F> 




From rhamph at gmail.com  Tue Jan 17 17:48:08 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 09:48:08 -0700
Subject: [Python-Dev] index (was str with base)
In-Reply-To: <ca471dc20601170726qf4c8097sde16fa1f87ed8b2c@mail.gmail.com>
References: <fb6fbf560601170649p402e09dfvec7c79ec4681d979@mail.gmail.com>
	<ca471dc20601170726qf4c8097sde16fa1f87ed8b2c@mail.gmail.com>
Message-ID: <aac2c7cb0601170848x44a16de8tecb5d566a371571e@mail.gmail.com>

On 1/17/06, Guido van Rossum <guido at python.org> wrote:
> On 1/17/06, Jim Jewett <jimjjewett at gmail.com> wrote:
> > Guido wrote:
> >
> > > more important to implement __index__() in Python 2.5.
> > > This behaves like __int__() for integral types, but is not
> > > defined for float or Decimal.
> >
> > Why not for Decimal, or even float?  I would not be surprised
> > if 10.798 failed, but I would expect 1000D to work.
> >
> > If indexing worked with more arbitrary extension integers,
> > then I would expect it work with Decimal, and possibly float,
> > when the number == a whole number, and to raise ValueError
> > otherwise.
>
> Sorry, I forgot to explain this. There's a very good reason why this
> should not be allowed.
>
> You don't know if the 1000D was the result of an exact or of an
> approximate calculation. It could be a pure coincidence that it's
> 1000D instead of 999.9D; and we certainly don't want the latter to be
> allowed for indexing. Floating point (including Decimal floating
> point) calculations are fraught with uncertainties about the precision
> / accuracy of the outcome. Even if when you test your algorithm you
> always get exact outcomes, there's no guarantee that with real input
> data the same will hold.
>
> Requiring the index to be an int by *type* solves this by forcing you
> to think about rounding/truncating. I talked to some Numeric folks and
> they understand and agree.

IMO this is better done by replacing float with an interval type, or
atleast float context objects with explicit rounding and precision
control.  The former would tell you when the result is inexact (the
gap or "length" is non-zero), the latter would let you pick rounding
behavior when you know enough math to back it up.

And rather than leave you all hanging I'll say that I believe it could
be done by using LLVM (circumventing C entierly.)
http://rhamphoryncus.blogspot.com/2006/01/floating-point-and-nans-in-python.html

(Why have float at all if the documentation essentially says it
channels /dev/urandom?)

--
Adam Olsen, aka Rhamphoryncus

From rhamph at gmail.com  Tue Jan 17 18:04:35 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 10:04:35 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <43CCA927.8030600@v.loewis.de>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<43CCA927.8030600@v.loewis.de>
Message-ID: <aac2c7cb0601170904x45cac571y6d76ea8e941f1bb9@mail.gmail.com>

On 1/17/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>   class Color:
>     msg = {'en':['red', 'green', 'blue'], 'de':['rot','gr?n','blau']}
>     def __str__(self, language='en'):
>       return self.msg[language][self.value]
>
>   red = Color(0)
>
> so you could say
>
>   print str(red, 'de')
>
> I don't think I like this direction.

I agree that *args makes the code non-obvious.  However, if **kwargs
is used instead:

  def str(obj, **kwargs):
    return obj.__str__(**kwargs)

  class Color:
    msg = {'en':['red', 'green', 'blue'], 'de':['rot','gr?n','blau']}
    def __str__(self, language='en'):
      return self.msg[language][self.value]

  red = Color(0)

  print str(red, language='de')

I find that quite readable.

--
Adam Olsen, aka Rhamphoryncus

From aahz at pythoncraft.com  Tue Jan 17 18:04:47 2006
From: aahz at pythoncraft.com (Aahz)
Date: Tue, 17 Jan 2006 09:04:47 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
Message-ID: <20060117170447.GA5411@panix.com>

On Mon, Jan 16, 2006, Alex Martelli wrote:
>
> Is it finally time in Python 2.5 to allow the "obvious" use of, say,  
> str(5,2) to give '101', just the converse of the way int('101',1)  
> gives 5?  I'm not sure why str has never allowed this obvious use --  
> any bright beginner assumes it's there and it's awkward to explain  
> why it's not!-).  I'll be happy to propose a patch if the BDFL  
> blesses this, but I don't even think it's worth a PEP... it's an  
> inexplicable though long-standing omission (given the argumentative  
> nature of this crowd I know I'll get pushback, but I still hope the  
> BDFL can Pronounce about it anyway;-).

-1

I agree with all the other comments about the functional asymmetry
between int() and str() in the Python universe, and that therefore str()
shouldn't necessarily mimic int()'s API.  Propose some other mechanism;
I so far haven't seen a good reasons to prefer any of the ones already
proposed.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From theller at python.net  Tue Jan 17 19:17:59 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 17 Jan 2006 19:17:59 +0100
Subject: [Python-Dev] Building on OS X 10.4 fails
Message-ID: <r7761zxk.fsf@python.net>

Building the readline on OS X 10.4 fails, is this known, or am I doing
something wrong?

Thanks,

Thomas

building 'readline' extension
gcc -fno-strict-aliasing -Wno-long-double -no-cpp-precomp -mno-fused-madd -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I/Users/theller/svn/trunk/./Include -I/Users/theller/svn/trunk/./Mac/Include -I./Include -I. -I/Users/theller/svn/trunk/Include -I/Users/theller/svn/trunk -c /Users/theller/svn/trunk/Modules/readline.c -o build/temp.darwin-8.4.0-Power_Macintosh-2.5/readline.o
/Users/theller/svn/trunk/Modules/readline.c: In function 'write_history_file':
/Users/theller/svn/trunk/Modules/readline.c:112: warning: implicit declaration of function 'history_truncate_file'
/Users/theller/svn/trunk/Modules/readline.c: In function 'py_remove_history':
/Users/theller/svn/trunk/Modules/readline.c:301: warning: implicit declaration of function 'remove_history'
/Users/theller/svn/trunk/Modules/readline.c:301: warning: assignment makes pointer from integer without a cast
/Users/theller/svn/trunk/Modules/readline.c:310: warning: passing argument 1 of 'free' discards qualifiers from pointer target type
/Users/theller/svn/trunk/Modules/readline.c:312: warning: passing argument 1 of 'free' discards qualifiers from pointer target type
/Users/theller/svn/trunk/Modules/readline.c: In function 'py_replace_history':
/Users/theller/svn/trunk/Modules/readline.c:338: warning: implicit declaration of function 'replace_history_entry'
/Users/theller/svn/trunk/Modules/readline.c:338: warning: assignment makes pointer from integer without a cast
/Users/theller/svn/trunk/Modules/readline.c:347: warning: passing argument 1 of 'free' discards qualifiers from pointer target type
/Users/theller/svn/trunk/Modules/readline.c:349: warning: passing argument 1 of 'free' discards qualifiers from pointer target type
/Users/theller/svn/trunk/Modules/readline.c: In function 'get_current_history_length':
/Users/theller/svn/trunk/Modules/readline.c:453: error: 'HISTORY_STATE' undeclared (first use in this function)
/Users/theller/svn/trunk/Modules/readline.c:453: error: (Each undeclared identifier is reported only once
/Users/theller/svn/trunk/Modules/readline.c:453: error: for each function it appears in.)
/Users/theller/svn/trunk/Modules/readline.c:453: error: 'hist_st' undeclared (first use in this function)
/Users/theller/svn/trunk/Modules/readline.c:455: warning: implicit declaration of function 'history_get_history_state'
/Users/theller/svn/trunk/Modules/readline.c: In function 'insert_text':
/Users/theller/svn/trunk/Modules/readline.c:503: warning: implicit declaration of function 'rl_insert_text'
/Users/theller/svn/trunk/Modules/readline.c: In function 'on_completion':
/Users/theller/svn/trunk/Modules/readline.c:637: error: 'rl_attempted_completion_over' undeclared (first use in this function)
/Users/theller/svn/trunk/Modules/readline.c: In function 'flex_complete':
/Users/theller/svn/trunk/Modules/readline.c:675: warning: passing argument 2 of 'completion_matches' from incompatible pointer type
/Users/theller/svn/trunk/Modules/readline.c: In function 'setup_readline':
/Users/theller/svn/trunk/Modules/readline.c:700: warning: passing argument 2 of 'rl_bind_key_in_map' from incompatible pointer type
/Users/theller/svn/trunk/Modules/readline.c:701: warning: passing argument 2 of 'rl_bind_key_in_map' from incompatible pointer type
/Users/theller/svn/trunk/Modules/readline.c: In function 'readline_until_enter_or_signal':
/Users/theller/svn/trunk/Modules/readline.c:758: warning: passing argument 2 of 'rl_callback_handler_install' from incompatible pointer type
/Users/theller/svn/trunk/Modules/readline.c:788: warning: implicit declaration of function 'rl_free_line_state'
/Users/theller/svn/trunk/Modules/readline.c:789: warning: implicit declaration of function 'rl_cleanup_after_signal'
/Users/theller/svn/trunk/Modules/readline.c: In function 'call_readline':
/Users/theller/svn/trunk/Modules/readline.c:883: error: 'HISTORY_STATE' undeclared (first use in this function)
/Users/theller/svn/trunk/Modules/readline.c:883: error: 'state' undeclared (first use in this function)
/Users/theller/svn/trunk/Modules/readline.c:885: warning: assignment discards qualifiers from pointer target type



From g.brandl-nospam at gmx.net  Tue Jan 17 19:35:18 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Tue, 17 Jan 2006 19:35:18 +0100
Subject: [Python-Dev] Python icon
In-Reply-To: <17354.20722.560136.55434@montanaro.dyndns.org>
References: <dqc0u2$ul0$1@sea.gmane.org>	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
	<17354.20722.560136.55434@montanaro.dyndns.org>
Message-ID: <dqjdd6$ro2$1@sea.gmane.org>

skip at pobox.com wrote:
>     >> does Python have an official icon?
> 
>     Ping> i found some images at http://www.pythonology.com/logos...
> 
> It appears the yin/yang Python's on that page are being used in the new site
> (beta.python.org).  I don't know if that makes it official or not though.

Interesting, didn't even know a new page was in the making...
Do you know who is responsible for the new page?

regards,
Georg


From skip at pobox.com  Tue Jan 17 19:45:58 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 17 Jan 2006 12:45:58 -0600
Subject: [Python-Dev] Python icon
In-Reply-To: <dqjdd6$ro2$1@sea.gmane.org>
References: <dqc0u2$ul0$1@sea.gmane.org>
	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
	<17354.20722.560136.55434@montanaro.dyndns.org>
	<dqjdd6$ro2$1@sea.gmane.org>
Message-ID: <17357.15206.570762.861782@montanaro.dyndns.org>


    Georg> Interesting, didn't even know a new page was in the making...  Do
    Georg> you know who is responsible for the new page?

Tim Parkin is heading things up.  Look here:

    http://beta.python.org/

Skip

From arigo at tunes.org  Tue Jan 17 19:09:05 2006
From: arigo at tunes.org (Armin Rigo)
Date: Tue, 17 Jan 2006 19:09:05 +0100
Subject: [Python-Dev] str with base
Message-ID: <20060117180905.GA15740@code1.codespeak.net>

Hi Alex,

> Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> str(5,2) to give '101', just the converse of the way int('101',1)
> gives 5?

+1.  That's obvious enough for me.  Clearly it should only work with int
and long arguments, just like int(x,base) only works if x is a str or
unicode argument (and not something that can be converted to a string,
or even a stringish type for some definition of stringish).


Armin

From bob at redivi.com  Tue Jan 17 20:17:38 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 11:17:38 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <r7761zxk.fsf@python.net>
References: <r7761zxk.fsf@python.net>
Message-ID: <3B20C03B-93A2-4C24-9D6C-67ADBD10A801@redivi.com>


On Jan 17, 2006, at 10:17 AM, Thomas Heller wrote:

> Building the readline on OS X 10.4 fails, is this known, or am I doing
> something wrong?

Mac OS X doesn't ship with readline.  It ships with BSD libedit  
symlinked to readline.  Not good enough for Python.  You need a third  
party copy.

I made an egg for it a while ago: http://python.org/pypi/readline

-bob


From barry at python.org  Tue Jan 17 20:19:47 2006
From: barry at python.org (Barry Warsaw)
Date: Tue, 17 Jan 2006 14:19:47 -0500
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <r7761zxk.fsf@python.net>
References: <r7761zxk.fsf@python.net>
Message-ID: <1137525587.9528.19.camel@geddy.wooz.org>

On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote:
> Building the readline on OS X 10.4 fails, is this known, or am I doing
> something wrong?

There are definitely serious issues with readline on OS X 10.4.  I've
hit them too but haven't had time to post about it yet.  I'm far from an
expert on them (Bob, Jack and others can definitely speak with more
knowledge on this) but my understanding is that the default readline
that comes with Tiger is not sufficient to build the Python extension
library.

However, I use DarwinPorts and IWBNI Python's build process could
search /opt/local for the necessary readline support, using that if
found.  I know that DP's Python 2.4.2 build does provide readline, so
that should definitely work.  I've glanced at the setup.py file, but
haven't had the time to dig in in enough detail to actually fix and
commit.

(There was something else that bugged me about the OS X 10.4 Python
from-source build but now I can't remember.)

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060117/01f4dc80/attachment.pgp 

From theller at python.net  Tue Jan 17 20:32:59 2006
From: theller at python.net (Thomas Heller)
Date: Tue, 17 Jan 2006 20:32:59 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <psms7b34.fsf@python.net>
	<ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>
	<43CC17FB.5020904@v.loewis.de>
Message-ID: <fynmvedw.fsf@python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Guido van Rossum wrote:
>> On 1/16/06, Thomas Heller <theller at python.net> wrote:
>> 
>>>It looks like we need a pronouncement now.
>> 
>> 
>> Sorry. It appeared to me that there was general agreement to using a
>> strongly worded warning in the docs, so I tuned out of the rest of the
>> discussion. So for the record, I'm fine with that.
>
> Ok. If Thomas checks in the code and the documentation, I'll do the
> Windows/setup part of it.
Great.

Now, I'm trying to build ctypes on OS X as the first non-windows
platform.  As I said, ctypes currently has its own source copy of libffi
(since there are no separate releases of libffi, and the only platform
that comes with libffi by default that I know is cygwin).

It is built into a static non-shared library with a call to configure
like this:

   cd <build_dir>
   <src_dir>/configure --disable-shared --enable-static \
     --enable-multilib=no --prefix=<inst_dir>
   make install

then libffi.a is linked into the _ctypes extension module.

I know near to nothing about 'configure', can/should a call to os.system
with the commands above be added to setup.py, or how could this be
approached?

Thomas


From brett at python.org  Tue Jan 17 21:47:53 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 17 Jan 2006 12:47:53 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <1137525587.9528.19.camel@geddy.wooz.org>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
Message-ID: <bbaeab100601171247p4db232act6a6d9c08bd45ebdd@mail.gmail.com>

On 1/17/06, Barry Warsaw <barry at python.org> wrote:
> On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote:
> > Building the readline on OS X 10.4 fails, is this known, or am I doing
> > something wrong?
>
> There are definitely serious issues with readline on OS X 10.4.  I've
> hit them too but haven't had time to post about it yet.  I'm far from an
> expert on them (Bob, Jack and others can definitely speak with more
> knowledge on this) but my understanding is that the default readline
> that comes with Tiger is not sufficient to build the Python extension
> library.
>
> However, I use DarwinPorts and IWBNI Python's build process could
> search /opt/local for the necessary readline support, using that if
> found.  I know that DP's Python 2.4.2 build does provide readline, so
> that should definitely work.  I've glanced at the setup.py file, but
> haven't had the time to dig in in enough detail to actually fix and
> commit.
>

OS X-specific package support has been removed from setup.py for the
more generic support of LDFLAGS and CPPFLAGS.  Set those properly for
/opt/local/(lib|include) and it will find readline installed by DP
fine (it's how I build my checkout with readline support).

-Brett

From aleaxit at gmail.com  Tue Jan 17 23:30:34 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Tue, 17 Jan 2006 14:30:34 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
Message-ID: <e8a0972d0601171430o7696ae49sf1e1c2501c81b8a2@mail.gmail.com>

On 1/16/06, Guido van Rossum <guido at python.org> wrote:
> On 1/16/06, Alex Martelli <aleaxit at gmail.com> wrote:
> > Is it finally time in Python 2.5 to allow the "obvious" use of, say,
> > str(5,2) to give '101', just the converse of the way int('101',1)
> [I'm sure you meant int('101', 2) here]

Yep.

> > gives 5?  I'm not sure why str has never allowed this obvious use --
> > any bright beginner assumes it's there and it's awkward to explain
> > why it's not!-).  I'll be happy to propose a patch if the BDFL
> > blesses this, but I don't even think it's worth a PEP... it's an
> > inexplicable though long-standing omission (given the argumentative
> > nature of this crowd I know I'll get pushback, but I still hope the
> > BDFL can Pronounce about it anyway;-).
>
> I wish you had an argument better than "every bright beginner assumes
> it exists". :-)

What about "it should obviously be there"?-)


> But (unlike for some other things that bright beginners might assume)
> I don't think there's a deep reason why this couldn't exist.
>
> The only reasons I can come up with is "because input and output are
> notoriously asymmetric in Python" and "because nobody submitted a
> patch". :-)

OK, so, should I just submit a patch?


> There are some corner cases to consider though.
>
> - Should repr() support the same convention? I think not.
> - Should str(3.1, n) be allowed? I think not.
> - Should str(x, n) call x.__str__(n)? Neither.
> - Should bases other than 2..36 be considered? I don't think so.

Agreed on all scores.

> Unfortunately this doesn't obsolete oct() and hex() -- oct(10) returns
> '012', while str(10, 8) soulc return '12'; hex(10) returns '0xa' while
> str(10, 16) would return 'a'.

Sure.  hex(x) is like '0x'+str(x,16) and oct(x) is like '0'+str(x,8)
but hex and oct are minutely more concise.


> I do think that before we add this end-user nicety, it's more
> important to implement __index__() in Python 2.5. This behaves like

More important, sure, but also proportionally more work, so I don't
see the two issues as "competing" against each other.

> __int__() for integral types, but is not defined for float or Decimal.
> Operations that intrinsically require an integral argument, like
> indexing and slicing, should call __index__() on their argument rather
> than __int__(), so as to support non-built-in integral arguments while
> still complaining about float arguments.

Hear, hear.  Multiplication of sequences, too.

> This is currently implemented
> by explicitly checking for float in a few places, which I find
> repulsing. __index__() won't be requested by bright beginners, but it

You might be surprised by just how bright some (Python) beginners with
deep maths background can be, though they might prefer to spell it
differently
(__int_and_i_really_mean_it__ for example'-) because it may not be
obvious (if they're not Dutch) that multiplying a sequence has to do
with 'indexing';-).

> is important e.g. to the Numeric Python folks, who like to implement
> their own integral types but are suffering from that their integers
> aren't usable everywhere.

As the author and maintainer of gmpy I entirely agree -- I never liked
the fact that instances of gmpy.mpq are "second class citizens" and i
need to plaster int(...) around them to use them as list indices or to
multiply sequences (I vaguely mentioned having in mind a 'baseinteger'
check, but a special method returning the integer value, such as the
__index__ you're talking about, is obviously better).


Alex

From aleaxit at gmail.com  Tue Jan 17 23:53:51 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Tue, 17 Jan 2006 14:53:51 -0800
Subject: [Python-Dev] basenumber redux
In-Reply-To: <43CCC0AE.5070800@egenix.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
	<43CC17B9.3060200@v.loewis.de>
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
	<43CCC0AE.5070800@egenix.com>
Message-ID: <e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>

On 1/17/06, M.-A. Lemburg <mal at egenix.com> wrote:
> Alex, I think you're missing a point here: what you are looking
> for is an interface, not a base class - simply because the

I expect numbers to support arithmetic operators, &c -- no need for
basenumber to "spell this out", i.e., "be an itnerface".

> If you look at the Python C API, you'll find that "a number"
> is actually never tested.

There being no way to generically test for "a number", that's unsurprising.

> The tests always ask for either
> integers or floats.

But this doesn't apply to the Python Standard Library, for example see
line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):".

> The addition of a basenumber base class won't make these any
> simpler.

Being able to change imaplib to use basenumber instead of (int, float)
won't make it SIMPLER, but it will surely make it BETTER -- why should
a long be rejected, or a Decimal,
for that matter?  Similarly, on line 1352 it should use the existing
basestring, though it now uses str (this function IS weird -- if it
finds date_time to be of an unknown TYPE it raises a *ValueError*
rather than a *TypeError* -- ah well).


Alex

From guido at python.org  Tue Jan 17 23:54:31 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jan 2006 14:54:31 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <e8a0972d0601171430o7696ae49sf1e1c2501c81b8a2@mail.gmail.com>
References: <AA8D468D-FF1A-4CE5-AEB9-31C9DF992120@gmail.com>
	<ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<e8a0972d0601171430o7696ae49sf1e1c2501c81b8a2@mail.gmail.com>
Message-ID: <ca471dc20601171454j224bcf4fqe473b52cf79c55f4@mail.gmail.com>

On 1/17/06, Alex Martelli <aleaxit at gmail.com> wrote:
> OK, so, should I just submit a patch?

Hmm, there are quite a few people who strongly dislike the particular
API you're proposing. The problem is, bright newbies might be led to
wanting str(i, base) as an analogy to int(s, base) only because they
see str() and int() as each other's opposites, not having seen other
uses of either (especially str()).

Given the amount of disagreement on this issue and my own lackluster
interest I don't want to pronounce str(i, base) to be the right
solution. Sorry!

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Jan 18 00:21:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Jan 2006 00:21:13 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>	
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>	
	<43CC17B9.3060200@v.loewis.de>	
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>	
	<43CCC0AE.5070800@egenix.com>
	<e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
Message-ID: <43CD7BE9.6060705@v.loewis.de>

Alex Martelli wrote:
> But this doesn't apply to the Python Standard Library, for example see
> line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):".
[...]
> Being able to change imaplib to use basenumber instead of (int, float)
> won't make it SIMPLER, but it will surely make it BETTER -- why should
> a long be rejected, or a Decimal, for that matter? 

Right. I think this function should read

  if isinstance(date_time, str) and \
     (date_time[0],date_time[-1]) == ('"','"'):
        return date_time        # Assume in correct format


  if isinstance(date_time, (tuple, time.struct_time)):
     tt = date_time
  else:
     tt = time.localtime(date_time)

If this is something that time.localtime can't handle, it will
give a TypeError. This is much better than

    raise ValueError("date_time not of a known type")
    # (why does it raise a ValueError if it says "type"?)

Regards,
Martin

From thomas at xs4all.net  Wed Jan 18 00:28:43 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 18 Jan 2006 00:28:43 +0100
Subject: [Python-Dev] str with base
In-Reply-To: <bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
Message-ID: <20060117232843.GV18916@xs4all.nl>

On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote:

> I think a method 5664400.to_base(13) sounds nice.
[And others suggested int-methods too]

I would like to point out that this is almost, but not quite, entirely as
inapropriate as using str(). Integers don't have a base. String
representations of integers -- and indeed, numbers in general, as the Python
tutorial explains in Appendix B -- have a base. Adding such a method to
integers (and, I presume, longs) would beg the question why floats, Decimals
and complex numbers don't have them.

In-favour-of-%2b-ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From rhamph at gmail.com  Wed Jan 18 00:29:35 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 16:29:35 -0700
Subject: [Python-Dev] basenumber redux
In-Reply-To: <e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
	<43CC17B9.3060200@v.loewis.de>
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
	<43CCC0AE.5070800@egenix.com>
	<e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
Message-ID: <aac2c7cb0601171529kdc66a32wb96d1174b4833231@mail.gmail.com>

On 1/17/06, Alex Martelli <aleaxit at gmail.com> wrote:
> Being able to change imaplib to use basenumber instead of (int, float)
> won't make it SIMPLER, but it will surely make it BETTER -- why should
> a long be rejected, or a Decimal,
> for that matter?

Because there's no guarantee that they'll produce correct results? 
All number types are approximations of true numbers, and they all
behave wrong in creative ways.

For example:

def xrange(start, stop, step):
  i = start
  while i < stop:
    yield i
    i += step

This works fine so long as you only give it int as input, and has no
maximum value.

>>> for i in xrange(2**53, 2**53+3, 1): print i
...
9007199254740992
9007199254740993
9007199254740994

Float inputs also work so long as you don't get large enough to
provoke rounding.  However, as soon as you do...

>>> for i in xrange(2**53, 2**53+3, 1.0): print i
...
9007199254740992
9.00719925474e+15
9.00719925474e+15
9.00719925474e+15
9.00719925474e+15
<snip>
9.00719925474e+15
9.00719925474e+15
974e+15
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
KeyboardInterrupt

The function fails.  Floating point, despite being a "number" and
supporting the "number interface", does not behave close enough to
what the programmer desires to work for all values.  There might be a
way to handle floats specially that a mathematician may understand,
but the only way I know of is to convert to int at the start of the
function.

def xrange(start, stop, step):
  start, stop, step = int(start), int(stop), int(step)
  i = start
  while i < stop:
    yield i
    i += step

>>> for i in xrange(2**53, 2**53+3, 1.0): print i
...
9007199254740992
9007199254740993
9007199254740994

That works so long as the floats are all integral values. 
Unfortunately a non-integral value will get truncated silently.  An
explicit check for equality after the conversion would have to be
added, or Guido's __index__ could be used, but __index__ seems
misnamed for this usage.

Another approach would be changing operations involving floats to
return intervals instead.  The high end of the interval would continue
to count up when rounding is provoked, and would raise an exception
when the i < stop is executed (due to being ambiguous).  Once float
uses intervals you could state that all number types are expected to
use intervals in the face of inexactness (and those who don't behave
as expected would have unexpected results.)

--
Adam Olsen, aka Rhamphoryncus

From rhamph at gmail.com  Wed Jan 18 00:38:30 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 16:38:30 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <20060117232843.GV18916@xs4all.nl>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
Message-ID: <aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>

On 1/17/06, Thomas Wouters <thomas at xs4all.net> wrote:
> On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote:
>
> > I think a method 5664400.to_base(13) sounds nice.
> [And others suggested int-methods too]
>
> I would like to point out that this is almost, but not quite, entirely as
> inapropriate as using str(). Integers don't have a base. String
> representations of integers -- and indeed, numbers in general, as the Python
> tutorial explains in Appendix B -- have a base. Adding such a method to
> integers (and, I presume, longs) would beg the question why floats, Decimals
> and complex numbers don't have them.

I dream of a day when str(3.25, base=2) == '11.01'.  That is the
number a float really represents.  It would be so much easier to
understand why floats behave the way they do if it were possible to
print them in binary.

To be fair, it's not str(x, base=n) I'm after here (although it seems
like a clean way to do it.)  Rather, I just want SOME way of printing
ints and floats in binary.


> In-favour-of-%2b-ly y'rs,

My only opposition to this is that the byte type may want to use it. 
I'd rather wait until byte is fully defined, implemented, and released
in a python version before that option is taken away.

--
Adam Olsen, aka Rhamphoryncus

From guido at python.org  Wed Jan 18 01:02:43 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jan 2006 16:02:43 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
Message-ID: <ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>

On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
> > In-favour-of-%2b-ly y'rs,
>
> My only opposition to this is that the byte type may want to use it.
> I'd rather wait until byte is fully defined, implemented, and released
> in a python version before that option is taken away.

Has this been proposed? What would %b print?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From rhamph at gmail.com  Wed Jan 18 01:09:41 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 17:09:41 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
Message-ID: <aac2c7cb0601171609v73db49bay28ee0c2fcdd4a5a8@mail.gmail.com>

On 1/17/06, Guido van Rossum <guido at python.org> wrote:
> On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
> > > In-favour-of-%2b-ly y'rs,
> >
> > My only opposition to this is that the byte type may want to use it.
> > I'd rather wait until byte is fully defined, implemented, and released
> > in a python version before that option is taken away.
>
> Has this been proposed? What would %b print?

I don't believe it's been proposed and I don't know what it'd print. 
Perhaps it indicates the bytes should be passed through without
conversion.

In any case I only advocate waiting until it's clear that bytes have
no need for it before we use it for binary conversions.

--
Adam Olsen, aka Rhamphoryncus

From anthony at interlink.com.au  Wed Jan 18 01:17:32 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 18 Jan 2006 11:17:32 +1100
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <1137525587.9528.19.camel@geddy.wooz.org>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
Message-ID: <200601181117.34549.anthony@interlink.com.au>

On Wednesday 18 January 2006 06:19, Barry Warsaw wrote:
> On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote:
> > Building the readline on OS X 10.4 fails, is this known, or am I
> > doing something wrong?
>
> There are definitely serious issues with readline on OS X 10.4. 
> I've hit them too but haven't had time to post about it yet.  I'm
> far from an expert on them (Bob, Jack and others can definitely
> speak with more knowledge on this) but my understanding is that the
> default readline that comes with Tiger is not sufficient to build
> the Python extension library.

It sounds like configure needs to grow a test to detect that a 
"libreadline" it finds is actually the crackful "libedit" and refuse 
to use it if so.

Anthony

From bob at redivi.com  Wed Jan 18 02:45:07 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 17:45:07 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <aac2c7cb0601171609v73db49bay28ee0c2fcdd4a5a8@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<aac2c7cb0601171609v73db49bay28ee0c2fcdd4a5a8@mail.gmail.com>
Message-ID: <98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com>


On Jan 17, 2006, at 4:09 PM, Adam Olsen wrote:

> On 1/17/06, Guido van Rossum <guido at python.org> wrote:
>> On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
>>>> In-favour-of-%2b-ly y'rs,
>>>
>>> My only opposition to this is that the byte type may want to use it.
>>> I'd rather wait until byte is fully defined, implemented, and  
>>> released
>>> in a python version before that option is taken away.
>>
>> Has this been proposed? What would %b print?
>
> I don't believe it's been proposed and I don't know what it'd print.
> Perhaps it indicates the bytes should be passed through without
> conversion.

That doesn't make any sense.  What is "without conversion"?  Does  
that mean UTF-8, UCS-2, UCS-4, latin-1, Shift-JIS?  You can't have  
unicode without some kind of conversion.

> In any case I only advocate waiting until it's clear that bytes have
> no need for it before we use it for binary conversions.

I don't see what business a byte type has mingling with string  
formatters other than the normal str and repr coercions via %s and %r  
respectively.

-bob


From jack at performancedrivers.com  Wed Jan 18 02:01:27 2006
From: jack at performancedrivers.com (Jack Diederich)
Date: Tue, 17 Jan 2006 20:01:27 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
Message-ID: <20060118010127.GB6152@performancedrivers.com>

On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote:
> On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
> > > In-favour-of-%2b-ly y'rs,
> >
> > My only opposition to this is that the byte type may want to use it.
> > I'd rather wait until byte is fully defined, implemented, and released
> > in a python version before that option is taken away.
> 
> Has this been proposed? What would %b print?
> 
It was proposed in this or another thread about the same in the last few
days (gmane search doesn't like the % in '%b').

The suggestion is to add 'b' as a sprintf-like format string
  %[<base>][.<pad>]b

Where the optional <base> is the base to print in and <pad> is the optional
minimum length of chars to print (as I recall).  Default is base 2.

Me?  I like it.

-Jack

From bob at redivi.com  Wed Jan 18 02:56:48 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 17:56:48 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
Message-ID: <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com>


On Jan 17, 2006, at 3:38 PM, Adam Olsen wrote:

> On 1/17/06, Thomas Wouters <thomas at xs4all.net> wrote:
>> On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote:
>>
>>> I think a method 5664400.to_base(13) sounds nice.
>> [And others suggested int-methods too]
>>
>> I would like to point out that this is almost, but not quite,  
>> entirely as
>> inapropriate as using str(). Integers don't have a base. String
>> representations of integers -- and indeed, numbers in general, as  
>> the Python
>> tutorial explains in Appendix B -- have a base. Adding such a  
>> method to
>> integers (and, I presume, longs) would beg the question why  
>> floats, Decimals
>> and complex numbers don't have them.
>
> I dream of a day when str(3.25, base=2) == '11.01'.  That is the
> number a float really represents.  It would be so much easier to
> understand why floats behave the way they do if it were possible to
> print them in binary.

Actually if you wanted something that closely represents what a  
floating point number is then you would want to see this::

	>>> str(3.25, base=2)
	'1.101e1'
	>>> str(0.25, base=2)
	'1.0e-10'

Printing the bits without an exponent is nearly as misleading as  
printing them in decimal.

-bob


From bob at redivi.com  Wed Jan 18 02:59:01 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 17:59:01 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <200601181117.34549.anthony@interlink.com.au>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
Message-ID: <A3C7BD63-8DDE-463E-9C4D-D01F5A946735@redivi.com>


On Jan 17, 2006, at 4:17 PM, Anthony Baxter wrote:

> On Wednesday 18 January 2006 06:19, Barry Warsaw wrote:
>> On Tue, 2006-01-17 at 19:17 +0100, Thomas Heller wrote:
>>> Building the readline on OS X 10.4 fails, is this known, or am I
>>> doing something wrong?
>>
>> There are definitely serious issues with readline on OS X 10.4.
>> I've hit them too but haven't had time to post about it yet.  I'm
>> far from an expert on them (Bob, Jack and others can definitely
>> speak with more knowledge on this) but my understanding is that the
>> default readline that comes with Tiger is not sufficient to build
>> the Python extension library.
>
> It sounds like configure needs to grow a test to detect that a
> "libreadline" it finds is actually the crackful "libedit" and refuse
> to use it if so.

Unless something has changed since 2.4.1, that is the current  
behavior.  Thomas' title should've probably been "Building readline  
on OS X 10.4 fails", rather than "Building on OS X 10.4 fails" (which  
reads as "Building Python on OS X 10.4 fails" given the list).

-bob


From bob at redivi.com  Wed Jan 18 03:11:36 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 18:11:36 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <20060118010127.GB6152@performancedrivers.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
Message-ID: <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>


On Jan 17, 2006, at 5:01 PM, Jack Diederich wrote:

> On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote:
>> On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
>>>> In-favour-of-%2b-ly y'rs,
>>>
>>> My only opposition to this is that the byte type may want to use it.
>>> I'd rather wait until byte is fully defined, implemented, and  
>>> released
>>> in a python version before that option is taken away.
>>
>> Has this been proposed? What would %b print?
>>
> It was proposed in this or another thread about the same in the  
> last few
> days (gmane search doesn't like the % in '%b').
>
> The suggestion is to add 'b' as a sprintf-like format string
>   %[<base>][.<pad>]b
>
> Where the optional <base> is the base to print in and <pad> is the  
> optional
> minimum length of chars to print (as I recall).  Default is base 2.
>
> Me?  I like it.

Personally I would prefer the "b" format code to behave similarly to  
"o", "d", and "d", except for binary instead of octal, decimal, and  
hexadecimal.  Something that needs to account for three factors (zero  
pad, space pad, base) should probably be a function (maybe a  
builtin).  Hell, maybe it could take a fourth argument to specify how  
a negative number should be printed (e.g. a number of bits to use for  
the 2's complement).

However... if %b were to represent arbitrary bases, I think that's  
backwards.  It should be %[<pad>][.<base>]b, which would do this:

	>>> '%08b %08o %08d %08x' % 12
	'00001100 00000014 00000012 0000000C'

Where your suggestion would have this behavior (or something close to  
it):

	>>> '%08b %08o %08d %08x' % 12
	'14 00000014 00000012 0000000C'

-bob


From rhamph at gmail.com  Wed Jan 18 03:17:27 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 19:17:27 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com>
Message-ID: <aac2c7cb0601171817n7c675deat6a93cc65b1814ed@mail.gmail.com>

On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
>
> On Jan 17, 2006, at 3:38 PM, Adam Olsen wrote:
>
> > I dream of a day when str(3.25, base=2) == '11.01'.  That is the
> > number a float really represents.  It would be so much easier to
> > understand why floats behave the way they do if it were possible to
> > print them in binary.
>
> Actually if you wanted something that closely represents what a
> floating point number is then you would want to see this::
>
>         >>> str(3.25, base=2)
>         '1.101e1'
>         >>> str(0.25, base=2)
>         '1.0e-10'
>
> Printing the bits without an exponent is nearly as misleading as
> printing them in decimal.

I disagree.  The exponent is involved in rounding to fit in compact
storage but once that is complete the value can be represented exactly
without it.

--
Adam Olsen, aka Rhamphoryncus

From rhamph at gmail.com  Wed Jan 18 03:26:53 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Tue, 17 Jan 2006 19:26:53 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<aac2c7cb0601171609v73db49bay28ee0c2fcdd4a5a8@mail.gmail.com>
	<98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com>
Message-ID: <aac2c7cb0601171826t1050b5c1kdbf9fe0693e1d0e6@mail.gmail.com>

On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
>
> On Jan 17, 2006, at 4:09 PM, Adam Olsen wrote:
>
> > On 1/17/06, Guido van Rossum <guido at python.org> wrote:
> >> On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
> >>>> In-favour-of-%2b-ly y'rs,
> >>>
> >>> My only opposition to this is that the byte type may want to use it.
> >>> I'd rather wait until byte is fully defined, implemented, and
> >>> released
> >>> in a python version before that option is taken away.
> >>
> >> Has this been proposed? What would %b print?
> >
> > I don't believe it's been proposed and I don't know what it'd print.
> > Perhaps it indicates the bytes should be passed through without
> > conversion.
>
> That doesn't make any sense.  What is "without conversion"?  Does
> that mean UTF-8, UCS-2, UCS-4, latin-1, Shift-JIS?  You can't have
> unicode without some kind of conversion.
>
> > In any case I only advocate waiting until it's clear that bytes have
> > no need for it before we use it for binary conversions.
>
> I don't see what business a byte type has mingling with string
> formatters other than the normal str and repr coercions via %s and %r
> respectively.

Is the byte type intended to be involved in string formatters at all? 
Does byte("%i") % 3 have the obvious effect, or is it an error?

Although upon further consideration I don't see any case where %s and
%b would have different effects.. *shrug* I never said it did have a
purpose, just that it *might* be given a purpose when byte was spec'd
out.

--
Adam Olsen, aka Rhamphoryncus

From jack at performancedrivers.com  Wed Jan 18 04:12:38 2006
From: jack at performancedrivers.com (Jack Diederich)
Date: Tue, 17 Jan 2006 22:12:38 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
Message-ID: <20060118031238.GC6152@performancedrivers.com>

On Tue, Jan 17, 2006 at 06:11:36PM -0800, Bob Ippolito wrote:
> 
> On Jan 17, 2006, at 5:01 PM, Jack Diederich wrote:
> 
> >On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote:
> >>On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
> >>>>In-favour-of-%2b-ly y'rs,
> >>>
> >>>My only opposition to this is that the byte type may want to use it.
> >>>I'd rather wait until byte is fully defined, implemented, and  
> >>>released
> >>>in a python version before that option is taken away.
> >>
> >>Has this been proposed? What would %b print?
> >>
> >It was proposed in this or another thread about the same in the  
> >last few
> >days (gmane search doesn't like the % in '%b').
> >
> >The suggestion is to add 'b' as a sprintf-like format string
> >  %[<base>][.<pad>]b
> >
> >Where the optional <base> is the base to print in and <pad> is the  
> >optional
> >minimum length of chars to print (as I recall).  Default is base 2.
> >
> >Me?  I like it.
> 
> Personally I would prefer the "b" format code to behave similarly to  
> "o", "d", and "d", except for binary instead of octal, decimal, and  
> hexadecimal.  Something that needs to account for three factors (zero  
> pad, space pad, base) should probably be a function (maybe a  
> builtin).  Hell, maybe it could take a fourth argument to specify how  
> a negative number should be printed (e.g. a number of bits to use for  
> the 2's complement).
> 
> However... if %b were to represent arbitrary bases, I think that's  
> backwards.  It should be %[<pad>][.<base>]b, which would do this:
> 
> 	>>> '%08b %08o %08d %08x' % 12
> 	'00001100 00000014 00000012 0000000C'
> 

Were I BDFAD (not to be confused with BD-FOAD) I'd add %b, %B and, binary()
to match %x, %X, and hex().  The arbitrary base case isn't even academic
or we would see homework questions about it on c.l.py.  No one asks about
hex or octal because they are there.  No one asks about base seven
formatting because everyone knows numerologists prefer Perl.

-Jack

nb, that's "For A Day."


From bob at redivi.com  Wed Jan 18 04:30:10 2006
From: bob at redivi.com (Bob Ippolito)
Date: Tue, 17 Jan 2006 19:30:10 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <20060118031238.GC6152@performancedrivers.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
Message-ID: <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>


On Jan 17, 2006, at 7:12 PM, Jack Diederich wrote:

> On Tue, Jan 17, 2006 at 06:11:36PM -0800, Bob Ippolito wrote:
>>
>> On Jan 17, 2006, at 5:01 PM, Jack Diederich wrote:
>>
>>> On Tue, Jan 17, 2006 at 04:02:43PM -0800, Guido van Rossum wrote:
>>>> On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
>>>>>> In-favour-of-%2b-ly y'rs,
>>>>>
>>>>> My only opposition to this is that the byte type may want to  
>>>>> use it.
>>>>> I'd rather wait until byte is fully defined, implemented, and
>>>>> released
>>>>> in a python version before that option is taken away.
>>>>
>>>> Has this been proposed? What would %b print?
>>>>
>>> It was proposed in this or another thread about the same in the
>>> last few
>>> days (gmane search doesn't like the % in '%b').
>>>
>>> The suggestion is to add 'b' as a sprintf-like format string
>>>  %[<base>][.<pad>]b
>>>
>>> Where the optional <base> is the base to print in and <pad> is the
>>> optional
>>> minimum length of chars to print (as I recall).  Default is base 2.
>>>
>>> Me?  I like it.
>>
>> Personally I would prefer the "b" format code to behave similarly to
>> "o", "d", and "d", except for binary instead of octal, decimal, and
>> hexadecimal.  Something that needs to account for three factors (zero
>> pad, space pad, base) should probably be a function (maybe a
>> builtin).  Hell, maybe it could take a fourth argument to specify how
>> a negative number should be printed (e.g. a number of bits to use for
>> the 2's complement).
>>
>> However... if %b were to represent arbitrary bases, I think that's
>> backwards.  It should be %[<pad>][.<base>]b, which would do this:
>>
>> 	>>> '%08b %08o %08d %08x' % 12
>> 	'00001100 00000014 00000012 0000000C'
>>
>
> Were I BDFAD (not to be confused with BD-FOAD) I'd add %b, %B and,  
> binary()
> to match %x, %X, and hex().  The arbitrary base case isn't even  
> academic
> or we would see homework questions about it on c.l.py.  No one asks  
> about
> hex or octal because they are there.  No one asks about base seven
> formatting because everyone knows numerologists prefer Perl.

There shouldn't be a %B for the same reason there isn't an %O or %D  
-- they're all just digits, so there's not a need for an uppercase  
variant.

The difference between hex() and oct() and the proposed binary() is  
that hex() and oct() return valid Python expressions in that base.   
In order for it to make sense, Python would need to grow some syntax.

If Python were to have syntax for binary literals, I'd propose a  
trailing b: "1100b".  It would be convenient at times to represent  
bit flags, but I'm not sure it's worth the syntax change.

binarydigit ::= ("0" | "1")
binaryinteger ::= binarydigit+ "b"
integer ::= decimalinteger | octinteger | hexinteger | binaryinteger

-bob


From simon at arrowtheory.com  Wed Jan 18 04:56:12 2006
From: simon at arrowtheory.com (Simon Burton)
Date: Wed, 18 Jan 2006 14:56:12 +1100
Subject: [Python-Dev] computed goto's in ceval loop
Message-ID: <20060118145612.2d4009ab.simon@arrowtheory.com>



I have been experimenting with replacing the big switch in ceval.c by
a computed goto construct [1]. It uses #define's to make it optional.
This work was inspired by Mono's MINT interpreter code,
and Neil Norwitz's attempt to use a function pointer table [2].

Result: about 1% slower on the pystone benchmark.

It also seems to have broken the interpreter; at least one test fails (test_StringIO). Whoops.

Not sure if the other switch-speedup hacks (eg. PREDICT(op)) conflict 
at all with this patch (ie. make it slower than it could be).

Mono uses a much larger opcode set, with 2-byte opcodes, that includes
type info in each opcode. This means that the actual case statements are much faster.

My initial thought about using computed goto's (January 2003) was that the python opcode cases
were much fatter than mono's (often involving a function call) so that the overhead of
branching on opcodes would be insignificant. It seems that this is true indeed.

The patch id is 1408710.

[1] http://developer.apple.com/documentation/DeveloperTools/gcc-4.0.1/gcc/Labels-as-Values.html
[2] http://mail.python.org/pipermail/python-dev/2003-February/033705.html

Simon.



-- 
Simon Burton, B.Sc.
Licensed PO Box 8066
ANU Canberra 2601
Australia
Ph. 61 02 6249 6940
http://arrowtheory.com 

From guido at python.org  Wed Jan 18 05:25:49 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 17 Jan 2006 20:25:49 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
Message-ID: <ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>

On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
> There shouldn't be a %B for the same reason there isn't an %O or %D
> -- they're all just digits, so there's not a need for an uppercase
> variant.

Right.

> The difference between hex() and oct() and the proposed binary() is

I'd propose bin() to stay in line with the short abbreviated names.

> that hex() and oct() return valid Python expressions in that base.
> In order for it to make sense, Python would need to grow some syntax.

Fair enough. So let's define it.

> If Python were to have syntax for binary literals, I'd propose a
> trailing b: "1100b".  It would be convenient at times to represent
> bit flags, but I'm not sure it's worth the syntax change.

Typically, suffixes are used to indicated *types*: 12L, 12j, and even
12e0 in some sense.

The binary type should have a 0b prefix.

Perhaps this could be implemented at the PyCon sprint?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From brett at python.org  Wed Jan 18 05:53:38 2006
From: brett at python.org (Brett Cannon)
Date: Tue, 17 Jan 2006 20:53:38 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
Message-ID: <bbaeab100601172053u28fa4d13gb53b1823232d8d1c@mail.gmail.com>

On 1/17/06, Guido van Rossum <guido at python.org> wrote:
> On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
> > There shouldn't be a %B for the same reason there isn't an %O or %D
> > -- they're all just digits, so there's not a need for an uppercase
> > variant.
>
> Right.
>
> > The difference between hex() and oct() and the proposed binary() is
>
> I'd propose bin() to stay in line with the short abbreviated names.
>
> > that hex() and oct() return valid Python expressions in that base.
> > In order for it to make sense, Python would need to grow some syntax.
>
> Fair enough. So let's define it.
>
> > If Python were to have syntax for binary literals, I'd propose a
> > trailing b: "1100b".  It would be convenient at times to represent
> > bit flags, but I'm not sure it's worth the syntax change.
>
> Typically, suffixes are used to indicated *types*: 12L, 12j, and even
> 12e0 in some sense.
>
> The binary type should have a 0b prefix.
>

0b101 for 5?

> Perhaps this could be implemented at the PyCon sprint?
>

Added to the wiki along with possibly hashing out the bytes type.

-Brett

From stephen at xemacs.org  Wed Jan 18 06:25:03 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 18 Jan 2006 14:25:03 +0900
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <200601181117.34549.anthony@interlink.com.au> (Anthony Baxter's
	message of "Wed, 18 Jan 2006 11:17:32 +1100")
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
Message-ID: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Anthony" == Anthony Baxter <anthony at interlink.com.au> writes:

    Anthony> It sounds like configure needs to grow a test to detect
    Anthony> that a "libreadline" it finds is actually the crackful
    Anthony> "libedit" and refuse to use it if so.

FYI: Real libreadline is GPL, and rms made a point of forcing
(Aladdin-licensed) Ghostscript to remove stanzas from the Makefile
that allowed linking to it as a user option.  Ie, this particular pain
in the neck is deliberate FSF policy, to encourage use of the GPL.

Unless rms has changed his position on this, or there has been
relevant legislation or a court decision in the meantime, explicitly
requiring or checking for "real" libreadline, even as a user option,
risks rms's wrath.  (Of course, Python could change its license to
GPL, which would undoubtedly flood Cambridge with tears of joy<wink>).

As long as the link to fake libreadline succeeds and the resulting
program works identically to one linked to real libreadline, he has no
complaint.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From nnorwitz at gmail.com  Wed Jan 18 07:08:06 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Tue, 17 Jan 2006 22:08:06 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
Message-ID: <ee2a432c0601172208j76af8ba1h2e9feb7b230af6b6@mail.gmail.com>

On 1/17/06, Guido van Rossum <guido at python.org> wrote:
>
> > The difference between hex() and oct() and the proposed binary() is
>
> I'd propose bin() to stay in line with the short abbreviated names.

Are these features used enough to have 3 builtins?
Would format(number, base) suffice?

  format(5, base=2) == '101'
  format(5, base=8) == '5'
  format(5, base=8, prefix=True) == '05'
  format(5, base=16) == '5'
  format(5, base=16, prefix=True) == '0x5'

Or something like that.  Then there can be symmetry with int()
(arbitrary bases) and we get rid of 2 other builtins eventually.  Not
sure if there are/should be uses other than number formating.

> > that hex() and oct() return valid Python expressions in that base.
> > In order for it to make sense, Python would need to grow some syntax.
>
> Fair enough. So let's define it.
>
> > If Python were to have syntax for binary literals, I'd propose a
> > trailing b: "1100b".  It would be convenient at times to represent
> > bit flags, but I'm not sure it's worth the syntax change.
>
> Typically, suffixes are used to indicated *types*: 12L, 12j, and even
> 12e0 in some sense.
>
> The binary type should have a 0b prefix.

-0.  Is this common enough to add (even in 3k)?  For the instances I
could have used this, it would have been completely impractical since
the hex strings were generally over 80 characters.

n

From andrew-pythondev at puzzling.org  Wed Jan 18 07:22:27 2006
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed, 18 Jan 2006 17:22:27 +1100
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
References: <2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
Message-ID: <20060118062227.GA28517@home.puzzling.org>

Guido van Rossum wrote:
[...]
> 
> I'd propose bin() to stay in line with the short abbreviated names.
> 
[...]
> 
> The binary type should have a 0b prefix.

It seems odd to me to add both a builtin *and* new syntax for something that's
occasionally handy, but only occasionally.  If we're going to clutter a module
with this function, why not e.g. the math module instead of builtins?  I thought
the consensus was that we had too many builtins already.

Similarly, the need for 0b101 syntax seems pretty low to me when you can
already do int("101", 2).

-Andrew.


From gisle at ActiveState.com  Wed Jan 18 08:35:26 2006
From: gisle at ActiveState.com (Gisle Aas)
Date: 17 Jan 2006 23:35:26 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <20060118031238.GC6152@performancedrivers.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
Message-ID: <lrhd82c7k1.fsf@caliper.activestate.com>

Jack Diederich <jack at performancedrivers.com> writes:

> > However... if %b were to represent arbitrary bases, I think that's  
> > backwards.  It should be %[<pad>][.<base>]b, which would do this:
> > 
> > 	>>> '%08b %08o %08d %08x' % 12
> > 	'00001100 00000014 00000012 0000000C'
> > 
> 
> Were I BDFAD (not to be confused with BD-FOAD) I'd add %b, %B and, binary()
> to match %x, %X, and hex().  The arbitrary base case isn't even academic
> or we would see homework questions about it on c.l.py.  No one asks about
> hex or octal because they are there.  No one asks about base seven
> formatting because everyone knows numerologists prefer Perl.

BTW, Perl already do binary literals and %b formatting so there is
some precedence for it:

  $ perl -e '$a = 0b1100; printf "%08b %08o %08d %08x\n", $a, $a, $a, $a'
  00001100 00000014 00000012 0000000c  

--Gisle

From steve at holdenweb.com  Wed Jan 18 09:31:18 2006
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Jan 2006 08:31:18 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <aac2c7cb0601171826t1050b5c1kdbf9fe0693e1d0e6@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>	<20060117100544.GC1419@craig-wood.com>	<1137498898.4342.68.camel@warna.dub.corp.google.com>	<2mek37oyh7.fsf@starship.python.net>	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>	<20060117232843.GV18916@xs4all.nl>	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>	<aac2c7cb0601171609v73db49bay28ee0c2fcdd4a5a8@mail.gmail.com>	<98102A23-B77F-4D39-B7E8-CE4893CCF291@redivi.com>
	<aac2c7cb0601171826t1050b5c1kdbf9fe0693e1d0e6@mail.gmail.com>
Message-ID: <dqkuch$g7t$3@sea.gmane.org>

Adam Olsen wrote:
> On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
> 
>>On Jan 17, 2006, at 4:09 PM, Adam Olsen wrote:
>>
>>
>>>On 1/17/06, Guido van Rossum <guido at python.org> wrote:
>>>
>>>>On 1/17/06, Adam Olsen <rhamph at gmail.com> wrote:
>>>>
>>>>>>In-favour-of-%2b-ly y'rs,
>>>>>
>>>>>My only opposition to this is that the byte type may want to use it.
>>>>>I'd rather wait until byte is fully defined, implemented, and
>>>>>released
>>>>>in a python version before that option is taken away.
>>>>
>>>>Has this been proposed? What would %b print?
>>>
>>>I don't believe it's been proposed and I don't know what it'd print.
>>>Perhaps it indicates the bytes should be passed through without
>>>conversion.
>>
>>That doesn't make any sense.  What is "without conversion"?  Does
>>that mean UTF-8, UCS-2, UCS-4, latin-1, Shift-JIS?  You can't have
>>unicode without some kind of conversion.
>>
>>
>>>In any case I only advocate waiting until it's clear that bytes have
>>>no need for it before we use it for binary conversions.
>>
>>I don't see what business a byte type has mingling with string
>>formatters other than the normal str and repr coercions via %s and %r
>>respectively.
> 
> 
> Is the byte type intended to be involved in string formatters at all? 
> Does byte("%i") % 3 have the obvious effect, or is it an error?
> 
> Although upon further consideration I don't see any case where %s and
> %b would have different effects.. *shrug* I never said it did have a
> purpose, just that it *might* be given a purpose when byte was spec'd
> out.
> 
I suppose we'd better reserve "%q" for 'quirky types we just invented', 
too? ;-)

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From steve at holdenweb.com  Wed Jan 18 09:41:09 2006
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Jan 2006 08:41:09 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <aac2c7cb0601171817n7c675deat6a93cc65b1814ed@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>	<20060117100544.GC1419@craig-wood.com>	<1137498898.4342.68.camel@warna.dub.corp.google.com>	<2mek37oyh7.fsf@starship.python.net>	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>	<20060117232843.GV18916@xs4all.nl>	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>	<1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com>
	<aac2c7cb0601171817n7c675deat6a93cc65b1814ed@mail.gmail.com>
Message-ID: <dqkuv0$joi$1@sea.gmane.org>

Adam Olsen wrote:
> On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
> 
>>On Jan 17, 2006, at 3:38 PM, Adam Olsen wrote:
>>
>>
>>>I dream of a day when str(3.25, base=2) == '11.01'.  That is the
>>>number a float really represents.  It would be so much easier to
>>>understand why floats behave the way they do if it were possible to
>>>print them in binary.
>>
>>Actually if you wanted something that closely represents what a
>>floating point number is then you would want to see this::
>>
>>        >>> str(3.25, base=2)
>>        '1.101e1'
>>        >>> str(0.25, base=2)
>>        '1.0e-10'
>>
>>Printing the bits without an exponent is nearly as misleading as
>>printing them in decimal.
> 
> 
> I disagree.  The exponent is involved in rounding to fit in compact
> storage but once that is complete the value can be represented exactly
> without it.
> 
Albeit with excessively long representations for the larger values one 
sometimes sees represented in float form.

Personally I wouldn't even be interested in seeing 
1.3407807929942597e+154 written in fixed point form *in decimal*, let 
alone in binary where the representation, though unambiguous, would have 
over 500 bits, most of them zeros.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From thomas at xs4all.net  Wed Jan 18 10:17:24 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 18 Jan 2006 10:17:24 +0100
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <20060118091724.GW18916@xs4all.nl>

On Wed, Jan 18, 2006 at 02:25:03PM +0900, Stephen J. Turnbull wrote:

>     Anthony> It sounds like configure needs to grow a test to detect
>     Anthony> that a "libreadline" it finds is actually the crackful
>     Anthony> "libedit" and refuse to use it if so.

> FYI: Real libreadline is GPL, and rms made a point of forcing
> (Aladdin-licensed) Ghostscript to remove stanzas from the Makefile
> that allowed linking to it as a user option.  Ie, this particular pain
> in the neck is deliberate FSF policy, to encourage use of the GPL.

[...]

> As long as the link to fake libreadline succeeds and the resulting
> program works identically to one linked to real libreadline, he has no
> complaint.

I don't think this applies to Python. The Aladdin license isn't
GPL-compatible, but the current PSF license is (according to rms himself.)
(Only, IIRC, 1.5.2-and-earlier, 2.0.1 and 2.1.1-and-later, not 1.6, 1.6.1[*],
2.0 or 2.1.) The Ghostscript check-for-readline is a case of "you are still
linking with readline, even when you aren't actually linking" -- but this
isn't a problem for (most versions of) Python, because rms said it isnt. As
long as the resulting Python binary is only covered by the GPL-compatible
PSF license, the GPL and no GPL-incompatible licenses, any form of linking
is fine, even configure-time-not-building linking.

[*] 1.6.1 is the release that contained the license change rms and his
laywers wanted, but that itself doesn't make the license GPL-compatible. It
apparently allows derived works from being GPL-compatible, though. 2.0 and
2.1 derive from 1.6, but 2.0.1 and 2.1.1 derive from 1.6.1.

I'm sure it makes sense to someone.

Go-figure'ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From martin at v.loewis.de  Wed Jan 18 10:19:58 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Jan 2006 10:19:58 +0100
Subject: [Python-Dev] subwcrev.exe
Message-ID: <43CE083E.5030205@v.loewis.de>

I just added svnversion support to the VC project files;
subwcrev.exe is expected to be found in the Tortoise installation,
if it cannot find tortoise in the registry, it falls back to
not doing svnversion.

If you find problems, please let me know.

Regards,
Martin

From steve at holdenweb.com  Wed Jan 18 10:28:34 2006
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Jan 2006 09:28:34 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <dqkuv0$joi$1@sea.gmane.org>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>	<20060117100544.GC1419@craig-wood.com>	<1137498898.4342.68.camel@warna.dub.corp.google.com>	<2mek37oyh7.fsf@starship.python.net>	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>	<20060117232843.GV18916@xs4all.nl>	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>	<1A1FD1BA-E80E-4F18-9EE2-CD73FF41B2DA@redivi.com>	<aac2c7cb0601171817n7c675deat6a93cc65b1814ed@mail.gmail.com>
	<dqkuv0$joi$1@sea.gmane.org>
Message-ID: <43CE0A42.3060608@holdenweb.com>

Steve Holden wrote:
[...]
> Personally I wouldn't even be interested in seeing 
> 1.3407807929942597e+154 written in fixed point form *in decimal*, let 
> alone in binary where the representation, though unambiguous, would have 
> over 500 bits, most of them zeros.
> 
Well, shot myself in the foot there of course, since the number I meant 
was actually 2.0 ** 512 (or 
13407807929942597099574024998205846127479365820592393377723561443721764030073546
976801874298166903427690031858186486050853753882811946569946433649006084096.0) 
  rather than the decimal approximation above. But I'm sure you get the 
point that fixed-point representations aren't always appropriate.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From anthony at interlink.com.au  Wed Jan 18 10:31:49 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed, 18 Jan 2006 20:31:49 +1100
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <r7761zxk.fsf@python.net>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200601182031.54111.anthony@interlink.com.au>

On Wednesday 18 January 2006 16:25, Stephen J. Turnbull wrote:
> Unless rms has changed his position on this, or there has been
> relevant legislation or a court decision in the meantime,
> explicitly requiring or checking for "real" libreadline, even as a
> user option, risks rms's wrath.  (Of course, Python could change
> its license to GPL, which would undoubtedly flood Cambridge with
> tears of joy<wink>).

Python's license is GPL-compatible, so this isn't an issue.

> As long as the link to fake libreadline succeeds and the resulting
> program works identically to one linked to real libreadline, he has
> no complaint.

My "complaint" is that libedit _calls_ itself libreadline, when it's 
pretty clear that it's not actually a drop-in replacement (or the 
readline module would build). Hence my use of the word "crackful" 
<wink>

Anthony
-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From mal at egenix.com  Wed Jan 18 11:03:38 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Wed, 18 Jan 2006 11:03:38 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>	
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>	
	<43CC17B9.3060200@v.loewis.de>	
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>	
	<43CCC0AE.5070800@egenix.com>
	<e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
Message-ID: <43CE127A.2050700@egenix.com>

Alex Martelli wrote:
> On 1/17/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> Alex, I think you're missing a point here: what you are looking
>> for is an interface, not a base class - simply because the
> 
> I expect numbers to support arithmetic operators, &c -- no need for
> basenumber to "spell this out", i.e., "be an itnerface".

If at all, basenumber would be an abstract class. However,
unlike for basestring, the interface (which methods it
supports, including operator methods) would not be well-
defined.

>> If you look at the Python C API, you'll find that "a number"
>> is actually never tested.
>
> There being no way to generically test for "a number", that's unsurprising.

Hmm, I lost you there. If it's unsurprising that there's
no check for "a number", then why would you want a
basenumber ?

>> The tests always ask for either
>> integers or floats.
> 
> But this doesn't apply to the Python Standard Library, for example see
> line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):".

Why not use the functions I added to my previous mail ?

>> The addition of a basenumber base class won't make these any
>> simpler.
> 
> Being able to change imaplib to use basenumber instead of (int, float)
> won't make it SIMPLER, but it will surely make it BETTER -- why should
> a long be rejected, or a Decimal,
> for that matter?  Similarly, on line 1352 it should use the existing
> basestring, though it now uses str (this function IS weird -- if it
> finds date_time to be of an unknown TYPE it raises a *ValueError*
> rather than a *TypeError* -- ah well).

Again, why not use floatnumber() instead, which takes care
of all the details behind finding out whether an object
should be considered a number and even converts it to
a float for you ?

Why try to introduce a low-level feature when a higher
level solution is readily available and more usable.

You will rarely really care for the type of an object
if all you're interested in is the float value of an
object (or the integer value).

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 18 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From abo at minkirri.apana.org.au  Wed Jan 18 12:48:11 2006
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Wed, 18 Jan 2006 11:48:11 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
References: <ca471dc20601162004y668204dbk1035c61b8d7ab9a9@mail.gmail.com>
	<002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117100544.GC1419@craig-wood.com>
	<1137498898.4342.68.camel@warna.dub.corp.google.com>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
Message-ID: <1137584891.16309.19.camel@warna.dub.corp.google.com>

On Tue, 2006-01-17 at 16:38 -0700, Adam Olsen wrote:
> On 1/17/06, Thomas Wouters <thomas at xs4all.net> wrote:
> > On Tue, Jan 17, 2006 at 09:23:29AM -0500, Jason Orendorff wrote:
[...]
> I dream of a day when str(3.25, base=2) == '11.01'.  That is the
> number a float really represents.  It would be so much easier to
> understand why floats behave the way they do if it were possible to
> print them in binary.
[...]

Heh... that's pretty much why I used base16 float notation when doing
fixed point stuff in assembler... uses less digits than binary, but
easily visualised as bits.

However, I do think that we could go overboard here... I don't know that
we really need arbitrary base string formatting for all numeric types. I
think this is a case of "very little gained for too much added
complexity".

If we really do, and someone is prepared to implement it, then I think
adding "@base" is the best way to do it (see my half joking post
earlier). 

If we only want arbitrary bases for integer types, the best way would be
to leverage off the existing ".precision" so that it means ".base" for
"%d".


> > In-favour-of-%2b-ly y'rs,
> 
> My only opposition to this is that the byte type may want to use it. 
> I'd rather wait until byte is fully defined, implemented, and released
> in a python version before that option is taken away.

There's always "B" for bytes and "b" for bits... though I can't imagine
why byte would need it's own conversion type.

I'm not entirely sure everyone is on the same page for "%b" here... it
would only be a shorthand for "binary" in the same way that "%x" is for
"hexidecimal". It would not support arbitrary bases, and thus "%2b"
would mean a binary string with minimum length of 2 characters.

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From abo at minkirri.apana.org.au  Wed Jan 18 13:16:20 2006
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Wed, 18 Jan 2006 12:16:20 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
Message-ID: <1137586580.16309.28.camel@warna.dub.corp.google.com>

On Tue, 2006-01-17 at 20:25 -0800, Guido van Rossum wrote:
> On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
> > There shouldn't be a %B for the same reason there isn't an %O or %D
> > -- they're all just digits, so there's not a need for an uppercase
[...]

so %b is "binary",

+1

> > The difference between hex() and oct() and the proposed binary() is
> 
> I'd propose bin() to stay in line with the short abbreviated names.
[...]

+1

> The binary type should have a 0b prefix.
[...]

+1

For those who argue "who would ever use it?", I would :-) 

Note that this does not support and is independent of supporting
arbitrary bases. I don't think we need to support arbitrary bases, but
if we did I would vote for ".precision" to mean ".base" for "%d"... ie;

"%3.3d" % 5 == " 12"

I think supporting arbitrary bases for floats is way overkill and not
worth considering.

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From stephen at xemacs.org  Wed Jan 18 14:24:52 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Wed, 18 Jan 2006 22:24:52 +0900
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <200601182031.54111.anthony@interlink.com.au> (Anthony Baxter's
	message of "Wed, 18 Jan 2006 20:31:49 +1100")
References: <r7761zxk.fsf@python.net>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<200601182031.54111.anthony@interlink.com.au>
Message-ID: <871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Anthony" == Anthony Baxter <anthony at interlink.com.au> writes:

    Anthony> Python's license is GPL-compatible, so this isn't an
    Anthony> issue.

I'm sorry, but you seem to misunderstand what "GPL compatibility"
means.  It is a _one-way_ street.  A license is GPL-compatible if its
terms permit the code it covers to be redistributed under GPL.  The
GPL, however, is "compatible" in this sense only with itself: _the
whole of any work_ derived from a GPL work must be licensed under the
GPL.  Not under a GPL-compatible license, under the GPL.

The import of the Ghostscript case is that the FSF considers a
Makefile stanza clearly intended to cause linkage to a GPL library,
even if optional and supplied by the user, to create a work derived
from that library.  A "GNU readline"-enabled Python is derived from
GNU readline, and must be distributed under the GPL or not distributed
at all.  I assume that the former is not acceptable to Python, and the
latter is clearly undesirable.

I assure you, Peter Deutsch was just as unbelieving as you are<wink>,
but facing a potentially expensive lawsuit, he caved in.  I can think
of many reasons why the FSF might not come after Python, but the risk
is real.  Please ask a lawyer.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From ncoghlan at gmail.com  Wed Jan 18 15:50:57 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 19 Jan 2006 00:50:57 +1000
Subject: [Python-Dev] Python icon
In-Reply-To: <17357.15206.570762.861782@montanaro.dyndns.org>
References: <dqc0u2$ul0$1@sea.gmane.org>	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>	<17354.20722.560136.55434@montanaro.dyndns.org>	<dqjdd6$ro2$1@sea.gmane.org>
	<17357.15206.570762.861782@montanaro.dyndns.org>
Message-ID: <43CE55D1.5030107@gmail.com>

skip at pobox.com wrote:
>     Georg> Interesting, didn't even know a new page was in the making...  Do
>     Georg> you know who is responsible for the new page?
> 
> Tim Parkin is heading things up.  Look here:
> 
>     http://beta.python.org/

I like the look of the new page, but it took a bit of digging to find anything 
about *how* it has been developed, or what needs to be done before the main 
URL can be switched to the new page (e.g. integrating the documentation 
properly). While it's in beta having that info prominent on the home page 
might be a good idea. Something like:

  "This is a beta of the redesigned www.python.org. If you have specific 
suggestions or display problems, submit them to the tracker <here>. General 
questions or comments should be posted to comp.lang.python"

I thought for a moment that this week's python-url was going to help me out, 
but it only linked the beta site without actually linking to any of the 
discussion it referenced in the relevant blurb. . .

Appropriate enlightenment was eventually attained through Google:
http://mail.python.org/pipermail/python-list/2006-January/319862.html

Cheers,
Nick.


-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Wed Jan 18 16:21:49 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 18 Jan 2006 10:21:49 -0500
Subject: [Python-Dev] [Python-checkins] r42090 - in python/trunk:
	Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj
	PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj
In-Reply-To: <20060118091354.A31B91E4002@bag.python.org>
References: <20060118091354.A31B91E4002@bag.python.org>
Message-ID: <1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com>

[martin.v.loewis <python-checkins at python.org>]
> Date: Wed Jan 18 10:13:51 2006
> New Revision: 42090
>
> Added:
>    python/trunk/PCbuild/make_buildinfo.vcproj
> Modified:
>    python/trunk/Modules/getbuildinfo.c
>    python/trunk/PCbuild/pcbuild.sln
>    python/trunk/PCbuild/pythoncore.vcproj
> Log:
> Generate getbuildinfo.o each time the linker is invoked;
> try to generate SVNVERSION information if subwcrev.exe
> can be found.

Umm ... looks like I don't know how to build Python anymore.  A batch
build crapped out with:

"""
------ Build started: Project: pythoncore, Configuration: Debug Win32 ------

generate buildinfo
'make_buildinfo.exe' is not recognized as an internal or external command,
operable program or batch file.
"""

Adding the `make_buildinfo` project to the batch-build set craps out
earlier:  the `make_buildinfo` project lists make_buildinfo.c as an
input file, but there is no such file:

"""
------ Build started: Project: make_buildinfo, Configuration: Debug Win32 ------

Compiling...
make_buildinfo.c
c1 : fatal error C1083: Cannot open source file: '.\make_buildinfo.c':
No such file or directory
"""

Did this checkin perhaps forget to add that file?

From aahz at pythoncraft.com  Wed Jan 18 16:23:56 2006
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 18 Jan 2006 07:23:56 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
References: <2mek37oyh7.fsf@starship.python.net>
	<bb8868b90601170623ybc69e6en6dd1c3312a83e8b4@mail.gmail.com>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
Message-ID: <20060118152356.GA20772@panix.com>

On Tue, Jan 17, 2006, Guido van Rossum wrote:
> On 1/17/06, Bob Ippolito <bob at redivi.com> wrote:
>> 
>> The difference between hex() and oct() and the proposed binary() is
> 
> I'd propose bin() to stay in line with the short abbreviated names.

There has been some previous discussion about removing hex()/oct() from
builtins for Python 3.0, IIRC.  I sure don't think bin() belongs there.

> The binary type should have a 0b prefix.

-0 on adding a new prefix; +1 on this syntax if we do.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From steve at holdenweb.com  Wed Jan 18 16:37:20 2006
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Jan 2006 15:37:20 +0000
Subject: [Python-Dev] Python icon
In-Reply-To: <43CE55D1.5030107@gmail.com>
References: <dqc0u2$ul0$1@sea.gmane.org>	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>	<17354.20722.560136.55434@montanaro.dyndns.org>	<dqjdd6$ro2$1@sea.gmane.org>	<17357.15206.570762.861782@montanaro.dyndns.org>
	<43CE55D1.5030107@gmail.com>
Message-ID: <43CE60B0.1070806@holdenweb.com>

Nick Coghlan wrote:
> skip at pobox.com wrote:
> 
>>    Georg> Interesting, didn't even know a new page was in the making...  Do
>>    Georg> you know who is responsible for the new page?
>>
>>Tim Parkin is heading things up.  Look here:
>>
>>    http://beta.python.org/
> 
> 
> I like the look of the new page, but it took a bit of digging to find anything 
> about *how* it has been developed, or what needs to be done before the main 
> URL can be switched to the new page (e.g. integrating the documentation 
> properly). While it's in beta having that info prominent on the home page 
> might be a good idea. Something like:
> 
>   "This is a beta of the redesigned www.python.org. If you have specific 
> suggestions or display problems, submit them to the tracker <here>. General 
> questions or comments should be posted to comp.lang.python"
> 
> I thought for a moment that this week's python-url was going to help me out, 
> but it only linked the beta site without actually linking to any of the 
> discussion it referenced in the relevant blurb. . .
> 
> Appropriate enlightenment was eventually attained through Google:
> http://mail.python.org/pipermail/python-list/2006-January/319862.html
> 
That's probably not a bad idea. We should also add script explaining how 
to download the beta site data and the generation software so people can 
play with it and get ready to be webmasters :-)

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From tim at pollenation.net  Wed Jan 18 16:47:42 2006
From: tim at pollenation.net (Tim Parkin)
Date: Wed, 18 Jan 2006 15:47:42 +0000
Subject: [Python-Dev] Python icon
In-Reply-To: <43CE60B0.1070806@holdenweb.com>
References: <dqc0u2$ul0$1@sea.gmane.org>	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>	<17354.20722.560136.55434@montanaro.dyndns.org>	<dqjdd6$ro2$1@sea.gmane.org>	<17357.15206.570762.861782@montanaro.dyndns.org>	<43CE55D1.5030107@gmail.com>
	<43CE60B0.1070806@holdenweb.com>
Message-ID: <43CE631E.1080708@pollenation.net>

Steve Holden wrote:

>Nick Coghlan wrote:
>  
>
>>skip at pobox.com wrote:
>>
>>    
>>
>>>   Georg> Interesting, didn't even know a new page was in the making...  Do
>>>   Georg> you know who is responsible for the new page?
>>>
>>>Tim Parkin is heading things up.  Look here:
>>>
>>>   http://beta.python.org/
>>>      
>>>
>>I like the look of the new page, but it took a bit of digging to find anything 
>>about *how* it has been developed, or what needs to be done before the main 
>>URL can be switched to the new page (e.g. integrating the documentation 
>>properly). While it's in beta having that info prominent on the home page 
>>might be a good idea. Something like:
>>
>>  "This is a beta of the redesigned www.python.org. If you have specific 
>>suggestions or display problems, submit them to the tracker <here>. General 
>>questions or comments should be posted to comp.lang.python"
>>
>>I thought for a moment that this week's python-url was going to help me out, 
>>but it only linked the beta site without actually linking to any of the 
>>discussion it referenced in the relevant blurb. . .
>>
>>Appropriate enlightenment was eventually attained through Google:
>>http://mail.python.org/pipermail/python-list/2006-January/319862.html
>>
>>    
>>
>That's probably not a bad idea. We should also add script explaining how 
>to download the beta site data and the generation software so people can 
>play with it and get ready to be webmasters :-)
>
>  
>
I'll try to change the home page content to have a comment about how the
site is in beta and what to do to get the content and build software.
I've also got to add a fix in on the current css.. If anybody needs the
graphics, they can still be found on psf.pollenation.net in the
resources folder (browser source > trunk > ...).

Tim


From raymond.hettinger at verizon.net  Wed Jan 18 17:00:03 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed, 18 Jan 2006 11:00:03 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <20060118152356.GA20772@panix.com>
Message-ID: <001501c61c48$3ed669a0$a3f9a244@oemcomputer>

> > I'd propose bin() to stay in line with the short abbreviated names.
> 
> There has been some previous discussion about removing hex()/oct()
from
> builtins for Python 3.0, IIRC.  I sure don't think bin() belongs
there.

Perhaps introduce a single function, base(val, radix=10, prefix=''), as
a universal base converter that could replace bin(), hex(), oct(), etc.

That would give us fewer builtins and provide an inverse for all the
int() conversions (i.e. arbitrary bases).  Also, it would allow an
unprefixed output which is what I usually need.


Raymond


From skip at pobox.com  Wed Jan 18 17:07:57 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 18 Jan 2006 10:07:57 -0600
Subject: [Python-Dev] str with base
In-Reply-To: <001501c61c48$3ed669a0$a3f9a244@oemcomputer>
References: <20060118152356.GA20772@panix.com>
	<001501c61c48$3ed669a0$a3f9a244@oemcomputer>
Message-ID: <17358.26589.750736.623827@montanaro.dyndns.org>


    Raymond> Perhaps introduce a single function, base(val, radix=10,
    Raymond> prefix=''), as a universal base converter that could replace
    Raymond> bin(), hex(), oct(), etc.

Would it (should it) work with floats, decimals, complexes?  I presume it
would work with ints and longs.

Skip

From jason.orendorff at gmail.com  Wed Jan 18 17:39:38 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 18 Jan 2006 11:39:38 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <1137586580.16309.28.camel@warna.dub.corp.google.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>
	<20060117232843.GV18916@xs4all.nl>
	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>
	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>
	<20060118010127.GB6152@performancedrivers.com>
	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>
	<20060118031238.GC6152@performancedrivers.com>
	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>
	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>
	<1137586580.16309.28.camel@warna.dub.corp.google.com>
Message-ID: <bb8868b90601180839r56691cbfj7caf6224e6402799@mail.gmail.com>

On 1/18/06, Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> I think supporting arbitrary bases for floats is way overkill and not
> worth considering.

If you mean actual base-3 floating-point arithmetic, I agree.  That's
outlandish.

But if there were a stdlib function to format floats losslessly in hex
or binary, Tim Peters would use it at least once every six weeks to
illustrate the finer points of floating point arithmetic. <0.00390625
wink>

+1.0

-j

From steven.bethard at gmail.com  Wed Jan 18 17:55:49 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 18 Jan 2006 09:55:49 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <17358.26589.750736.623827@montanaro.dyndns.org>
References: <20060118152356.GA20772@panix.com>
	<001501c61c48$3ed669a0$a3f9a244@oemcomputer>
	<17358.26589.750736.623827@montanaro.dyndns.org>
Message-ID: <d11dcfba0601180855q7863e1b7p865eec3466031497@mail.gmail.com>

[Raymond]
> Perhaps introduce a single function, base(val, radix=10,
> prefix=''), as a universal base converter that could replace
> bin(), hex(), oct(), etc.

+1 on introducing base()

[Skip]
> Would it (should it) work with floats, decimals, complexes?  I presume it
> would work with ints and longs.

While support for floats, decimals, etc. might be nice (and I
certainly wouldn't complain if someone wanted to supply the patch) I
don't think those features should be necessary for base()'s initial
introduction.  If they're there, great, but if not, I don't think that
should hold up the patch...

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From guido at python.org  Wed Jan 18 18:52:59 2006
From: guido at python.org (Guido van Rossum)
Date: Wed, 18 Jan 2006 09:52:59 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>

On 1/17/06, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> >>>>> "Anthony" == Anthony Baxter <anthony at interlink.com.au> writes:
>
>     Anthony> It sounds like configure needs to grow a test to detect
>     Anthony> that a "libreadline" it finds is actually the crackful
>     Anthony> "libedit" and refuse to use it if so.
>
> FYI: Real libreadline is GPL, and rms made a point of forcing
> (Aladdin-licensed) Ghostscript to remove stanzas from the Makefile
> that allowed linking to it as a user option.  Ie, this particular pain
> in the neck is deliberate FSF policy, to encourage use of the GPL.

Can we just all agree that RMS is an asshole now? Bah.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From steve at holdenweb.com  Wed Jan 18 18:59:25 2006
From: steve at holdenweb.com (Steve Holden)
Date: Wed, 18 Jan 2006 17:59:25 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <bb8868b90601180839r56691cbfj7caf6224e6402799@mail.gmail.com>
References: <002f01c61b1c$5fa63480$e221cb97@oemcomputer>	<20060117232843.GV18916@xs4all.nl>	<aac2c7cb0601171538u6525610ai6214bb697cc72c79@mail.gmail.com>	<ca471dc20601171602t449de0b5h7d5a8d3a40e5ba92@mail.gmail.com>	<20060118010127.GB6152@performancedrivers.com>	<8BEAD4AB-6F49-4901-BBE7-77870E953AD5@redivi.com>	<20060118031238.GC6152@performancedrivers.com>	<4B72B1E6-EEA1-4DE1-A096-5E6D478340C2@redivi.com>	<ca471dc20601172025i7779a936m902d6fc6199b4b71@mail.gmail.com>	<1137586580.16309.28.camel@warna.dub.corp.google.com>
	<bb8868b90601180839r56691cbfj7caf6224e6402799@mail.gmail.com>
Message-ID: <dqlvlp$fg4$2@sea.gmane.org>

Jason Orendorff wrote:
> On 1/18/06, Donovan Baarda <abo at minkirri.apana.org.au> wrote:
> 
>>I think supporting arbitrary bases for floats is way overkill and not
>>worth considering.
> 
> 
> If you mean actual base-3 floating-point arithmetic, I agree.  That's
> outlandish.
> 
> But if there were a stdlib function to format floats losslessly in hex
> or binary, Tim Peters would use it at least once every six weeks to
> illustrate the finer points of floating point arithmetic. <0.00390625
> wink>
> 
> +1.0
> 
Nah, Tim's got the chops to use the struct model to get his point across.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From skip at pobox.com  Wed Jan 18 19:34:16 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 18 Jan 2006 12:34:16 -0600
Subject: [Python-Dev] Python icon
In-Reply-To: <43CE60B0.1070806@holdenweb.com>
References: <dqc0u2$ul0$1@sea.gmane.org>
	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
	<17354.20722.560136.55434@montanaro.dyndns.org>
	<dqjdd6$ro2$1@sea.gmane.org>
	<17357.15206.570762.861782@montanaro.dyndns.org>
	<43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com>
Message-ID: <17358.35368.931097.508092@montanaro.dyndns.org>


    Steve> We should also add script explaining how to download the beta
    Steve> site data and the generation software so people can play with it
    Steve> and get ready to be webmasters :-)

My first attempt ended almost immediately.  Too much software to download
and install for anything like casual use.

Skip

From skip at pobox.com  Wed Jan 18 19:37:39 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 18 Jan 2006 12:37:39 -0600
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
Message-ID: <17358.35571.540508.624539@montanaro.dyndns.org>


    Anthony> It sounds like configure needs to grow a test to detect that a
    Anthony> "libreadline" it finds is actually the crackful "libedit" and
    Anthony> refuse to use it if so.

    >> FYI: Real libreadline is GPL, ...

Didn't Python's readline module work with libedit once upon a time?  I
believe that's the readline semi-clone that Rich Salz quite awhile ago.
I've forgotten the details of the difference (missing history capabilities
perhaps).  Is there some way we can make libedit palatable again?

Skip

From brett at python.org  Wed Jan 18 20:09:00 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 18 Jan 2006 11:09:00 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <001501c61c48$3ed669a0$a3f9a244@oemcomputer>
References: <20060118152356.GA20772@panix.com>
	<001501c61c48$3ed669a0$a3f9a244@oemcomputer>
Message-ID: <bbaeab100601181109j6db93090wba8ac65804c3fd74@mail.gmail.com>

On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> > > I'd propose bin() to stay in line with the short abbreviated names.
> >
> > There has been some previous discussion about removing hex()/oct()
> from
> > builtins for Python 3.0, IIRC.  I sure don't think bin() belongs
> there.
>
> Perhaps introduce a single function, base(val, radix=10, prefix=''), as
> a universal base converter that could replace bin(), hex(), oct(), etc.
>
> That would give us fewer builtins and provide an inverse for all the
> int() conversions (i.e. arbitrary bases).  Also, it would allow an
> unprefixed output which is what I usually need.
>

+1.  Differs from Neal's format() function by not magically
determining the prefix from the radix which I like.

-Brett

From bob at redivi.com  Wed Jan 18 20:24:23 2006
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 18 Jan 2006 11:24:23 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <200601182031.54111.anthony@interlink.com.au>
References: <r7761zxk.fsf@python.net>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<200601182031.54111.anthony@interlink.com.au>
Message-ID: <807E9CF0-1742-41A2-8DB4-96D9C9424BDD@redivi.com>


On Jan 18, 2006, at 1:31 AM, Anthony Baxter wrote:

> On Wednesday 18 January 2006 16:25, Stephen J. Turnbull wrote:
>> Unless rms has changed his position on this, or there has been
>> relevant legislation or a court decision in the meantime,
>> explicitly requiring or checking for "real" libreadline, even as a
>> user option, risks rms's wrath.  (Of course, Python could change
>> its license to GPL, which would undoubtedly flood Cambridge with
>> tears of joy<wink>).
>
> Python's license is GPL-compatible, so this isn't an issue.
>
>> As long as the link to fake libreadline succeeds and the resulting
>> program works identically to one linked to real libreadline, he has
>> no complaint.
>
> My "complaint" is that libedit _calls_ itself libreadline, when it's
> pretty clear that it's not actually a drop-in replacement (or the
> readline module would build). Hence my use of the word "crackful"
> <wink>

That's just something stupid Apple did.  Not libedit's fault.

-bob


From martin at v.loewis.de  Wed Jan 18 20:32:42 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Jan 2006 20:32:42 +0100
Subject: [Python-Dev] [Python-checkins] r42090 - in python/trunk:
 Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj PCbuild/pcbuild.sln
 PCbuild/pythoncore.vcproj
In-Reply-To: <1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com>
References: <20060118091354.A31B91E4002@bag.python.org>
	<1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com>
Message-ID: <43CE97DA.5080302@v.loewis.de>

Tim Peters wrote:
> Did this checkin perhaps forget to add that file?

Oops, indeed - please try again.

Regards,
Martin

From jason.orendorff at gmail.com  Wed Jan 18 21:02:22 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 18 Jan 2006 15:02:22 -0500
Subject: [Python-Dev] basenumber redux
In-Reply-To: <43CD7BE9.6060705@v.loewis.de>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>
	<43CC17B9.3060200@v.loewis.de>
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>
	<43CCC0AE.5070800@egenix.com>
	<e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>
	<43CD7BE9.6060705@v.loewis.de>
Message-ID: <bb8868b90601181202k3cac4f3eje4c05133f8f0b117@mail.gmail.com>

On 1/17/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Alex Martelli wrote:
> > But this doesn't apply to the Python Standard Library, for example see
> > line 1348 of imaplib.py: "if isinstance(date_time, (int, float)):".
> [...]
> > Being able to change imaplib to use basenumber instead of (int, float)
> > won't make it SIMPLER, but it will surely make it BETTER -- why should
> > a long be rejected, or a Decimal, for that matter?
>
> Right. I think this function should read
>
>   if isinstance(date_time, str) and \
>      (date_time[0],date_time[-1]) == ('"','"'):
>         return date_time        # Assume in correct format
>
>   if isinstance(date_time, (tuple, time.struct_time)):
>      tt = date_time
>   else:
>      tt = time.localtime(date_time)

So... arbitrary number-like objects should work, but arbitrary
sequence-like objects should fail?  Hmmm.  Maybe that "if
isinstance()" line should say "if hasattr(date_time, '__getitem__'):".
 Am I sure?  No.  The original author of imaplib apparently got it
wrong, and Martin got it wrong, and they're both smarter than me.

Really this is just further proof that type-checking is a royal pain
in Python.  Or rather, it's not hard to cover the builtin and stdlib
types, but it's hard to support "duck typing" too.  Are we going about
this the right way?  Options:

1.  Redesign the API so each parameter has a clearly defined set of
operations it must support, thus eliminating the need for
type-checking.  Drawback:  An annoying API might be worse than the
problem we're trying to solve.

2.  Write a set of imprecise, general-purpose type-checking functions
(is_real_number(v), is_sequence(v), ...) and use those.  (They are
imprecise because the requirements are vague and because it's not
really possible to pin them down.)  Drawback:  Error-prone, compounded
by deceptively clean appearance.

3.  Write very specific custom type-checking code each time you need
it (the imaplib approach).  Drawbacks:  Error-prone (as we've seen),
precarious, tedious, unreadable.

4.  Use the "better-to-ask-forgiveness-than-permission" idiom. 
Drawback:  Potential bad behavior on error, again potentially worse
than the original problem.

Yuck.  Does anyone have the answer to this one?  Or is the problem not
as bad as it looks?

-j

From tim.peters at gmail.com  Wed Jan 18 21:04:15 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 18 Jan 2006 15:04:15 -0500
Subject: [Python-Dev] [Python-checkins] r42090 - in python/trunk:
	Modules/getbuildinfo.c PCbuild/make_buildinfo.vcproj
	PCbuild/pcbuild.sln PCbuild/pythoncore.vcproj
In-Reply-To: <43CE97DA.5080302@v.loewis.de>
References: <20060118091354.A31B91E4002@bag.python.org>
	<1f7befae0601180721vbc67e95u54bfafb0569357fe@mail.gmail.com>
	<43CE97DA.5080302@v.loewis.de>
Message-ID: <1f7befae0601181204pdf33b70me0109b0c747a7d71@mail.gmail.com>

[Tim]
>> Did this checkin perhaps forget to add that file?

[Martin]
> Oops, indeed - please try again.

It may ;-) be better now.  Looks like the release build finished, but
the debug build died:

"""
------ Build started: Project: pythoncore, Configuration: Debug Win32 ------

generate buildinfo
C:\Program Files\TortoiseSVN\bin\subwcrev.exe ..
..\Modules\getbuildinfo.c getbuildinfo2.c
'C:\Program' is not recognized as an internal or external command,
operable program or batch file.
cl.exe -c -D_WIN32 -DUSE_DL_EXPORT -D_WINDOWS -DWIN32 -D_WINDLL
-D_DEBUG -MDd getbuildinfo2.c -DSUBWCREV  -Fogetbuildinfo.o
-I..\Include -I..\PC
Microsoft (R) 32-bit C/C++ Optimizing Compiler Version 13.10.3077 for 80x86
Copyright (C) Microsoft Corporation 1984-2002. All rights reserved.
getbuildinfo2.c
c1 : fatal error C1083: Cannot open source file: 'getbuildinfo2.c': No
such file or directory
Linking...
LINK : fatal error LNK1181: cannot open input file 'getbuildinfo.o'
"""

OK, I added code to getbuildinfo.c to quote the path to the executable
before invoking system(), and that seemed to fix it.  On my box.

From jimjjewett at gmail.com  Wed Jan 18 21:22:37 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Wed, 18 Jan 2006 15:22:37 -0500
Subject: [Python-Dev] str with base
Message-ID: <fb6fbf560601181222p3746a6fdm43f0a9737f5afbad@mail.gmail.com>

Jack wrote:

> The arbitrary base case isn't even academic
> or we would see homework questions about it
> on c.l.py.  No one asks about
> hex or octal because they are there.

I have wanted base-36 far more often than I've
wanted base-8.  I haven't needed any base
(except 10) often enough to justify putting it
in builtins rather than a stdlib module.  I do like
the idea of adding Raymond's

    def base(number, base, prefix)

to the stdlib (possibly in math).

-jJ

From martin at v.loewis.de  Wed Jan 18 21:36:20 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Wed, 18 Jan 2006 21:36:20 +0100
Subject: [Python-Dev] basenumber redux
In-Reply-To: <bb8868b90601181202k3cac4f3eje4c05133f8f0b117@mail.gmail.com>
References: <DEE520E7-2BDB-40E7-A030-594B6892DBFF@gmail.com>	
	<ca471dc20601160753l1d90dce4pb33a4d33b87236fe@mail.gmail.com>	
	<17A2FF78-0847-499D-B06E-2351DF462CE2@gmail.com>	
	<43CC17B9.3060200@v.loewis.de>	
	<37211B72-FCE2-4C51-A2D8-FE9733CD67E4@gmail.com>	
	<43CCC0AE.5070800@egenix.com>	
	<e8a0972d0601171453n27fe0f7je466242965b9ce7d@mail.gmail.com>	
	<43CD7BE9.6060705@v.loewis.de>
	<bb8868b90601181202k3cac4f3eje4c05133f8f0b117@mail.gmail.com>
Message-ID: <43CEA6C4.5060201@v.loewis.de>

Jason Orendorff wrote:
> Really this is just further proof that type-checking is a royal pain
> in Python.  Or rather, it's not hard to cover the builtin and stdlib
> types, but it's hard to support "duck typing" too.  Are we going about
> this the right way?

It's not as bad. There is nothing wrong with restricting the set of
acceptable types if callers would have no problems to convert their
input into one of the acceptable types.

In the imaplib example, requesting that a broken-down time is passed
as a tuple or a time.struct_time is not too hard for a caller.
It will be formatted as

  dt = time.strftime("%d-%b-%Y %H:%M:%S", tt)

so it needs to have the right number of fields. Callers having
other kinds of sequence can easily use tuple(L) to convert their
data into what the function accepts.

Regards,
Martin

From tim at pollenation.net  Wed Jan 18 21:40:44 2006
From: tim at pollenation.net (Tim Parkin)
Date: Wed, 18 Jan 2006 20:40:44 +0000
Subject: [Python-Dev] Python icon
In-Reply-To: <17358.35368.931097.508092@montanaro.dyndns.org>
References: <dqc0u2$ul0$1@sea.gmane.org>	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>	<17354.20722.560136.55434@montanaro.dyndns.org>	<dqjdd6$ro2$1@sea.gmane.org>	<17357.15206.570762.861782@montanaro.dyndns.org>	<43CE55D1.5030107@gmail.com>
	<43CE60B0.1070806@holdenweb.com>
	<17358.35368.931097.508092@montanaro.dyndns.org>
Message-ID: <43CEA7CC.4090602@pollenation.net>

skip at pobox.com wrote:

>    Steve> We should also add script explaining how to download the beta
>    Steve> site data and the generation software so people can play with it
>    Steve> and get ready to be webmasters :-)
>
>My first attempt ended almost immediately.  Too much software to download
>and install for anything like casual use.
>
>  
>
For casual use why not just edit the rest file? That was one of main
points about creating a file based admin system... Most users can just
edit the rest files or data files (one of the library dependences could
have been avoided by using xml for the data files but I'm not sure this
would have gone down well). Creating new pages is also mostly just
writing rest content (if you want to do this but can't be bothered to
install the software, send your work over to me and I'll add it). You
only need to have all of the software if you are creating whole new
sections, in which case it wouldn't really be casual use.

I may be wrong in saying this, but I thought the main idea of having
software libraries was to use them? I avoided writing specific modules
because the software was out there that did things a lot better than I
could rewrite. Is there a level of library use that is acceptable beyond
which we should rewrite components or bundle libraries? I would have
thought most applications that involved web site creation would have at
least three or four external library dependencies (templating, data
interface, web server/http module).

Tim Parkin

From skip at pobox.com  Wed Jan 18 22:53:26 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 18 Jan 2006 15:53:26 -0600
Subject: [Python-Dev] Python icon
In-Reply-To: <43CEA7CC.4090602@pollenation.net>
References: <dqc0u2$ul0$1@sea.gmane.org>
	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
	<17354.20722.560136.55434@montanaro.dyndns.org>
	<dqjdd6$ro2$1@sea.gmane.org>
	<17357.15206.570762.861782@montanaro.dyndns.org>
	<43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com>
	<17358.35368.931097.508092@montanaro.dyndns.org>
	<43CEA7CC.4090602@pollenation.net>
Message-ID: <17358.47318.720591.658564@montanaro.dyndns.org>


    >> My first attempt ended almost immediately.  Too much software to download
    >> and install for anything like casual use.

    Tim> For casual use why not just edit the rest file?

Maybe I misread the directions.  I thought I had to install some new library
I'd never heard of (syck), Python bindings for the same, and maybe some
other stuff.  It clearly wasn't just "svn co ..." and start editing.  In any
case, I couldn't tell what needed doing from the trac site.

I'll take another look when I have a chance.  I would be happy to simple
edit rest files if I knew what to edit.

Skip

From tim at pollenation.net  Wed Jan 18 23:26:40 2006
From: tim at pollenation.net (Tim Parkin)
Date: Wed, 18 Jan 2006 22:26:40 +0000
Subject: [Python-Dev] Python icon
In-Reply-To: <17358.47318.720591.658564@montanaro.dyndns.org>
References: <dqc0u2$ul0$1@sea.gmane.org>
	<Pine.LNX.4.58.0601141835280.6831@server1.LFW.org>
	<17354.20722.560136.55434@montanaro.dyndns.org>
	<dqjdd6$ro2$1@sea.gmane.org>
	<17357.15206.570762.861782@montanaro.dyndns.org>
	<43CE55D1.5030107@gmail.com> <43CE60B0.1070806@holdenweb.com>
	<17358.35368.931097.508092@montanaro.dyndns.org>
	<43CEA7CC.4090602@pollenation.net>
	<17358.47318.720591.658564@montanaro.dyndns.org>
Message-ID: <43CEC0A0.20805@pollenation.net>

skip at pobox.com wrote:
>     >> My first attempt ended almost immediately.  Too much software to download
>     >> and install for anything like casual use.
> 
>     Tim> For casual use why not just edit the rest file?
> 
> Maybe I misread the directions.  I thought I had to install some new library
> I'd never heard of (syck), Python bindings for the same, and maybe some
> other stuff.  It clearly wasn't just "svn co ..." and start editing.  In any
> case, I couldn't tell what needed doing from the trac site.
> 
> I'll take another look when I have a chance.  I would be happy to simple
> edit rest files if I knew what to edit.

To build all of the web pages, you need to have a yaml parser installed.
As much as I'd like a pure python yaml parser, the syck c parser is
currently the definitive implementation (and the pysyck wrapper at
xitology the definitive python wrapper). The syck files are used to
manage the data for the templates but 95% of the pages I've migrated
have just been a simple nav yaml file with a rest file for the body of
the content. The beta site is being rebuilt on commit so if you see any
content missing, it needs generating. I've added a couple of tickets
highlighting large areas that need work (the tickets on
psf.pollenation.net with "CONTENT" as a prefix).

If you want to proof content you can just go to the appropriate data
directory (matching the url of the page in question) and edit the rest
file. If you want to add new pages, it would be good to be able to build
the site to check what you have added is working. However, as there
aren't many people working on the site, I'd be more than happy to take
whatever you can supply and integrate it into the site.

At the moment I'm torn between making the software easier to use or
making sure content is migrated. It should only take about 10 or 20
minutes to get the software installed (if it takes longer then please
tell me what the problems are and I'll try to document any issues and
streamline the process - I'd rather not optimise the install/docs until
there are specific problems - I think my time is better spent getting
some of the more complex content migrated, and ironing out the last
problems with the css, and getting new images for the home page and
writing new content for the 'for ...' sections and getting the success
stories copied over, and getting wiki content integrated and getting the
docs integrated into the site better, and ..well .. other stuff).

I really want to make the website easier for people to work on and fully
intend to provide some form of simple web based content management when
enough content has been migrated to allow the site to go live.

I hope that wasn't too much of a rant.. :-) Drop me an email if I can
help in any way.

Tim




From ncoghlan at gmail.com  Thu Jan 19 01:08:41 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 19 Jan 2006 10:08:41 +1000
Subject: [Python-Dev] str with base
In-Reply-To: <bbaeab100601181109j6db93090wba8ac65804c3fd74@mail.gmail.com>
References: <20060118152356.GA20772@panix.com>	<001501c61c48$3ed669a0$a3f9a244@oemcomputer>
	<bbaeab100601181109j6db93090wba8ac65804c3fd74@mail.gmail.com>
Message-ID: <43CED889.3010306@gmail.com>

Brett Cannon wrote:
> On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
>>>> I'd propose bin() to stay in line with the short abbreviated names.
>>> There has been some previous discussion about removing hex()/oct()
>> from
>>> builtins for Python 3.0, IIRC.  I sure don't think bin() belongs
>> there.
>>
>> Perhaps introduce a single function, base(val, radix=10, prefix=''), as
>> a universal base converter that could replace bin(), hex(), oct(), etc.
>>
>> That would give us fewer builtins and provide an inverse for all the
>> int() conversions (i.e. arbitrary bases).  Also, it would allow an
>> unprefixed output which is what I usually need.
>>
> 
> +1.  Differs from Neal's format() function by not magically
> determining the prefix from the radix which I like.

+1 here, too, particularly if hex/oct acquire Deprecation (or even just 
PendingDeprecation) warnings at the same time.

I have my own reason for wanting to avoid the name format() - I'd still like 
to see it used one day to provide a builtin way to use string.Template syntax 
for arbitrary string formatting.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From anthony at interlink.com.au  Thu Jan 19 02:36:36 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu, 19 Jan 2006 12:36:36 +1100
Subject: [Python-Dev] Python icon
In-Reply-To: <17358.47318.720591.658564@montanaro.dyndns.org>
References: <dqc0u2$ul0$1@sea.gmane.org> <43CEA7CC.4090602@pollenation.net>
	<17358.47318.720591.658564@montanaro.dyndns.org>
Message-ID: <200601191236.41719.anthony@interlink.com.au>

On Thursday 19 January 2006 08:53, skip at pobox.com wrote:
> Maybe I misread the directions.  I thought I had to install some
> new library I'd never heard of (syck), Python bindings for the
> same, and maybe some other stuff.  It clearly wasn't just "svn co
> ..." and start editing.  In any case, I couldn't tell what needed
> doing from the trac site.

We obviously need to get everything that's needed to use setuptools 
and eggs so people can just have things "just work". Yay for PJE!


-- 
Anthony Baxter     <anthony at interlink.com.au>
It's never too late to have a happy childhood.

From aleaxit at gmail.com  Thu Jan 19 03:16:03 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 18 Jan 2006 18:16:03 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <bbaeab100601181109j6db93090wba8ac65804c3fd74@mail.gmail.com>
References: <20060118152356.GA20772@panix.com>
	<001501c61c48$3ed669a0$a3f9a244@oemcomputer>
	<bbaeab100601181109j6db93090wba8ac65804c3fd74@mail.gmail.com>
Message-ID: <050D3FEE-E91C-4015-ADD9-228422DC39FA@gmail.com>

On Jan 18, 2006, at 11:09 AM, Brett Cannon wrote:

> On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
>>>> I'd propose bin() to stay in line with the short abbreviated names.
>>>
>>> There has been some previous discussion about removing hex()/oct()
>> from
>>> builtins for Python 3.0, IIRC.  I sure don't think bin() belongs
>> there.
>>
>> Perhaps introduce a single function, base(val, radix=10,  
>> prefix=''), as
>> a universal base converter that could replace bin(), hex(), oct(),  
>> etc.
>>
>> That would give us fewer builtins and provide an inverse for all the
>> int() conversions (i.e. arbitrary bases).  Also, it would allow an
>> unprefixed output which is what I usually need.
>
> +1.  Differs from Neal's format() function by not magically
> determining the prefix from the radix which I like.

I'm not sure I see the advantage of, say,

print base(x, radix=2, prefix='0b')

versus

print '0b'+base(x, radix=2)

IOW, if the prefix needs to be explicitly specified anyway, what's  
the advantage of specifying it as an argument to base, rather than  
just string-concatenating it?

Apart from that quibble, the base function appears to cover all the  
use cases for my proposed str-with-base, so, since it appears to  
attract less arguments, I'm definitely +1 on it.


Alex



From brett at python.org  Thu Jan 19 03:50:53 2006
From: brett at python.org (Brett Cannon)
Date: Wed, 18 Jan 2006 18:50:53 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <050D3FEE-E91C-4015-ADD9-228422DC39FA@gmail.com>
References: <20060118152356.GA20772@panix.com>
	<001501c61c48$3ed669a0$a3f9a244@oemcomputer>
	<bbaeab100601181109j6db93090wba8ac65804c3fd74@mail.gmail.com>
	<050D3FEE-E91C-4015-ADD9-228422DC39FA@gmail.com>
Message-ID: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>

On 1/18/06, Alex Martelli <aleaxit at gmail.com> wrote:
> On Jan 18, 2006, at 11:09 AM, Brett Cannon wrote:
>
> > On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> >>>> I'd propose bin() to stay in line with the short abbreviated names.
> >>>
> >>> There has been some previous discussion about removing hex()/oct()
> >> from
> >>> builtins for Python 3.0, IIRC.  I sure don't think bin() belongs
> >> there.
> >>
> >> Perhaps introduce a single function, base(val, radix=10,
> >> prefix=''), as
> >> a universal base converter that could replace bin(), hex(), oct(),
> >> etc.
> >>
> >> That would give us fewer builtins and provide an inverse for all the
> >> int() conversions (i.e. arbitrary bases).  Also, it would allow an
> >> unprefixed output which is what I usually need.
> >
> > +1.  Differs from Neal's format() function by not magically
> > determining the prefix from the radix which I like.
>
> I'm not sure I see the advantage of, say,
>
> print base(x, radix=2, prefix='0b')
>
> versus
>
> print '0b'+base(x, radix=2)
>
> IOW, if the prefix needs to be explicitly specified anyway, what's
> the advantage of specifying it as an argument to base, rather than
> just string-concatenating it?

It collects the data that is expected to be used in the common case in
a single location/operation. This would allow you to do something like
``base(x, **radix_and_prefix_dict)`` and have everythihng in a nice,
neat package.

Plus the operation would be faster if base() is written in C.  =)

The other option is to go with Neal's solution for automatically
including the prefix for known prefix types, but instead of it being a
boolean, let it be a string argument.  That means if you want no
prefix you would just set the argument to the empty string.  Not
setting it will just use the most sensible prefix or none if one is
not known for the specified radix.  Could have something somewhere,
like string or math, where more radix/prefix pairs can be added by the
user and have base() reference that for its prefix values.

IOW I am +0 on prefix in one of these forms.

-Brett

From aahz at pythoncraft.com  Thu Jan 19 05:40:17 2006
From: aahz at pythoncraft.com (Aahz)
Date: Wed, 18 Jan 2006 20:40:17 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
Message-ID: <20060119044017.GB11147@panix.com>

On Wed, Jan 18, 2006, Guido van Rossum wrote:
>
> Can we just all agree that RMS is an asshole now? Bah.

"Citing RMS's insanity is a great way to get my blood steaming."  --GvR
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From foom at fuhm.net  Thu Jan 19 05:47:59 2006
From: foom at fuhm.net (James Y Knight)
Date: Wed, 18 Jan 2006 23:47:59 -0500
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <20060119044017.GB11147@panix.com>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
	<20060119044017.GB11147@panix.com>
Message-ID: <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net>


On Jan 18, 2006, at 11:40 PM, Aahz wrote:

> On Wed, Jan 18, 2006, Guido van Rossum wrote:
>>
>> Can we just all agree that RMS is an asshole now? Bah.
>
> "Citing RMS's insanity is a great way to get my blood steaming."  -- 
> GvR

Ya know, you don't *have* to use his software. For example, python  
could stop supporting readline and support libedit instead.

James


From bob at redivi.com  Thu Jan 19 05:54:06 2006
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 18 Jan 2006 20:54:06 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
	<20060119044017.GB11147@panix.com>
	<188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net>
Message-ID: <3A4F7AED-0D0B-4D31-806B-C3F8886C05CA@redivi.com>


On Jan 18, 2006, at 8:47 PM, James Y Knight wrote:

>
> On Jan 18, 2006, at 11:40 PM, Aahz wrote:
>
>> On Wed, Jan 18, 2006, Guido van Rossum wrote:
>>>
>>> Can we just all agree that RMS is an asshole now? Bah.
>>
>> "Citing RMS's insanity is a great way to get my blood steaming."  --
>> GvR
>
> Ya know, you don't *have* to use his software. For example, python
> could stop supporting readline and support libedit instead.

It'd probably be better to just use something like pyrepl <http:// 
codespeak.net/pyrepl/>, which builds on top of curses and termios.   
More Python code, less GPL mess.

-bob


From aleaxit at gmail.com  Thu Jan 19 05:57:45 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Wed, 18 Jan 2006 20:57:45 -0800
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net>
References: <r7761zxk.fsf@python.net> <1137525587.9528.19.camel@geddy.wooz.org>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<ca471dc20601180952t34a85f4jd0cd42d46b03b4eb@mail.gmail.com>
	<20060119044017.GB11147@panix.com>
	<188C313F-D2C6-4315-9710-7FACA3D87799@fuhm.net>
Message-ID: <FAE6C1E7-C643-493E-9218-ABF9CA2D78D5@gmail.com>


On Jan 18, 2006, at 8:47 PM, James Y Knight wrote:

> On Jan 18, 2006, at 11:40 PM, Aahz wrote:
>
>> On Wed, Jan 18, 2006, Guido van Rossum wrote:
>>>
>>> Can we just all agree that RMS is an asshole now? Bah.
>>
>> "Citing RMS's insanity is a great way to get my blood steaming."  --
>> GvR
>
> Ya know, you don't *have* to use his software. For example, python
> could stop supporting readline and support libedit instead.

In Py3k this should indeed be considered; I don't think it can be  
within Python 2.*, since it would be a major and never fully  
backwards-compatible change.


Alex


From nnorwitz at gmail.com  Thu Jan 19 07:29:50 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 18 Jan 2006 22:29:50 -0800
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <43C37C02.8030005@egenix.com>
References: <43B3ECEE.9060300@v.loewis.de>
	<20051229173041.GA19575@code1.codespeak.net>
	<43B50B64.20904@v.loewis.de>
	<20060105194538.GB11736@code1.codespeak.net>
	<43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com>
Message-ID: <ee2a432c0601182229w7523ce41nab44a81e4e2329bc@mail.gmail.com>

On 1/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
>
> We'd also have to make sure that old extensions don't
> just import with a warning, since the change will introduce
> buffer overflows and seg faults in extensions that are not
> aware of the change.

I agree that on 64-bit platforms we should prevent the import.  In the
past we only provided a warning and the users were on their own.  This
is different.

If you read my massive checkin to check the return results of
Py_InitModule*(), you'll realize this isn't as simple as just failing
in Py_InitMethod*().  I was hoping to just modify Py_InitModule4() in
Python/modsupport.c to fail and return NULL.  That doesn't seem
practical given that we didn't check return results.  We will just
crash the interpreter with standard python 2.4 modules.

ISTM we need to modify _PyImport_LoadDynamicModule() in
Python/importdl.c before calling the init function (line 56, (*p)())
to check for some magic symbol that is defined only when compiling 2.5
and above.  For example we could add a static int  _64_bit_clean = 1;
in modsupport.h.  Without some trickery we will get this defined in
every .o file though, not just modules.

Other ideas?

n

From python at rcn.com  Thu Jan 19 07:55:42 2006
From: python at rcn.com (Raymond Hettinger)
Date: Thu, 19 Jan 2006 01:55:42 -0500
Subject: [Python-Dev] basenumber redux
In-Reply-To: <43CEA6C4.5060201@v.loewis.de>
Message-ID: <001c01c61cc5$6084f8a0$50b02c81@oemcomputer>



> -----Original Message-----
> From: python-dev-bounces+python=rcn.com at python.org [mailto:python-dev-
> bounces+python=rcn.com at python.org] On Behalf Of Martin v. L?wis
> Sent: Wednesday, January 18, 2006 3:36 PM
> To: Jason Orendorff
> Cc: python-dev at python.org
> Subject: Re: [Python-Dev] basenumber redux
> 
> Jason Orendorff wrote:
> > Really this is just further proof that type-checking is a royal pain
> > in Python.  Or rather, it's not hard to cover the builtin and stdlib
> > types, but it's hard to support "duck typing" too.  Are we going
about
> > this the right way?
> 
> It's not as bad. There is nothing wrong with restricting the set of
> acceptable types if callers would have no problems to convert their
> input into one of the acceptable types.

Somehow my earlier post on this thread didn't seem to take.

There are problems for callers converting their inputs:

* currently existing number-like objects would need be retro-fitted with
a new base class and possibly change their behavior from old-style to
new-style.

* some useful classes (such as a symbolic type) already inherit from
str.  That prevents them from also being able to inherit from
basenumber.

I'm -1 on the proposal because the benefits are dubious (at best it
simplifies just a handful of code fragments); it adds yet another API to
learn and remember; it is darned inconvenient for existing code seeking
to emulate number-like behavior; and it precludes number emulation for
classes that already have a built-in base class.

For the most part, functions that enforce type checking are up to no
good and make life harder for their callers.  If Python ultimately grows
interfaces, I hope they remain optional; as soon as functions start
insisting on interface checking, then it will spread like cancer.  The
basenumber proposal is essentially a step down that slippery slope.


Raymond


Raymond

From raymond.hettinger at verizon.net  Thu Jan 19 08:26:12 2006
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Thu, 19 Jan 2006 02:26:12 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
Message-ID: <003801c61cc9$a0f73e80$50b02c81@oemcomputer>

Guido, we may be converging on a consensus for my proposal:

    base(value, radix=2)

So far no one has shot at it, and it has gathered +1's from Steven,
Alex, Brett, and Nick.

To keep it simple, the proposal is for the value to be any int or long.
With an underlying __base__ method call, it wouldn't be hard for someone
to build it out to support other numeric types if the need arises.

The output would have no prefixes.  As Alex pointed out, it is easier
and more direct to add those after the fact if needed.

Care to pronounce on it?


Raymond




From nnorwitz at gmail.com  Thu Jan 19 08:37:16 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 18 Jan 2006 23:37:16 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <003801c61cc9$a0f73e80$50b02c81@oemcomputer>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
Message-ID: <ee2a432c0601182337h26f1c3ffj71489949e42108bf@mail.gmail.com>

On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> Guido, we may be converging on a consensus for my proposal:
>
>     base(value, radix=2)
>
> So far no one has shot at it, and it has gathered +1's from Steven,
> Alex, Brett, and Nick.

+1 for me too, but I'd also like to deprecate hex() and oct() and
slate them for removal in 3k.

To expand, valid radix values would be 2..36 (ie, same as for int). 
It was discussed putting base() in some module.  Was there consensus
about builtin vs a module?  I'd prefer a module, but builtin is ok
with me.

> To keep it simple, the proposal is for the value to be any int or long.
> With an underlying __base__ method call, it wouldn't be hard for someone
> to build it out to support other numeric types if the need arises.
>
> The output would have no prefixes.  As Alex pointed out, it is easier
> and more direct to add those after the fact if needed.

+1

> Care to pronounce on it?
>
>
> Raymond

From bob at redivi.com  Thu Jan 19 08:58:01 2006
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 18 Jan 2006 23:58:01 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ee2a432c0601182337h26f1c3ffj71489949e42108bf@mail.gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ee2a432c0601182337h26f1c3ffj71489949e42108bf@mail.gmail.com>
Message-ID: <71F6D09D-93E0-44E1-8B58-EC3A9A37D1C6@redivi.com>


On Jan 18, 2006, at 11:37 PM, Neal Norwitz wrote:

> On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
>> Guido, we may be converging on a consensus for my proposal:
>>
>>     base(value, radix=2)
>>
>> So far no one has shot at it, and it has gathered +1's from Steven,
>> Alex, Brett, and Nick.
>
> +1 for me too, but I'd also like to deprecate hex() and oct() and
> slate them for removal in 3k.
>
> To expand, valid radix values would be 2..36 (ie, same as for int).
> It was discussed putting base() in some module.  Was there consensus
> about builtin vs a module?  I'd prefer a module, but builtin is ok
> with me.

I'd drop the default radix, or make it something common like 16...  
especially if hex and oct are to be py3k deprecated.

+1 for: base(value, radix)
+1 for: "%b" % (integer,)
+0 for binary literals: 0b01101

-bob


From mal at egenix.com  Thu Jan 19 10:18:58 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 19 Jan 2006 10:18:58 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <ee2a432c0601182229w7523ce41nab44a81e4e2329bc@mail.gmail.com>
References: <43B3ECEE.9060300@v.loewis.de>	
	<20051229173041.GA19575@code1.codespeak.net>	
	<43B50B64.20904@v.loewis.de>	
	<20060105194538.GB11736@code1.codespeak.net>	
	<43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com>
	<ee2a432c0601182229w7523ce41nab44a81e4e2329bc@mail.gmail.com>
Message-ID: <43CF5982.5050802@egenix.com>

Neal Norwitz wrote:
> On 1/10/06, M.-A. Lemburg <mal at egenix.com> wrote:
>> We'd also have to make sure that old extensions don't
>> just import with a warning, since the change will introduce
>> buffer overflows and seg faults in extensions that are not
>> aware of the change.
> 
> I agree that on 64-bit platforms we should prevent the import.  In the
> past we only provided a warning and the users were on their own.  This
> is different.
> 
> If you read my massive checkin to check the return results of
> Py_InitModule*(), you'll realize this isn't as simple as just failing
> in Py_InitMethod*().  I was hoping to just modify Py_InitModule4() in
> Python/modsupport.c to fail and return NULL.  That doesn't seem
> practical given that we didn't check return results.  We will just
> crash the interpreter with standard python 2.4 modules.
> 
> ISTM we need to modify _PyImport_LoadDynamicModule() in
> Python/importdl.c before calling the init function (line 56, (*p)())
> to check for some magic symbol that is defined only when compiling 2.5
> and above.  For example we could add a static int  _64_bit_clean = 1;
> in modsupport.h.  Without some trickery we will get this defined in
> every .o file though, not just modules.
> 
> Other ideas?

We could explicitly break binary compatibility for Python 2.5
on 64-bit platforms, by changing the name of an often used
API, e.g. the Py_InitModule*() APIs.

This is how Unicode does it - we map the various APIs to
either ...UCS2 or ...UCS4, so that you cannot import an
extension compiled for e.g. UCS2 into a Python interpreter
compiled for UCS4. If we didn't, you'd get seg faults and
buffer overflows the same way you would with the ssize_t
change on 64-bit platforms.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 19 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From tim at pollenation.net  Thu Jan 19 10:46:32 2006
From: tim at pollenation.net (Tim Parkin)
Date: Thu, 19 Jan 2006 09:46:32 +0000
Subject: [Python-Dev] Python icon
In-Reply-To: <200601191236.41719.anthony@interlink.com.au>
References: <dqc0u2$ul0$1@sea.gmane.org> <43CEA7CC.4090602@pollenation.net>
	<17358.47318.720591.658564@montanaro.dyndns.org>
	<200601191236.41719.anthony@interlink.com.au>
Message-ID: <43CF5FF8.2000803@pollenation.net>

Anthony Baxter wrote:
> On Thursday 19 January 2006 08:53, skip at pobox.com wrote:
> 
>>Maybe I misread the directions.  I thought I had to install some
>>new library I'd never heard of (syck), Python bindings for the
>>same, and maybe some other stuff.  It clearly wasn't just "svn co
>>..." and start editing.  In any case, I couldn't tell what needed
>>doing from the trac site.
> 
> 
> We obviously need to get everything that's needed to use setuptools 
> and eggs so people can just have things "just work". Yay for PJE!
> 
> 

:-D  thats the plan!! only problem is getting zope interface to build as
an egg :-(  any ideas?

Tim

From gmccaughan at synaptics-uk.com  Thu Jan 19 11:23:30 2006
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu, 19 Jan 2006 10:23:30 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <d11dcfba0601180855q7863e1b7p865eec3466031497@mail.gmail.com>
References: <20060118152356.GA20772@panix.com>
	<17358.26589.750736.623827@montanaro.dyndns.org>
	<d11dcfba0601180855q7863e1b7p865eec3466031497@mail.gmail.com>
Message-ID: <200601191023.30902.gmccaughan@synaptics-uk.com>

On Wednesday 2006-01-18 16:55, Steven Bethard wrote:
> [Raymond]
> > Perhaps introduce a single function, base(val, radix=10,
> > prefix=''), as a universal base converter that could replace
> > bin(), hex(), oct(), etc.
> 
> +1 on introducing base()

Introducing a new builtin with a name that's a common, short
English word is a bit disagreeable. The other thing about the
name "base" is that it's not entirely obvious which way it
converts: do you say

    base(123,5)

to get a string representing 123 in base 5, or

    base("123",5)

to get the integer whose base 5 representation is "123"?
Well, one option would be to have both of those work :-).
(Some people may need to do some deep breathing while
reciting the mantra "practicality beats purity" in order
to contemplate that with equanimity.)

Alternatively, a name like "to_base" that clarifies the
intent and is less likely to clash with variable names
might be an improvement.

Or there's always %b, whether that ends up standing for
"binary" or "base". Or %b for binary and %r for radix,
not forgetting the modifiers to get numbers formatted
as Roman numerals.

-- 
Gareth McCaughan


From thomas at xs4all.net  Thu Jan 19 12:15:50 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 19 Jan 2006 12:15:50 +0100
Subject: [Python-Dev] str with base
In-Reply-To: <200601191023.30902.gmccaughan@synaptics-uk.com>
References: <20060118152356.GA20772@panix.com>
	<17358.26589.750736.623827@montanaro.dyndns.org>
	<d11dcfba0601180855q7863e1b7p865eec3466031497@mail.gmail.com>
	<200601191023.30902.gmccaughan@synaptics-uk.com>
Message-ID: <20060119111550.GX18916@xs4all.nl>

On Thu, Jan 19, 2006 at 10:23:30AM +0000, Gareth McCaughan wrote:

> > +1 on introducing base()

> Introducing a new builtin with a name that's a common, short
> English word is a bit disagreeable.

While I don't particularly mind the new function in either the builtin
module or another, like math, I don't understand the problem with the name.
Most builtin names are short and english-ish words. I like that, I'm glad
they are that way, and the two names I dislike most are 'isinstance' and
'issubclass'.

> The other thing about the name "base" is that it's not entirely obvious
> which way it converts: do you say
> 
>     base(123,5)
> 
> to get a string representing 123 in base 5, or
> 
>     base("123",5)
> 
> to get the integer whose base 5 representation is "123"?

This is an argument for 'str(123, 5)', but I don't agree. Not _everything_
has to be obvious at first glance. The very same could be said about hex(),
oct(), dir(), even names like list() (what does it list?), str() (stripping
something?), etc. Having int() do it one way and base() the other makes fine
sense to me, and I don't see it as any harder to explain than, say, why
hex("123") doesn't return 291. I've personally never had to explain
hex/oct's behaviour.

While I think 'str' would be a slightly better name than 'base' (despite the
specialcasing of numbers,) I don't mind either. I do mind names like
'to_base' or 'as_str' or 'shouldbestr' or other names that make me turn on
autocompletion in my editor.

> Alternatively, a name like "to_base" that clarifies the intent and is less
> likely to clash with variable names might be an improvement.

Builtins aren't reserved names, so the clash is minimal.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From gmccaughan at synaptics-uk.com  Thu Jan 19 13:20:52 2006
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Thu, 19 Jan 2006 12:20:52 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <20060119111550.GX18916@xs4all.nl>
References: <20060118152356.GA20772@panix.com>
	<200601191023.30902.gmccaughan@synaptics-uk.com>
	<20060119111550.GX18916@xs4all.nl>
Message-ID: <200601191220.53551.gmccaughan@synaptics-uk.com>

On Thursday 2006-01-19 11:15, Thomas Wouters wrote:
> On Thu, Jan 19, 2006 at 10:23:30AM +0000, Gareth McCaughan wrote:
...
> > Introducing a new builtin with a name that's a common, short
> > English word is a bit disagreeable.
> 
> While I don't particularly mind the new function in either the builtin
> module or another, like math, I don't understand the problem with the name.
> Most builtin names are short and english-ish words. I like that, I'm glad
> they are that way, and the two names I dislike most are 'isinstance' and
> 'issubclass'.

"issubclass" is horrible because wordsjammedtogetherlikethisarehardtoread,
especially when they're misleading as to pronunciation ("iss-ubclass").

Short English words are nice because they're easy to type and
(at least sometimes) their meanings are immediately obvious.
For the same reason, they're useful as variable names. Of course
the world doesn't end if a builtin name is the same as a variable
name you'd like to use, but it's ... well, "a bit disagreeable"
probably expresses about the right degree of nuisance.

> > The other thing about the name "base" is that it's not entirely obvious
> > which way it converts: do you say
> > 
> >     base(123,5)
> > 
> > to get a string representing 123 in base 5, or
> > 
> >     base("123",5)
> > 
> > to get the integer whose base 5 representation is "123"?
> 
> This is an argument for 'str(123, 5)', but I don't agree.

It's not (intended as) an argument *for* any particular form.

>                                                           Not _everything_
> has to be obvious at first glance. The very same could be said about hex(),
> oct(), dir(), even names like list() (what does it list?), str() (stripping
> something?), etc.

Actually, I happen to dislike hex() slightly -- I never use or see
oct(), so don't much care about that -- for exactly that reason.

>                   Having int() do it one way and base() the other makes fine
> sense to me, and I don't see it as any harder to explain than, say, why
> hex("123") doesn't return 291. I've personally never had to explain
> hex/oct's behaviour.

To me, base() is less obvious than hex(), which itself is just
ambiguous enough to cost me maybe one second per month. Not a big
deal at all, but not zero.

> While I think 'str' would be a slightly better name than 'base' (despite the
> specialcasing of numbers,) I don't mind either. I do mind names like
> 'to_base' or 'as_str' or 'shouldbestr' or other names that make me turn on
> autocompletion in my editor.

You need to turn off the Python Mind Control feature. :-)

I think math.base (apart from sounding like it ought to be a
variable that controls the base in which numbers are represented,
or something of the sort) is about as much typing as to_base,
so I'm not sure how the latter can be much worse in this respect.

> > Alternatively, a name like "to_base" that clarifies the intent and is less
> > likely to clash with variable names might be an improvement.
> 
> Builtins aren't reserved names, so the clash is minimal.

Sure, it's not disabling. But in practice it's nice to be able
to avoid builtin names, and "base" is a word I'd rather not have
to take measures to avoid: too many meanings, some of them
quite common.

(I don't care much about this, and if base() gets introduced
I shan't complain.)

-- 
g


From guido at python.org  Thu Jan 19 18:31:30 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Jan 2006 09:31:30 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <003801c61cc9$a0f73e80$50b02c81@oemcomputer>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
Message-ID: <ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>

On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> Guido, we may be converging on a consensus for my proposal:
>
>     base(value, radix=2)
>
> So far no one has shot at it, and it has gathered +1's from Steven,
> Alex, Brett, and Nick.

I think we ought to let this sit for a while and come back to it in a
few week's time. Is 'base' really the right name? It could just as
well be considered a conversion in the other direction. In common
usage, 'base' and 'radix' are about synonymous (except no-one uses
radix). Pethaps the 2nd argument should not be a keyword argument?

Also, this discussion made me wonder if conversions using other bases
than 10 should even be built-ins. A library module seems a more
appropriate place. The prevalence here of people who actually use hex
numbers on a regular basis is probably easily explained by a
combination of old-timers, language implementers, and super-geeks;
hardly the typical Python user. The desire of (bright) beginners to do
any kind of non-decimal conversion probably stems from a misguided
meme (dating back to the invention of computers) that in order to
learn about computers you ought to begin by learning about Boolean
algebra and binary numbers. That might be true long ago, but today,
binary, octal and hexadecimal numbers are mostly a curiosity used in
obscure low-level APIs like ioctl().

> To keep it simple, the proposal is for the value to be any int or long.
> With an underlying __base__ method call, it wouldn't be hard for someone
> to build it out to support other numeric types if the need arises.

Let's not. What would 3.14 be expressed in base 3?

> The output would have no prefixes.  As Alex pointed out, it is easier
> and more direct to add those after the fact if needed.

Agreed.

> Care to pronounce on it?

Rather not yet.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From jeremy at alum.mit.edu  Thu Jan 19 19:24:00 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 19 Jan 2006 10:24:00 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
Message-ID: <e8bf7a530601191024l505e2228r90f048b27f17be74@mail.gmail.com>

I'm not sure I believe this should be a builtin.  I think the
threshold for new builtins ought to be nearly as high as the threshold
for new keywords.  Or the proposer ought to make an argument about
what the function should not go in a module.

Jeremy

On 1/19/06, Guido van Rossum <guido at python.org> wrote:
> On 1/18/06, Raymond Hettinger <raymond.hettinger at verizon.net> wrote:
> > Guido, we may be converging on a consensus for my proposal:
> >
> >     base(value, radix=2)
> >
> > So far no one has shot at it, and it has gathered +1's from Steven,
> > Alex, Brett, and Nick.
>
> I think we ought to let this sit for a while and come back to it in a
> few week's time. Is 'base' really the right name? It could just as
> well be considered a conversion in the other direction. In common
> usage, 'base' and 'radix' are about synonymous (except no-one uses
> radix). Pethaps the 2nd argument should not be a keyword argument?
>
> Also, this discussion made me wonder if conversions using other bases
> than 10 should even be built-ins. A library module seems a more
> appropriate place. The prevalence here of people who actually use hex
> numbers on a regular basis is probably easily explained by a
> combination of old-timers, language implementers, and super-geeks;
> hardly the typical Python user. The desire of (bright) beginners to do
> any kind of non-decimal conversion probably stems from a misguided
> meme (dating back to the invention of computers) that in order to
> learn about computers you ought to begin by learning about Boolean
> algebra and binary numbers. That might be true long ago, but today,
> binary, octal and hexadecimal numbers are mostly a curiosity used in
> obscure low-level APIs like ioctl().
>
> > To keep it simple, the proposal is for the value to be any int or long.
> > With an underlying __base__ method call, it wouldn't be hard for someone
> > to build it out to support other numeric types if the need arises.
>
> Let's not. What would 3.14 be expressed in base 3?
>
> > The output would have no prefixes.  As Alex pointed out, it is easier
> > and more direct to add those after the fact if needed.
>
> Agreed.
>
> > Care to pronounce on it?
>
> Rather not yet.
>
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/jeremy%40alum.mit.edu
>

From shane.holloway at ieee.org  Thu Jan 19 19:28:23 2006
From: shane.holloway at ieee.org (Shane Holloway (IEEE))
Date: Thu, 19 Jan 2006 11:28:23 -0700
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
Message-ID: <ACEB41D3-89B3-421C-BCC2-39005BC4CE53@ieee.org>


On Jan 19, 2006, at 10:31, Guido van Rossum wrote:

>> To keep it simple, the proposal is for the value to be any int or  
>> long.
>> With an underlying __base__ method call, it wouldn't be hard for  
>> someone
>> to build it out to support other numeric types if the need arises.
>
> Let's not. What would 3.14 be expressed in base 3?

10.010210001

It turned out to be a fun aside, and I've attached my quick and dirty  
script as a strawman.

For what it's worth, I don't like the name base() because it sounds  
like something I would call on a class to get it's bases.  Perhaps  
nbase?  And maybe fbase for the floating point one...

Thanks,
-Shane Holloway

-------------- next part --------------
A non-text attachment was scrubbed...
Name: bases.py
Type: text/x-python-script
Size: 1385 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060119/a9b6a3d8/attachment.bin 

From collinw at gmail.com  Thu Jan 19 19:58:00 2006
From: collinw at gmail.com (Collin Winter)
Date: Thu, 19 Jan 2006 19:58:00 +0100
Subject: [Python-Dev] Additions to the functional module
Message-ID: <43aa6ff70601191058y27b1e50ard658f1a103d52b67@mail.gmail.com>

Hello,

I just submitted patch #1410119, adding foldl and foldr functions to
Modules/functionalmodule.c (plus docs and tests in the appropriate
places).

If there's interest from the higher-ups, I'm interested in adding
numerous other functional programming tools to this module. In
addition to C implementations, I would also provide documentation and
further additions to test/test_functional.py to cover the new
functions.

Also, I've started a project to offer a pure-python implementation of
the C-language functional module, which I plan to keep in sync with
the module in the standard library. If there's any interest in
including something like this in Demo/, I'd be keen to contribute it
as well.

Thanks,
Collin Winter

From fredrik at pythonware.com  Thu Jan 19 19:27:14 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 19 Jan 2006 19:27:14 +0100
Subject: [Python-Dev] str with base
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com><003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
Message-ID: <dqolm4$4l5$1@sea.gmane.org>

Guido van Rossum wrote:

> I think we ought to let this sit for a while and come back to it in a
> few week's time. Is 'base' really the right name? It could just as
> well be considered a conversion in the other direction.

the same applies to hex and oct, of course.

as for base itself, I'm more concerned about the google product place-
ment here.  what's next?  a froogle builtin?

</F>




From guido at python.org  Thu Jan 19 20:12:38 2006
From: guido at python.org (Guido van Rossum)
Date: Thu, 19 Jan 2006 11:12:38 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <dqolm4$4l5$1@sea.gmane.org>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
Message-ID: <ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>

On 1/19/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> Guido van Rossum wrote:
>
> > I think we ought to let this sit for a while and come back to it in a
> > few week's time. Is 'base' really the right name? It could just as
> > well be considered a conversion in the other direction.
>
> the same applies to hex and oct, of course.

Right. And this is not a hypothetical issue either -- in Perl, hex and
oct *do* work the other way I believe. More reasons to get rid of
these in Python 3000. Perhaps we should also get rid of hex/oct
lterals?

> as for base itself, I'm more concerned about the google product place-
> ment here.  what's next?  a froogle builtin?

The default __import__ will use Google Code to locate an appropriate
module to import instead of restricting itself to the boring and
predictable sys.path.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Thu Jan 19 20:13:13 2006
From: aahz at pythoncraft.com (Aahz)
Date: Thu, 19 Jan 2006 11:13:13 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <e8bf7a530601191024l505e2228r90f048b27f17be74@mail.gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<e8bf7a530601191024l505e2228r90f048b27f17be74@mail.gmail.com>
Message-ID: <20060119191313.GA25544@panix.com>

On Thu, Jan 19, 2006, Jeremy Hylton wrote:
>
> I'm not sure I believe this should be a builtin.  I think the
> threshold for new builtins ought to be nearly as high as the threshold
> for new keywords.  Or the proposer ought to make an argument about
> what the function should not go in a module.

The way I'd put it, any function that wants to go in builtins should
require a formal PEP.

And in case it isn't clear, I'm +1 on deprecating oct()/hex() (or moving
them into another module as convenience functions for base() -- just to
make conversion easier).
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From martin at v.loewis.de  Thu Jan 19 20:14:05 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 19 Jan 2006 20:14:05 +0100
Subject: [Python-Dev] New PEP: Using ssize_t as the index type
In-Reply-To: <ee2a432c0601182229w7523ce41nab44a81e4e2329bc@mail.gmail.com>
References: <43B3ECEE.9060300@v.loewis.de>	
	<20051229173041.GA19575@code1.codespeak.net>	
	<43B50B64.20904@v.loewis.de>	
	<20060105194538.GB11736@code1.codespeak.net>	
	<43BD8464.4060801@v.loewis.de> <43C37C02.8030005@egenix.com>
	<ee2a432c0601182229w7523ce41nab44a81e4e2329bc@mail.gmail.com>
Message-ID: <43CFE4FD.7050108@v.loewis.de>

Neal Norwitz wrote:
> Other ideas?

We could rename essential function symbols, like PyInitModule4
(to, say, PyInitModule4b).

Regards,
Martin

From Scott.Daniels at Acm.Org  Thu Jan 19 20:59:52 2006
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Thu, 19 Jan 2006 11:59:52 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <003b01c61b22$0bbfe040$e221cb97@oemcomputer>
References: <e8bf7a530601162047ud296dffo8f2315be674376a2@mail.gmail.com>
	<003b01c61b22$0bbfe040$e221cb97@oemcomputer>
Message-ID: <dqor35$roq$1@sea.gmane.org>

Raymond Hettinger wrote:
> That suggests that it would be better to simply add an int method:
>      x.convert_to_base(7)

I'd suggest allowing:

     x.convert_to_base('0123456')

Where the (str or unicode) argument is the list of digits in order.
This would allow easy converting to base-64 and other weird formats, as
well as providing decimal conversion into some unicode number ranges
outside the ASCII group.

--Scott David Daniels
Scott.Daniels at Acm.Org


From jeremy at alum.mit.edu  Thu Jan 19 21:13:33 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu, 19 Jan 2006 12:13:33 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <20060119191313.GA25544@panix.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<e8bf7a530601191024l505e2228r90f048b27f17be74@mail.gmail.com>
	<20060119191313.GA25544@panix.com>
Message-ID: <e8bf7a530601191213n6a4ce34ctc106fc89a42b81f5@mail.gmail.com>

On 1/19/06, Aahz <aahz at pythoncraft.com> wrote:
> On Thu, Jan 19, 2006, Jeremy Hylton wrote:
> >
> > I'm not sure I believe this should be a builtin.  I think the
> > threshold for new builtins ought to be nearly as high as the threshold
> > for new keywords.  Or the proposer ought to make an argument about
> > what the function should not go in a module.
>
> The way I'd put it, any function that wants to go in builtins should
> require a formal PEP.

I'm suggesting a criterion for evaluating the choice of builtin vs.
module, not making a suggestion about the process.

Jeremy

From bob at redivi.com  Thu Jan 19 21:44:09 2006
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 19 Jan 2006 12:44:09 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
Message-ID: <6E62A903-3CFF-4CCA-8459-F0A966E2FF73@redivi.com>


On Jan 19, 2006, at 11:12 AM, Guido van Rossum wrote:

> On 1/19/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
>> Guido van Rossum wrote:
>>
>>> I think we ought to let this sit for a while and come back to it  
>>> in a
>>> few week's time. Is 'base' really the right name? It could just as
>>> well be considered a conversion in the other direction.
>>
>> the same applies to hex and oct, of course.
>
> Right. And this is not a hypothetical issue either -- in Perl, hex and
> oct *do* work the other way I believe. More reasons to get rid of
> these in Python 3000. Perhaps we should also get rid of hex/oct
> lterals?

In Perl, hex(n) is like int(n, 16) and oct(n) is like int(n, 8) --  
but they "try very hard" to make sense out of the given scalar (e.g.  
more like int(n, 0) with a suggestion for base).

$ perl -e 'print hex("12") . " " . oct("12") . " " . oct("0x12") . "  
" . hex("fubar")'
18 10 18 15

If you notice, oct("0x12") gives the hexadecimal result, and the  
functions will try and make a value even out of garbage data.

In Ruby, you have the optional radix argument to_s and to_i, where  
to_i will just return 0 for invalid values.  They will take any radix  
from 2..36.

$ irb
irb(main):001:0> 12.to_s
=> "12"
irb(main):002:0> 12.to_s(16)
=> "c"
irb(main):003:0> 12.to_s(8)
=> "14"
irb(main):004:0> "12".to_i(8)
=> 10
irb(main):005:0> "12".to_i(16)
=> 18
irb(main):006:0> "0x12".to_i(16)
=> 18
irb(main):007:0> "0x12".to_i(8)
=> 0
irb(main):008:0> "0x12".to_i
=> 0
irb(main):009:0> "fubar".to_i
=> 0
irb(main):010:0> "fubar".to_i(36)
=> 26608563

-bob


From ncoghlan at gmail.com  Thu Jan 19 21:56:23 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 20 Jan 2006 06:56:23 +1000
Subject: [Python-Dev] str with base
In-Reply-To: <ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
Message-ID: <43CFFCF7.1070302@gmail.com>

Guido van Rossum wrote:
> On 1/19/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
>> Guido van Rossum wrote:
>>
>>> I think we ought to let this sit for a while and come back to it in a
>>> few week's time. Is 'base' really the right name? It could just as
>>> well be considered a conversion in the other direction.
>> the same applies to hex and oct, of course.
> 
> Right. And this is not a hypothetical issue either -- in Perl, hex and
> oct *do* work the other way I believe. More reasons to get rid of
> these in Python 3000. Perhaps we should also get rid of hex/oct
> lterals?

I'm not aware of anyone that would miss octal literals, but there are plenty 
of hardware weenies like me that would find "int("DEAD", 16)" less convenient 
than "0xDEAD". Python is a bit too heavyweight for a lot of embedded work, but 
its *great* for writing host-based test harnesses.

I quite like the suggestion of using 'math.base' rather than a builtin, but 
there are still issues to be figured out there:
   - the math module is currently a thin wrapper around C's "math.h". Do we 
really want to change that by adding more methods?
   - is 'base' the right name?
   - should we allow a "digits" argument, or just the radix argument?

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From steve at holdenweb.com  Thu Jan 19 22:18:08 2006
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 19 Jan 2006 21:18:08 +0000
Subject: [Python-Dev] str with base
In-Reply-To: <43CFFCF7.1070302@gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>	<dqolm4$4l5$1@sea.gmane.org>	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
	<43CFFCF7.1070302@gmail.com>
Message-ID: <dqovm7$g9o$1@sea.gmane.org>

Nick Coghlan wrote:
> Guido van Rossum wrote:
> 
>>On 1/19/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
>>
>>>Guido van Rossum wrote:
>>>
>>>
>>>>I think we ought to let this sit for a while and come back to it in a
>>>>few week's time. Is 'base' really the right name? It could just as
>>>>well be considered a conversion in the other direction.
>>>
>>>the same applies to hex and oct, of course.
>>
>>Right. And this is not a hypothetical issue either -- in Perl, hex and
>>oct *do* work the other way I believe. More reasons to get rid of
>>these in Python 3000. Perhaps we should also get rid of hex/oct
>>lterals?
> 
> 
> I'm not aware of anyone that would miss octal literals, but there are plenty 
> of hardware weenies like me that would find "int("DEAD", 16)" less convenient 
> than "0xDEAD". Python is a bit too heavyweight for a lot of embedded work, but 
> its *great* for writing host-based test harnesses.
> 
> I quite like the suggestion of using 'math.base' rather than a builtin, but 
> there are still issues to be figured out there:
>    - the math module is currently a thin wrapper around C's "math.h". Do we 
> really want to change that by adding more methods?
>    - is 'base' the right name?
>    - should we allow a "digits" argument, or just the radix argument?
> 
Another possibility, since Python 3 can break backward compatibility: we 
could take a page out of Icon's book and use an "rN" suffix for 
non-decimal literals.

     23 == 27r8 == 17r16

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From theller at python.net  Thu Jan 19 22:24:08 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 19 Jan 2006 22:24:08 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <psms7b34.fsf@python.net>
	<ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>
	<43CC17FB.5020904@v.loewis.de>
Message-ID: <ek33ud1j.fsf@python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> Guido van Rossum wrote:
>> On 1/16/06, Thomas Heller <theller at python.net> wrote:
>> 
>>>It looks like we need a pronouncement now.
>> 
>> 
>> Sorry. It appeared to me that there was general agreement to using a
>> strongly worded warning in the docs, so I tuned out of the rest of the
>> discussion. So for the record, I'm fine with that.
>

ctypes includes libffi.  libffi is build with GNU tools, which would
have to be committed in the python svn repository.

Several of these files are licensed under GPL:

aclocal.m4 config-ml.in config.guess config.sub depcomp ltcf-c.sh
ltconfig missing

AFAICT, these files are only used to *build* libffi, and are not linked
into or have to be distributed with the binaries.

Is it a problem to have these files in the Python source code?

Maybe it would be possible to reimplement the functionality in the
setup.py script, but this is way beyond my expertise.

Thanks,

Thomas


From martin at v.loewis.de  Thu Jan 19 23:11:52 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 19 Jan 2006 23:11:52 +0100
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ek33ud1j.fsf@python.net>
References: <u0cbyfa1.fsf@python.net>	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>	<43C58200.20803@v.loewis.de>
	<psms7b34.fsf@python.net>	<ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>	<43CC17FB.5020904@v.loewis.de>
	<ek33ud1j.fsf@python.net>
Message-ID: <43D00EA8.2070904@v.loewis.de>

Thomas Heller wrote:
> Is it a problem to have these files in the Python source code?

I would think so, yes. Including them in the Python distribution,
without licensing the entire Python distribution under GPL, would be
a GPL violation.

The actual libffi appears to have a fairly liberal license
(also it still requires to include the LICENSE file, atleast
in the source distribution).

Importing the original source in /external would be no problem
(AFAICT), as long as you then delete them during/after the svn cp.

> Maybe it would be possible to reimplement the functionality in the
> setup.py script, but this is way beyond my expertise.

Right. Given the large number of people that want to see this happen
(not necessarily on python-dev, but on comp.lang.python), I'm sure
there is somebody who can come up with a build process.

If nobody volunteers to do the actual work, I have no problems with
dropping the idea for now. I wouldn't have any problems with just
including a Windows port, either.

Regards,
Martin

From foom at fuhm.net  Thu Jan 19 23:31:25 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 19 Jan 2006 17:31:25 -0500
Subject: [Python-Dev] Include ctypes into core Python?
In-Reply-To: <ek33ud1j.fsf@python.net>
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <psms7b34.fsf@python.net>
	<ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>
	<43CC17FB.5020904@v.loewis.de> <ek33ud1j.fsf@python.net>
Message-ID: <03C7403A-6B16-4727-B2C6-5FE133E69734@fuhm.net>


On Jan 19, 2006, at 4:24 PM, Thomas Heller wrote:

>
> Several of these files are licensed under GPL:
>
> aclocal.m4 config-ml.in config.guess config.sub depcomp ltcf-c.sh
> ltconfig missing
>

Are you sure? The copies of aclocal.m4 and config-ml.in both disagree  
with you. aclocal seems to have a completely liberal license, and  
config-ml has a "whatever the license of the program it's building"  
license.

James

aclocal.m4::
# generated automatically by aclocal 1.7.3 -*- Autoconf -*-

# Copyright (C) 1996, 1997, 1998, 1999, 2000, 2001, 2002
# Free Software Foundation, Inc.
# This file is free software; the Free Software Foundation
# gives unlimited permission to copy and/or distribute it,
# with or without modifications, as long as this notice is preserved.

# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY, to the extent permitted by law; without
# even the implied warranty of MERCHANTABILITY or FITNESS FOR A
# PARTICULAR PURPOSE.

# libtool.m4 - Configure libtool for the host system. -*-Autoconf-*-

config-ml.in::
# Configure fragment invoked in the post-target section for subdirs
# wanting multilib support.
#
# Copyright (C) 1995, 1997, 1998, 1999, 2000, 2001, 2002, 2003
#   Free Software Foundation, Inc.
#
# This file is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2 of the License, or
# (at your option) any later version.
#
# This program is distributed in the hope that it will be useful,
# but WITHOUT ANY WARRANTY; without even the implied warranty of
# MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
# GNU General Public License for more details.
#
# You should have received a copy of the GNU General Public License
# along with this program; if not, write to the Free Software
# Foundation, Inc., 59 Temple Place - Suite 330,
# Boston, MA 02111-1307, USA.
#
# As a special exception to the GNU General Public License, if you
# distribute this file as part of a program that contains a
# configuration script generated by Autoconf, you may include it under
# the same distribution terms that you use for the rest of that program.


From tim.peters at gmail.com  Fri Jan 20 01:07:19 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 19 Jan 2006 19:07:19 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <43CFFCF7.1070302@gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
	<43CFFCF7.1070302@gmail.com>
Message-ID: <1f7befae0601191607i36bee147i34e96cd764d12a0e@mail.gmail.com>

[Nick Coghlan]
> ...
> I quite like the suggestion of using 'math.base' rather than a builtin, but
> there are still issues to be figured out there:
>    - the math module is currently a thin wrapper around C's "math.h". Do we
> really want to change that by adding more methods?

That's not an issue.  Some math functions go beyond C's (like
2-argument log(), and all flavors of log() returning sensible results
for inputs larger than the largest C double), and some functions
aren't part of C at all (like math.radians()).

A stronger reason to keep it out of `math` is that all functions there
operate on, and return, floats:  it's a bizarre place to put an
integer->string function.

>    - is 'base' the right name?
>    - should we allow a "digits" argument, or just the radix argument?

Add to_base(self, radix=16) as a new numeric method.  End of problem ;-)

From thomas at xs4all.net  Fri Jan 20 01:17:14 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 20 Jan 2006 01:17:14 +0100
Subject: [Python-Dev] str with base
In-Reply-To: <43CFFCF7.1070302@gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
	<43CFFCF7.1070302@gmail.com>
Message-ID: <20060120001714.GY18916@xs4all.nl>

On Fri, Jan 20, 2006 at 06:56:23AM +1000, Nick Coghlan wrote:

> I'm not aware of anyone that would miss octal literals,

Except anyone who uses os.chmod. I would be mighty sad if we removed octal
and hexadecimal literals for 'cleanliness' reasons alone.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From tim.peters at gmail.com  Fri Jan 20 02:07:43 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 19 Jan 2006 20:07:43 -0500
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <43CBC95F.3020907@core-sdi.com>
References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net>
	<43C7BB1F.3000904@corest.com> <43CBC95F.3020907@core-sdi.com>
Message-ID: <1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com>

[Gabriel Becedillas]
> Can anybody tell me if the patch I suggested is ok ?
> That will be to add the following code at the end of PyThreadState_Delete:
>
> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
>      PyThread_delete_key_value(autoTLSkey);

It needs a little work, but I think it should be fine, and people tend
to listen to me on issues like this ;-)

The reason it can't work as-is is shallow:  autoTLSkey doesn't exist
unless Python is compiled with WITH_THREAD defined.  So you need to
wrap that inside an "#ifdef WITH_THREAD" block; or add it similarly to
tstate_delete_common(), and remove it from
PyThreadState_DeleteCurrent().

The primary reason people find this so confusing is because it is :-).
 In the dim mists of history, neither multiple interpreters nor
PyThreadState_DeleteCurrent() existed.  While multiple interpreters do
exist now, they're poorly understood, and the PEP that introduced the
GIL state API explicitly disavowed any responsibility for the new API
working at all with multiple interpreters:

    http://www.python.org/peps/pep-0311.html

    This proposal identifies a solution for extension authors with
    complex multi-threaded requirements, but that only require a
    single "PyInterpreterState".  There is no attempt to cater for
    extensions that require multiple interpreter states.  At the time
    of writing, no extension has been identified that requires
    multiple PyInterpreterStates, and indeed it is not clear if that
    facility works correctly in Python itself.

In a parallel development, thread races during Python shutdown were
discovered that could lead to segfaults, and
PyThreadState_DeleteCurrent() was introduced to repair those.  As a
result, it so happens that core Python never uses the original
PyThreadState_Delete() anymore, except when Py_NewInterpreter() has to
throw away the brand new thread state it created because it turns out
it can't create a new interpreter.

Since core Python never calls Py_NewInterpreter() or
PyThreadState_Delete(), you're in an intersection of areas no current
Python developer even thinks about, let alone tests.  But by the same
token, _because_ it's unused, there's nothing you can do to
PyThreadState_Delete() that can hurt core Python either.  That's why
people should be more encouraging here ;-)  It certainly makes sense
to me that if a thread state is going away, any record of it in the
auto-GIL-state machinery must also go away.

> Thank you very much.

Thank you!  If you put a patch on SourceForge, I'll probably be happy
to check it in.

From bob at redivi.com  Fri Jan 20 03:11:19 2006
From: bob at redivi.com (Bob Ippolito)
Date: Thu, 19 Jan 2006 18:11:19 -0800
Subject: [Python-Dev] str with base
In-Reply-To: <20060120001714.GY18916@xs4all.nl>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
	<43CFFCF7.1070302@gmail.com> <20060120001714.GY18916@xs4all.nl>
Message-ID: <F99F0086-8C6D-47DD-845D-3B554CAD5F5D@redivi.com>


On Jan 19, 2006, at 4:17 PM, Thomas Wouters wrote:

> On Fri, Jan 20, 2006 at 06:56:23AM +1000, Nick Coghlan wrote:
>
>> I'm not aware of anyone that would miss octal literals,
>
> Except anyone who uses os.chmod. I would be mighty sad if we  
> removed octal
> and hexadecimal literals for 'cleanliness' reasons alone.

I have a LOT of code that has hex literals, and a little code with  
oct literals (as you say, just os.chmod).  I'm -1 on removing hex and  
oct, and +0 on adding binary.

As a point of reference, both Perl and Ruby support 0b110 binary  
literal syntax

$ ruby -e 'print 0b110, "\n"'
6
$ perl -e 'print 0b110 . "\n"'
6


-bob


From jason.orendorff at gmail.com  Fri Jan 20 04:41:23 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Thu, 19 Jan 2006 22:41:23 -0500
Subject: [Python-Dev] PEP 343 and __context__()
Message-ID: <bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>

I just noticed that my name is in PEP 343 attached to the idea of the
__context__() method, and I'm slightly queasy over it.

The rationale was to help e.g. decimal.DecimalContext support 'with'. 
Maybe that's a bad idea.

DecimalContext has a few problems.  In code where it matters, every
function you write has to worry about it. (That is, you can't just
write __decimal_context__ = ... at the top of the file and be done
with it, the way you can with, say, __metaclass__.)  And
DecimalContext doesn't fit in with generators.

sys.stdout has similar problems.

It feels like PEP 343 will make people want to follow this model. 
That is, we'll see more global behavior-controlling variables in the
future.  There are grizzlier fates; I just wondered if anyone had
thought of this.

Cheers,
-j

From barry at python.org  Fri Jan 20 05:29:11 2006
From: barry at python.org (Barry Warsaw)
Date: Thu, 19 Jan 2006 23:29:11 -0500
Subject: [Python-Dev] str with base
In-Reply-To: <43CFFCF7.1070302@gmail.com>
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
	<43CFFCF7.1070302@gmail.com>
Message-ID: <1137731351.8802.9.camel@geddy.wooz.org>

On Fri, 2006-01-20 at 06:56 +1000, Nick Coghlan wrote:

> I'm not aware of anyone that would miss octal literals, but there are plenty 
> of hardware weenies like me that would find "int("DEAD", 16)" less convenient 
> than "0xDEAD".

Although octal literals is handy for things like os.chmod().  Unix
weenies shouldn't be totally forgotten in P3K.  I'm also for keeping
hex() and oct() although if you want to move them out of builtins,
that's fine.  +0b1 for binary literals and %b.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060119/2063725b/attachment.pgp 

From ncoghlan at gmail.com  Fri Jan 20 10:21:43 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 20 Jan 2006 19:21:43 +1000
Subject: [Python-Dev] PEP 343 and __context__()
In-Reply-To: <bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>
References: <bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>
Message-ID: <43D0ABA7.4080203@gmail.com>

Jason Orendorff wrote:
> I just noticed that my name is in PEP 343 attached to the idea of the
> __context__() method, and I'm slightly queasy over it.
> 
> The rationale was to help e.g. decimal.DecimalContext support 'with'. 
> Maybe that's a bad idea.
> 
> DecimalContext has a few problems.  In code where it matters, every
> function you write has to worry about it. (That is, you can't just
> write __decimal_context__ = ... at the top of the file and be done
> with it, the way you can with, say, __metaclass__.)

No, you write "decimal.setcontext(...)" instead. You then only need to worry 
if you have multiple threads, and you can even fix that by modifying 
decimal.DefaultContext while the program is still single threaded. The latter 
approach actually works even for a single-threaded program, but should only be 
done in an application script, never in an imported module.

>  And
> DecimalContext doesn't fit in with generators.

It does fit actually - you simply have to remember to restore the original 
context around any invocations of yield.

> sys.stdout has similar problems.
> 
> It feels like PEP 343 will make people want to follow this model. 
> That is, we'll see more global behavior-controlling variables in the
> future.  There are grizzlier fates; I just wondered if anyone had
> thought of this.

Yeah, it came up in response to PJE's suggestion of task-local variables for 
generators. The basic concept we came up with is that if you're writing a 
generator that uses a specific context, remember to save the original and 
restore it around any calls to yield (at least, I came up with the idea and 
no-one objected to the approach [1]).

In the case of decimal.Context:

      def iter_sin(iterable):
         orig_ctx = decimal.getcontext()
         with orig_ctx as ctx:
             ctx.prec += 10
             for r in iterable:
                 y = sin(r) # Very high precision during calculation
                 with orig_ctx:
                     yield +y # Yielded results have normal precision
                 # We get "ctx" back here
         # We get "orig_ctx" back here

Similarly for sys.stdout (only not thread-safe due to the global state):

      def log_yielded_items(iterable, logfile):
         orig_stdout = sys.stdout
         with redirected_stdout(logfile):
             for r in iterable:
                 print "Yielding:", r
                 with redirected_stdout(orig_stdout):
                     yield r # stdout goes back to normal here
                 # stdout is going to the logfile again here
         # And stdout is back to normal at the end


The key properties of a "well-behaved" context are:
   1. Use thread-local state for contexts in order to be multithreading safe
     decimal.Context obeys this, sys.stdout doesn't

   2. Provide a way to retrieve the current state for explicit restoration
     decimal.getcontext() and sys.stdout allow this as shown above

This kind of thing is always going to be a pain, but PEP 343 makes it 
significantly less so.

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-dev/2005-October/057493.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From mwh at python.net  Fri Jan 20 10:48:00 2006
From: mwh at python.net (Michael Hudson)
Date: Fri, 20 Jan 2006 09:48:00 +0000
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com> (Tim
	Peters's message of "Thu, 19 Jan 2006 20:07:43 -0500")
References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net>
	<43C7BB1F.3000904@corest.com> <43CBC95F.3020907@core-sdi.com>
	<1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com>
Message-ID: <2mirsfnsbz.fsf@starship.python.net>

Tim Peters <tim.peters at gmail.com> writes:

> [Gabriel Becedillas]
>> Can anybody tell me if the patch I suggested is ok ?
>> That will be to add the following code at the end of PyThreadState_Delete:
>>
>> if (autoTLSkey && PyThread_get_key_value(autoTLSkey) == tstate)
>>      PyThread_delete_key_value(autoTLSkey);
>
> It needs a little work, but I think it should be fine, and people tend
> to listen to me on issues like this ;-)

Oh good, I've been meaning to scrape together the brain cells to think
about this but PyPy is pulping them as fast as I find them...

> The reason it can't work as-is is shallow:  autoTLSkey doesn't exist
> unless Python is compiled with WITH_THREAD defined.  So you need to
> wrap that inside an "#ifdef WITH_THREAD" block; or add it similarly to
> tstate_delete_common(), and remove it from
> PyThreadState_DeleteCurrent().
>
> The primary reason people find this so confusing is because it is :-).
>  In the dim mists of history, neither multiple interpreters nor
> PyThreadState_DeleteCurrent() existed.  While multiple interpreters do
> exist now, they're poorly understood, and the PEP that introduced the
> GIL state API explicitly disavowed any responsibility for the new API
> working at all with multiple interpreters:
>
>     http://www.python.org/peps/pep-0311.html
>
>     This proposal identifies a solution for extension authors with
>     complex multi-threaded requirements, but that only require a
>     single "PyInterpreterState".  There is no attempt to cater for
>     extensions that require multiple interpreter states.  At the time
>     of writing, no extension has been identified that requires
>     multiple PyInterpreterStates, and indeed it is not clear if that
>     facility works correctly in Python itself.
>
> In a parallel development, thread races during Python shutdown were
> discovered that could lead to segfaults, and
> PyThreadState_DeleteCurrent() was introduced to repair those.  As a
> result, it so happens that core Python never uses the original
> PyThreadState_Delete() anymore, except when Py_NewInterpreter() has to
> throw away the brand new thread state it created because it turns out
> it can't create a new interpreter.

Um, PyThreadState_Delete() is called from zapthreads() is called from
PyInterpreterState_Delete() is called from Py_Finalize()... so I don't
entirely believe you here :)

> Since core Python never calls Py_NewInterpreter() or
> PyThreadState_Delete(), you're in an intersection of areas no current
> Python developer even thinks about, let alone tests.  But by the same
> token, _because_ it's unused, there's nothing you can do to
> PyThreadState_Delete() that can hurt core Python either.  That's why
> people should be more encouraging here ;-)  It certainly makes sense
> to me that if a thread state is going away, any record of it in the
> auto-GIL-state machinery must also go away.

I guess if the patch fixes the original problem, it should go in -- my
uneasiness stems not from worrying that it might do harm, but rather
from the fact that I think it's incomplete.  Maybe I just haven't
thought it through enough.

Also, every time I look at pystate.c, I think of things that could be
done differently or better -- for example, I think it would be sane
and possibly even sensible to only call PyThread_set_key_value() in
_PyGILState_NoteThreadState() if "tstate->interp ==
autoInterpreterState".

>> Thank you very much.
>
> Thank you!  If you put a patch on SourceForge, I'll probably be happy
> to check it in.

That would get me off the "svn blame" hook :)

Cheers,
mwh

-- 
  My first thought was someone should offer Mr. Bush elocution
  lessons.  But he'd probably say he knew how to work them chairs
  already.                            -- Internet Oracularity #1294-01

From thomas.mangin at exa-networks.co.uk  Tue Jan 17 16:07:27 2006
From: thomas.mangin at exa-networks.co.uk (Thomas Mangin)
Date: Tue, 17 Jan 2006 15:07:27 +0000
Subject: [Python-Dev] site triggering a bug in urllib2
Message-ID: <43CD082F.1090606@exa-networks.co.uk>

Hello,

I am contacting the list in the hope that someone will be able to understand 
what I am seeing.

I have hit a bug with python 2.4.2 (on Mandriva 2006) using urllib2.
The code which trigger the bug is as follow..

import urllib2
req = urllib2.Request("http://66.117.37.13/")

# makes no difference ..
req.add_header('Connection', 'close')

handle = urllib2.urlopen(req)
data = handle.read()
print data

using a timeout on the socket does not work neither.

The page is well sent and received (verified with tcpdump) but the read call is 
never returning. strace shows lots of activity on the socket: select constantly 
returning a file descriptor with nothing to read on.

I would like to avoid to start to use timers to kill the connection manually, 
any advise on how to deal with this would be welcome.

Thank you for your time and help.

Thomas
-- 
Exa Networks Limited - UK - AS30740 - www.exa-networks.co.uk
nic-handle : MANG-RIPE   website  : thomas.mangin.me.uk
GPG key ID : 0xFB8B81A1  PGP key  : /pgp.html
Inoc-DBA # : 30740*TOM   Office # : +44 (0) 845 145 1234
-------------- next part --------------
A non-text attachment was scrubbed...
Name: signature.asc
Type: application/pgp-signature
Size: 189 bytes
Desc: OpenPGP digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20060117/d513da05/attachment-0001.pgp 

From connellybarnes at yahoo.com  Tue Jan 17 23:08:36 2006
From: connellybarnes at yahoo.com (Connelly Barnes)
Date: Tue, 17 Jan 2006 14:08:36 -0800 (PST)
Subject: [Python-Dev] timeit module
Message-ID: <20060117220836.52102.qmail@web54303.mail.yahoo.com>


Connelly Barnes wrote:
>> Hi,
>> 
>> Perhaps I am the only one bothered by the timeit
>> module, but it seems poorly designed to me.
>> 
>> First of all, it should use a geometric series with a
>> timeout value to detect how many iterations it should
>> perform.  Currently, the user is required to manually
>> specify the number of iterations (the default is 1
>
>The provision of a default value generally seems to chop the legs from

>your argument that the number of iterations is required.

Suppose we wish to time two functions: f1 and f2.  Function f1 takes
1e-5
seconds to run, and function f2 takes 0.1 seconds to run.  If these
functions
are defined in interactive mode, we can time them with timeit via:

>>> timeit.Timer('f1()', 'from __main__ import
f1').timeit(1000000)/1000000
>>> timeit.Timer('f2()', 'from __main__ import f2').timeit(10)/10

The first line gives the approximate time to run f1 and the second line
gives
the approximate time to run f2.  The number parameter to timeit() must
be
supplied for the second line, since the default 1 million iterations
would
take too long to be practical.  The number parameter to timeit() is
also
supplied for the first line, since it makes the code more robust in
case
the default argument value in timeit() is changed.

My first observation is that this is a lot of contortion to get the
time of
functions f1() and f2().  Something simpler would be nice.  For
example,
using Recipe 440657 this can be simplified to:

>>> pytime.pytime(f1, ())
>>> pytime.pytime(f2, ())

Here the timing routine calculates the number of iterations to make,
and
the timing routine divides by the number of iterations of the supplied
callable.

This is also convenient if you move from a computer to one that is
faster or slower.  If you move to a computer that is 10 times slower,
then the timeit.Timer() code above will take 10 times as long to run.
The usual solution is for the programmer to adjust the number of
iterations as he changes computers (so he doesn't wait around all day
for his benchmarks to run).  Likewise, if one optimizes f2() so that
it runs 100 times faster, one would again have to adjust the number
of iterations in order to gain an accurate timing result (if the total
time spent in the timing routine is less than 1 msec, then the
resulting time for f2() is off by more than 10% on my Windows machine,
which is undesirable, so the number of iterations cannot be made too
small).

Summary:
Module timeit is complicated to use on an existing function, it does
not time the number of seconds to execute 1 iteration of the function
(this is the value one usually wants in benchmarking, or its
reciprocal), and it requires that the end user tweak the number of
iterations as he or she changes between computers with
different CPU speed, or as he or she optimizes code.  It is easy to
make a cleaner module that fixes these problems.  It might be good
to incorporate such a module into Python, due to its general
usefulness.

>> million).  If the user optimizes his or her code, then
>> the number of iterations must be changed.  If the user
>> moves to a slower or faster computer, then the number
>> of iterations must be changed again.  This is
>> annoying.
>> 
>What? The purpose of timeit is to give an approximation to the length
of 
>time to run a specific piece ofcode.
>
>Why must the number of iterations be changed when moving to a slower
or 
>faster computer?
>
>> Secondly, there should be a way to time a callable
>> directly.  That is, without finding the string name of
>> the callable and using a string "import X" statement. 
>> These contortions violate rules #1 and #3 of the Zen
>> of Python.
>> 
>Presumably for benchmarking purposes the function call overhead would
be 
>present for all compaered techniques. Do you mean rules #0 and #2?

I mean "Beautiful is better than ugly" and "Simple is better than
complex."


> > [...]
>regards
>  Steve
>-- 
>Steve Holden       +44 150 684 7255  +1 800 494 3119
>Holden Web LLC                     www.holdenweb.com
>PyCon TX 2006                  www.python.org/pycon/


__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around 
http://mail.yahoo.com 

From t-meyer at ihug.co.nz  Fri Jan 20 12:19:54 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Sat, 21 Jan 2006 00:19:54 +1300
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <56E4F12D-9477-4F58-9D85-3D159B3F1B71@ihug.co.nz>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
	<e04bdf310601090908m4633eeb8t@mail.gmail.com>
	<089A37AE-E802-4903-98A5-33BBA0D6E635@ihug.co.nz>
	<e04bdf310601111514n20e5c63aj@mail.gmail.com>
	<56E4F12D-9477-4F58-9D85-3D159B3F1B71@ihug.co.nz>
Message-ID: <A66B7A2A-C76C-4855-85A4-8D5E1F0074B6@ihug.co.nz>

[Tony Meyer]
> Allowing 'surgical' editing of configuration files, as has been
> proposed many times both here and c.l.p would not require
> ConfigParser to be entirely rewritten (just more extensive
> modification of the write() method).

After writing the summary of this thread, I figured I might as well  
prove this by submitting a patch, so did so:

[ 1410680 ] Add 'surgical editing' to ConfigParser
<http://python.org/sf/1410680>

So there are now at least three open ConfigParser patches, all  
addressing similar things, although not in the same way, and all  
three could actually be applied.

My patch (which includes new unittests and a marked-up doc patch)  
runs through a file and updates it to match the in-memory  
configuration object.  Comments, blank lines, and order are  
preserved.  By default (this can be turned off with a keyword arg)  
new sections/options are added to the file as well (new options in  
existing sections are added at the end of the last instance of that  
section, new sections are added at the end of the file).  These new  
options are added in alphabetical (well, list.sort()) order, so  
ordering is consistent, and if an empty file is 'updated' one ends up  
with the output of write() but sorted.

This is essentially a tidied-up version of the SpamBayes code I wrote  
a few years back.

If anyone is still interested in adding this, feel free to take a  
look at the tracker :)

=Tony.Meyer

From stephen at xemacs.org  Fri Jan 20 13:07:41 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Fri, 20 Jan 2006 21:07:41 +0900
Subject: [Python-Dev] str with base
In-Reply-To: <1137731351.8802.9.camel@geddy.wooz.org> (Barry Warsaw's
	message of "Thu, 19 Jan 2006 23:29:11 -0500")
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<003801c61cc9$a0f73e80$50b02c81@oemcomputer>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
	<43CFFCF7.1070302@gmail.com> <1137731351.8802.9.camel@geddy.wooz.org>
Message-ID: <87y81bhzle.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "BAW" == Barry Warsaw <barry at python.org> writes:

    BAW> Unix weenies shouldn't be totally forgotten in P3K.

Great idea!  Put all this stuff in a "weenie" module.  You can have
weenie.unix and weenie.vms and weenie.unicode, besides the weenie.math
that got all this started.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From param at cs.wisc.edu  Fri Jan 20 15:23:25 2006
From: param at cs.wisc.edu (Paramjit Oberoi)
Date: Fri, 20 Jan 2006 06:23:25 -0800
Subject: [Python-Dev] ConfigParser to save with order
In-Reply-To: <ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
References: <e04bdf310601071017u72ae5b75h@mail.gmail.com>
	<ca471dc20601071710u2228d6f9x11ca7aa0bd99ec29@mail.gmail.com>
Message-ID: <e443ad0e0601200623n6530f26dne8f513a196750fd4@mail.gmail.com>

> I think it's moot unless you also preserve comments. Ideally would be
> something that prserved everything (ordering, blank lines, comments
> etc.) from how it was read in. Modifying a value should keep its
> position. Adding a value should add it to the end of the section it's
> in (unless there are cases where ordering is important to the
> semantics -- are there?).

I wrote some code to do just that sometime back:

    http://www.cs.wisc.edu/~param/software/cfgparse/

It would probably need some cleanup and restructuring to be used, but
it may be a good starting point if configparser is to be rewritten.

-param

From ncoghlan at gmail.com  Fri Jan 20 16:26:19 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Jan 2006 01:26:19 +1000
Subject: [Python-Dev] timeit module
In-Reply-To: <dqg5mf$8mr$1@sea.gmane.org>
References: <20060116052723.31287.qmail@web54314.mail.yahoo.com>
	<dqg5mf$8mr$1@sea.gmane.org>
Message-ID: <43D1011B.20103@gmail.com>

Steve Holden wrote:
> Connelly Barnes wrote:
>> Hi,
>>
>> Perhaps I am the only one bothered by the timeit
>> module, but it seems poorly designed to me.
>>
>> First of all, it should use a geometric series with a
>> timeout value to detect how many iterations it should
>> perform.  Currently, the user is required to manually
>> specify the number of iterations (the default is 1
> 
> The provision of a default value generally seems to chop the legs from 
> your argument that the number of iterations is required.

 From the command line, timeit autocalibrates to use the smallest power of ten 
that results in taking at least 0.2 seconds to execute all iterations.

IMO, making this the default behaviour of the timer objects themselves would 
be *much* more useful behaviour than the current default to 10e6 iterations. 
I've been caught a few times hammering on Ctrl-C when I realised that I'd just 
told the interpreter to go do a million iterations of something that took a 
few seconds for each attempt.

>> million).  If the user optimizes his or her code, then
>> the number of iterations must be changed.  If the user
>> moves to a slower or faster computer, then the number
>> of iterations must be changed again.  This is
>> annoying.
>>
> What? The purpose of timeit is to give an approximation to the length of 
> time to run a specific piece ofcode.
> 
> Why must the number of iterations be changed when moving to a slower or 
> faster computer?

Because the default of 10e6 may be entirely inappropriate depending on how 
long an individual iteration is. If a single iteration takes a microsecond, 
great, but if it takes a second, better send out for pizza (or maybe go for a 
trip around the world in the 4 months that sucker is going to take to execute!)

If Timer was changed as follows, it would "just work", even when each 
iteration took a few seconds (6 seconds per iteration would give 1 minute for 
each execution of the timeit method):

class Timer:
     def __init__(self, stmt="pass", setup="pass", timer=default_timer):
         """Constructor.  See class doc string."""
         # ... existing constructor
         self.default_number = None

     def timeit(self, number=None): # Note changed default
         if number is None:
             if self.default_number is None:
                 self.default_number = self.calibrate()
             number = self.default_number
         # ... existing timeit method

     def calibrate(self): # New method, taken from script code at end of module
         # find smallest power of 10 >= 10 so that 0.2 seconds <= total time
         # capped at 10 billion (10e10) iterations, as reaching that limit
         # implies fewer than 20 picoseconds (2e-11) per iteration (i.e. Fast!)
         for i in range(1, 10):
             number = 10**i
             x = self.timeit(number)
             if x >= 0.2:
                 break
         self.default_number = number


>> Secondly, there should be a way to time a callable
>> directly.  That is, without finding the string name of
>> the callable and using a string "import X" statement. 
>> These contortions violate rules #1 and #3 of the Zen
>> of Python.
>>
> Presumably for benchmarking purposes the function call overhead would be 
> present for all compaered techniques. Do you mean rules #0 and #2?

Timing an existing function really is a pain - timeit expects source code it 
can plug into a code template, so timing an existing function from an 
interactive session isn't as easy as it could be (you have to figure out a way 
to give strings to Timer that it can plug into its string template, rather 
than just giving it the callable directly).

A subclass would deal with that quite handily:

class CallTimer(Timer):
     # Use lambda or functional.partial to pass arguments to the callable
     def __init__(self, callable, timer=default_timer):
         def inner(_it, _timer):
             _t0 = _timer()
             for _i in _it:
                 callable()
             _t1 = _timer()
             return _t1 - _t0
         self.timer = timer
         self.inner = inner
         self.src = None

     # Using a real function, so leave linecache alone when printing exceptions
     def print_exc(self, file=None):
         import traceback
         traceback.print_exc(file=file)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Fri Jan 20 17:08:08 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 20 Jan 2006 11:08:08 -0500
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <2mirsfnsbz.fsf@starship.python.net>
References: <43C68823.3000105@corest.com> <2mwth4qw3q.fsf@starship.python.net>
	<43C7BB1F.3000904@corest.com> <43CBC95F.3020907@core-sdi.com>
	<1f7befae0601191707q57552dc8lb75c90aa30e2924a@mail.gmail.com>
	<2mirsfnsbz.fsf@starship.python.net>
Message-ID: <1f7befae0601200808x7ba224dft44e8e4e64feab9f2@mail.gmail.com>

[Tim]
>> ...
>> As a result, it so happens that core Python never uses the original
>> PyThreadState_Delete() anymore, except when Py_NewInterpreter() has
>> to throw away the brand new thread state it created because it turns out
>> it can't create a new interpreter.

[Michael]
> Um, PyThreadState_Delete() is called from zapthreads() is called from
> PyInterpreterState_Delete() is called from Py_Finalize()... so I don't
> entirely believe you here :)

That's only because I was wrong ;-)  Thanks!  That use was in fact
half the cause of Python's segfaulting shutdown races, so it was
exceptionally stupid of me to miss that one.

>> Since core Python never calls Py_NewInterpreter() or
>> PyThreadState_Delete(), you're in an intersection of areas no current
>> Python developer even thinks about, let alone tests.  But by the same
>> token, _because_ it's unused, there's nothing you can do to
>> PyThreadState_Delete() that can hurt core Python either.  That's why
>> people should be more encouraging here ;-)  It certainly makes sense
>> to me that if a thread state is going away, any record of it in the
>> auto-GIL-state machinery must also go away.

> I guess if the patch fixes the original problem, it should go in -- my
> uneasiness stems not from worrying that it might do harm, but rather
> from the fact that I think it's incomplete.  Maybe I just haven't
> thought it through enough.

Oh, I don't know whether it's "complete".  I don't really understand
multiple interpreters, have no idea why the OP is calling
PyThreadState_Delete(), and fully expect that nobody is going to add
tests to Python to check any of this.  But Gabriel showed real skill
at tracking down problems in this area, so if he creates new problems
for himself by doing this (who else plays in this intersection? 
nobody I know of), I'm sure he'll figure them out ;-)

For example, it seems possible that he'll hit the same kinds of
segfaulting shutdown races the Python core used to suffer.  If a
thread T releases the GIL, and goes on to delete its own thread state
via PyThreadState_Delete(T's_thread_state), that _is_ a race with
Python shutdown's zapthreads trying to delete the same thing.  That's
why PyThreadState_DeleteCurrent(void) was introduced, and any call to
PyThreadState_Delete() is suspect on this count.

> Also, every time I look at pystate.c, I think of things that could be
> done differently or better -- for example, I think it would be sane
> and possibly even sensible to only call PyThread_set_key_value() in
> _PyGILState_NoteThreadState() if "tstate->interp ==
> autoInterpreterState".

I joined Mark Hammond in ignoring the possibility of multiple
interpreters when this stuff went in.  The best thing to do with
untested gimmicks without an audience willing to work on them is to
remove them.  If you want to "rehabilitate" multiple interpreters
here, the best first step would be to write an addendum to PEP 311
documenting an intended design.  The GIL state API is documented well
enough on its own, but the _intended_ semantics when mixing threads
with multiple interpreters is clear as mud.  PEP 311 asked "but who
cares?", and nobody answered.

From pje at telecommunity.com  Fri Jan 20 18:48:19 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 20 Jan 2006 12:48:19 -0500
Subject: [Python-Dev] PEP 343 and __context__()
In-Reply-To: <43D0ABA7.4080203@gmail.com>
References: <bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>
	<bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060120123739.02521d00@mail.telecommunity.com>

At 07:21 PM 01/20/2006 +1000, Nick Coghlan wrote:
>Yeah, it came up in response to PJE's suggestion of task-local variables for
>generators. The basic concept we came up with is that if you're writing a
>generator that uses a specific context, remember to save the original and
>restore it around any calls to yield (at least, I came up with the idea and
>no-one objected to the approach [1]).

That's only because Guido shut down the thread.  :)

For the use cases I had in mind, your approach simply doesn't scale, 
because it requires every yield-point to have global knowledge of all 
"changes to global state" that may have occurred somewhere above it in the 
coroutine call stack.  The only reason I didn't point this out is that 
Guido had asked for a postponement of the discussion.

Since then, I've begun implementing a library that accomodates such use 
cases, as well as a wide variety of other context-management use 
cases.  Source code is at http://svn.eby-sarna.com/Contextual/ and the docs 
are at http://svn.eby-sarna.com/Contextual/context.txt?view=markup

The library uses the current draft of PEP 343 as the basis for 
implementation, and includes a few helper functions that can be used to 
emulate the "with:" statement in Python 2.4 so that it can be used and 
tested.  So far, my impression is that the current 343 spec is at least 
adequate, and maybe even elegant.


From guido at python.org  Fri Jan 20 19:17:46 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Jan 2006 10:17:46 -0800
Subject: [Python-Dev] yield back-and-forth?
Message-ID: <ca471dc20601201017k1e14e0e3t34070e0576129b3@mail.gmail.com>

The discussion about PEP 343 reminds me of the following. Bram Cohen
pointed out in private email that, before PEP 342, there wasn't a big
need for a shortcut to pass control to a "sub-generator" because the
following for-loop works well enough:

 def main_generator():
      ...
      for value in sub_generator():
          yield value

but now that yield can return a value, that value might have to be
passed into sub_generator(), not to mention of exceptions. I'm sure
there's a way to write that (although I haven't found the time to
figure it out) but I expect it to be cumbersome and subtle. I don't
think a helper function can be created to solve this problem, because
the yield syntax is essential.

Bram suggested the following syntax:

  def main_generator():
      ...
      yieldthrough sub_generator()

I'm not keen on that particular keyword, but I do believe a syntactic
solution is needed, if the problem is important enough to be solved.

Thoughts?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Fri Jan 20 19:46:35 2006
From: aahz at pythoncraft.com (Aahz)
Date: Fri, 20 Jan 2006 10:46:35 -0800
Subject: [Python-Dev] site triggering a bug in urllib2
In-Reply-To: <43CD082F.1090606@exa-networks.co.uk>
References: <43CD082F.1090606@exa-networks.co.uk>
Message-ID: <20060120184635.GA12559@panix.com>

On Tue, Jan 17, 2006, Thomas Mangin wrote:
>
> I am contacting the list in the hope that someone will be able to 
> understand what I am seeing.

You'll probably get more help by subscribing and posting to
comp.lang.python.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From tanzer at swing.co.at  Fri Jan 20 19:25:10 2006
From: tanzer at swing.co.at (Christian Tanzer)
Date: Fri, 20 Jan 2006 19:25:10 +0100
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: Your message of "Fri, 20 Jan 2006 10:17:46 PST."
	<ca471dc20601201017k1e14e0e3t34070e0576129b3@mail.gmail.com>
Message-ID: <E1F00wg-0006V6-8c@swing.co.at>


Guido van Rossum <guido at python.org> wrote:

> The discussion about PEP 343 reminds me of the following. Bram Cohen
> pointed out in private email that, before PEP 342, there wasn't a big
> need for a shortcut to pass control to a "sub-generator" because the
> following for-loop works well enough:
>
>  def main_generator():
>       ...
>       for value in sub_generator():
>           yield value

For small values of `well enough`. I sometimes override a generator in
a subclass and it sucks to add the loop just because the derived
generator wants to add a value at the beginning or end.

> but now that yield can return a value, that value might have to be
> passed into sub_generator(), not to mention of exceptions. I'm sure
> there's a way to write that (although I haven't found the time to
> figure it out) but I expect it to be cumbersome and subtle. I don't
> think a helper function can be created to solve this problem, because
> the yield syntax is essential.
>
> Bram suggested the following syntax:
>
>   def main_generator():
>       ...
>       yieldthrough sub_generator()
>
> I'm not keen on that particular keyword, but I do believe a syntactic
> solution is needed, if the problem is important enough to be solved.

How about:

  def main_generator():
      ...
      yield * sub_generator()

Ducking-ly yrs,

-- 
Christian Tanzer                                    http://www.c-tanzer.at/


From pje at telecommunity.com  Fri Jan 20 20:05:30 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 20 Jan 2006 14:05:30 -0500
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <ca471dc20601201017k1e14e0e3t34070e0576129b3@mail.gmail.com
 >
Message-ID: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>

At 10:17 AM 01/20/2006 -0800, Guido van Rossum wrote:
>The discussion about PEP 343 reminds me of the following. Bram Cohen
>pointed out in private email that, before PEP 342, there wasn't a big
>need for a shortcut to pass control to a "sub-generator" because the
>following for-loop works well enough:
>
>  def main_generator():
>       ...
>       for value in sub_generator():
>           yield value
>
>but now that yield can return a value, that value might have to be
>passed into sub_generator(), not to mention of exceptions. I'm sure
>there's a way to write that (although I haven't found the time to
>figure it out) but I expect it to be cumbersome and subtle. I don't
>think a helper function can be created to solve this problem, because
>the yield syntax is essential.
>
>Bram suggested the following syntax:
>
>   def main_generator():
>       ...
>       yieldthrough sub_generator()
>
>I'm not keen on that particular keyword, but I do believe a syntactic
>solution is needed, if the problem is important enough to be solved.

What's the use case for this?  In the coroutine use case, the PEP 342 
sample trampoline takes care of this.  If you yield a sub-generator (i.e. 
'yield sub_generator()'), the trampoline effectively replaces the parent 
with the child until the child is complete.

So, that leaves only non-coroutine use cases, and I'm having a hard time 
thinking of any where there would be bidirectional communication.


>Thoughts?

If we have to have a syntax, "yield from sub_generator()" seems clearer 
than "yieldthrough", and doesn't require a new keyword.


From janssen at parc.com  Fri Jan 20 20:08:12 2006
From: janssen at parc.com (Bill Janssen)
Date: Fri, 20 Jan 2006 11:08:12 PST
Subject: [Python-Dev] site triggering a bug in urllib2
In-Reply-To: Your message of "Fri, 20 Jan 2006 10:46:35 PST."
	<20060120184635.GA12559@panix.com> 
Message-ID: <06Jan20.110813pst."58633"@synergy1.parc.xerox.com>

Or the Web-SIG mailing list.

Bill

From guido at python.org  Fri Jan 20 20:19:30 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Jan 2006 11:19:30 -0800
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
Message-ID: <ca471dc20601201119i2296af58m6d91e389b62b2b00@mail.gmail.com>

On 1/20/06, Phillip J. Eby <pje at telecommunity.com> wrote:
> At 10:17 AM 01/20/2006 -0800, Guido van Rossum wrote:
> >The discussion about PEP 343 reminds me of the following. Bram Cohen
> >pointed out in private email that, before PEP 342, there wasn't a big
> >need for a shortcut to pass control to a "sub-generator" because the
> >following for-loop works well enough:
> >
> >  def main_generator():
> >       ...
> >       for value in sub_generator():
> >           yield value
> >
> >but now that yield can return a value, that value might have to be
> >passed into sub_generator(), not to mention of exceptions. I'm sure
> >there's a way to write that (although I haven't found the time to
> >figure it out) but I expect it to be cumbersome and subtle. I don't
> >think a helper function can be created to solve this problem, because
> >the yield syntax is essential.
> >
> >Bram suggested the following syntax:
> >
> >   def main_generator():
> >       ...
> >       yieldthrough sub_generator()
> >
> >I'm not keen on that particular keyword, but I do believe a syntactic
> >solution is needed, if the problem is important enough to be solved.
>
> What's the use case for this?  In the coroutine use case, the PEP 342
> sample trampoline takes care of this.  If you yield a sub-generator (i.e.
> 'yield sub_generator()'), the trampoline effectively replaces the parent
> with the child until the child is complete.

That's a rather specialized, subtle and elaborate framework though,
and at this time I believe it isn't planned to be part of Python 2.5
(right?).

> So, that leaves only non-coroutine use cases, and I'm having a hard time
> thinking of any where there would be bidirectional communication.

Any other use of generators where the return value of yield is used;
as soon as you do this you may later want to refactor the code to to
include sub-iterators. (There *are*other uses besides the trampoline,
right? :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From edcjones at comcast.net  Fri Jan 20 20:51:39 2006
From: edcjones at comcast.net (Edward C. Jones)
Date: Fri, 20 Jan 2006 14:51:39 -0500
Subject: [Python-Dev] yield back-and-forth?
Message-ID: <43D13F4B.2030406@comcast.net>

Guido van Rossum <guido at python.org> wrote:

The discussion about PEP 343 reminds me of the following. Bram Cohen
pointed out in private email that, before PEP 342, there wasn't a big
need for a shortcut to pass control to a "sub-generator" because the
following for-loop works well enough:

 def main_generator():
      ...
      for value in sub_generator():
          yield value

This is an important programming trick. It allows recursive use of 
generators for walking trees, etc. See the source code for os.walk for 
an example.



From jjl at pobox.com  Fri Jan 20 21:28:23 2006
From: jjl at pobox.com (John J Lee)
Date: Fri, 20 Jan 2006 20:28:23 +0000 (GMT Standard Time)
Subject: [Python-Dev] site triggering a bug in urllib2
In-Reply-To: <43CD082F.1090606@exa-networks.co.uk>
References: <43CD082F.1090606@exa-networks.co.uk>
Message-ID: <Pine.WNT.4.64.0601202004320.3928@shaolin>

On Tue, 17 Jan 2006, Thomas Mangin wrote:
[...]
> I have hit a bug with python 2.4.2 (on Mandriva 2006) using urllib2.
> The code which trigger the bug is as follow..
>
> import urllib2
> req = urllib2.Request("http://66.117.37.13/")
>
> # makes no difference ..
> req.add_header('Connection', 'close')
>
> handle = urllib2.urlopen(req)
> data = handle.read()
> print data
>
> using a timeout on the socket does not work neither.

This is a real bug, I think.  I filed a report on the SF bug tracker:

http://python.org/sf/1411097


The problem seems to be the (ab)use of socket._fileobject in urllib2 (I 
believe this was introduced when urllib2 switched to using 
httplib.HTTPConnection).  The purpose of the hack (as commented in 
AbstractHTTPHandler.do_open()) is to provide .readline() and .readlines() 
methods on the response object returned by urllib2.urlopen().

Workaround if you're not using .readline() or .readlines() (against 2.4.2, 
but should apply against current SVN):

--- urllib2.py.orig     Fri Jan 20 20:10:56 2006
+++ urllib2.py  Fri Jan 20 20:12:07 2006
@@ -1006,8 +1006,7 @@
          # XXX It might be better to extract the read buffering code
          # out of socket._fileobject() and into a base class.

-        r.recv = r.read
-        fp = socket._fileobject(r)
+        fp = r.fp

          resp = addinfourl(fp, r.msg, req.get_full_url())
          resp.code = r.status


Not sure yet what the actual problem/cure is...


John

From jason.orendorff at gmail.com  Fri Jan 20 21:34:52 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Fri, 20 Jan 2006 15:34:52 -0500
Subject: [Python-Dev] PEP 343 and __context__()
In-Reply-To: <43D0ABA7.4080203@gmail.com>
References: <bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>
	<43D0ABA7.4080203@gmail.com>
Message-ID: <bb8868b90601201234re002f2cq8b5509efb36b46eb@mail.gmail.com>

On 1/20/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
> Jason Orendorff wrote:
> > DecimalContext has a few problems.  In code where it matters, every
> > function you write has to worry about it. (That is, you can't just
> > write __decimal_context__ = ... at the top of the file and be done
> > with it, the way you can with, say, __metaclass__.)
>
> No, you write "decimal.setcontext(...)" instead.

You seem to be implying these are roughly equal in convenience; I
disagree.  Suppose I have banking.py, in which it's important to use a
particular precision and rounding.  Now I have to put context-munging
code in every single function that banking.py exposes.  And around
every 'yield'.  Even with 'with', that's a lot of extra lines of code.

I'd much prefer to put a one-liner at the top of the file, if it were
possible (...but I don't see how, yet).

Again, none of this is likely to matter--unless you're interleaving
banking and heavy scientific calculations, which I try to avoid.  So,
not a big deal.  Thanks for the response.

> >  And
> > DecimalContext doesn't fit in with generators.
>
> It does fit actually - you simply have to remember to restore the original
> context around any invocations of yield.

Feh!  "Fit" is to "can be made to work with a bit of effort, just
don't forget to follow the rules" as Python is to C++.

-j

From thomas at xs4all.net  Fri Jan 20 21:43:29 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 20 Jan 2006 21:43:29 +0100
Subject: [Python-Dev] [Python-checkins] r42116 -
	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <20060120175503.4C0C71E4002@bag.python.org>
References: <20060120175503.4C0C71E4002@bag.python.org>
Message-ID: <20060120204329.GZ18916@xs4all.nl>

On Fri, Jan 20, 2006 at 06:55:03PM +0100, georg.brandl wrote:
> Author: georg.brandl
> Date: Fri Jan 20 18:55:02 2006
> New Revision: 42116
> 
> Modified:
>    python/branches/release24-maint/Lib/unittest.py
> Log:
> Patch #1388073: Make unittest.TestCase easier to subclass

I don't believe this belongs in 2.4, since it can, actually, break code.
Code that depends on the current situation, _TestCase__attributename.
Fragile code, to be sure, but still. If there were a compelling reason to
backport, I guess it could be hacked to work right-ish, but subclassing
TestCase in this way, while convenient, isn't important enough to warrant
this (IMHO).

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From guido at python.org  Fri Jan 20 21:50:16 2006
From: guido at python.org (Guido van Rossum)
Date: Fri, 20 Jan 2006 12:50:16 -0800
Subject: [Python-Dev] [Python-checkins] r42116 -
	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <20060120204329.GZ18916@xs4all.nl>
References: <20060120175503.4C0C71E4002@bag.python.org>
	<20060120204329.GZ18916@xs4all.nl>
Message-ID: <ca471dc20601201250w1dd7ce49w2b1e93fa51caea72@mail.gmail.com>

Right. This *definitely* is a feature, not a bug.

On 1/20/06, Thomas Wouters <thomas at xs4all.net> wrote:
> On Fri, Jan 20, 2006 at 06:55:03PM +0100, georg.brandl wrote:
> > Author: georg.brandl
> > Date: Fri Jan 20 18:55:02 2006
> > New Revision: 42116
> >
> > Modified:
> >    python/branches/release24-maint/Lib/unittest.py
> > Log:
> > Patch #1388073: Make unittest.TestCase easier to subclass
>
> I don't believe this belongs in 2.4, since it can, actually, break code.
> Code that depends on the current situation, _TestCase__attributename.
> Fragile code, to be sure, but still. If there were a compelling reason to
> backport, I guess it could be hacked to work right-ish, but subclassing
> TestCase in this way, while convenient, isn't important enough to warrant
> this (IMHO).
>
> --
> Thomas Wouters <thomas at xs4all.net>
>
> Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Fri Jan 20 21:53:06 2006
From: barry at python.org (Barry Warsaw)
Date: Fri, 20 Jan 2006 15:53:06 -0500
Subject: [Python-Dev] [Python-checkins] r42116
	-	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <20060120204329.GZ18916@xs4all.nl>
References: <20060120175503.4C0C71E4002@bag.python.org>
	<20060120204329.GZ18916@xs4all.nl>
Message-ID: <1137790386.8802.38.camel@geddy.wooz.org>

On Fri, 2006-01-20 at 21:43 +0100, Thomas Wouters wrote:

> I don't believe this belongs in 2.4, since it can, actually, break code.
> Code that depends on the current situation, _TestCase__attributename.
> Fragile code, to be sure, but still. If there were a compelling reason to
> backport, I guess it could be hacked to work right-ish, but subclassing
> TestCase in this way, while convenient, isn't important enough to warrant
> this (IMHO).

Exactly right.
-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060120/5c18d196/attachment.pgp 

From g.brandl-nospam at gmx.net  Fri Jan 20 22:18:20 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Fri, 20 Jan 2006 22:18:20 +0100
Subject: [Python-Dev] [Python-checkins] r42116 -
	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <1137790386.8802.38.camel@geddy.wooz.org>
References: <20060120175503.4C0C71E4002@bag.python.org>	<20060120204329.GZ18916@xs4all.nl>
	<1137790386.8802.38.camel@geddy.wooz.org>
Message-ID: <dqrk2s$8n0$1@sea.gmane.org>

Barry Warsaw wrote:
> On Fri, 2006-01-20 at 21:43 +0100, Thomas Wouters wrote:
> 
>> I don't believe this belongs in 2.4, since it can, actually, break code.
>> Code that depends on the current situation, _TestCase__attributename.
>> Fragile code, to be sure, but still. If there were a compelling reason to
>> backport, I guess it could be hacked to work right-ish, but subclassing
>> TestCase in this way, while convenient, isn't important enough to warrant
>> this (IMHO).
> 
> Exactly right.

You're right. Next time I'm going to decide in favor of not backporting.
I reverted both checkins, this and the urlparse one.


For 2.5, this would be a good opportunity to add additional schemes
to urlparse. Candidates?

Georg


From pje at telecommunity.com  Fri Jan 20 22:39:13 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri, 20 Jan 2006 16:39:13 -0500
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <ca471dc20601201119i2296af58m6d91e389b62b2b00@mail.gmail.co
 m>
References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
	<5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
Message-ID: <5.1.1.6.0.20060120143818.00a981e0@mail.telecommunity.com>

At 11:19 AM 01/20/2006 -0800, Guido van Rossum wrote:
>(There *are*other uses besides the trampoline,
>right? :-)

It's easy to come up with use cases where you feed data *into* a generator 
(parsers and pipelines, for example).  I just don't know of any 
"simultaneous bidirectional" uses other than in association with a 
coroutine scheduler, or a dedicated pipelining tool.  In both the coroutine 
and pipelining cases, generator composition is an easy job for whatever 
coroutine or pipelining library is in use; it just happens external to the 
generator, perhaps by yielding a special value.


From ncoghlan at gmail.com  Sat Jan 21 03:07:45 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Jan 2006 12:07:45 +1000
Subject: [Python-Dev] PEP 343 and __context__()
In-Reply-To: <bb8868b90601201234re002f2cq8b5509efb36b46eb@mail.gmail.com>
References: <bb8868b90601191941r5c4c99c0pe7b42e84088707c3@mail.gmail.com>	<43D0ABA7.4080203@gmail.com>
	<bb8868b90601201234re002f2cq8b5509efb36b46eb@mail.gmail.com>
Message-ID: <43D19771.1040900@gmail.com>

Jason Orendorff wrote:
> On 1/20/06, Nick Coghlan <ncoghlan at gmail.com> wrote:
>> Jason Orendorff wrote:
>>> DecimalContext has a few problems.  In code where it matters, every
>>> function you write has to worry about it. (That is, you can't just
>>> write __decimal_context__ = ... at the top of the file and be done
>>> with it, the way you can with, say, __metaclass__.)
>> No, you write "decimal.setcontext(...)" instead.
> 
> You seem to be implying these are roughly equal in convenience; I
> disagree.  Suppose I have banking.py, in which it's important to use a
> particular precision and rounding.  Now I have to put context-munging
> code in every single function that banking.py exposes.  And around
> every 'yield'.  Even with 'with', that's a lot of extra lines of code.

Ah, I understand your point better now.

> I'd much prefer to put a one-liner at the top of the file, if it were
> possible (...but I don't see how, yet).

Me neither. The best I can come up with is a "precise" decorator that takes 
care of the context munging. Which would at least reduce the boilerplate to 
"@precise" on functions and "@precise_gen" on generators.

> Again, none of this is likely to matter--unless you're interleaving
> banking and heavy scientific calculations, which I try to avoid.  So,
> not a big deal.  Thanks for the response.

No worries. I agree there's still some awkwardness, but I do think it's an 
improvement on the status quo (where we have the same problems with 
interactions between modules and generators and thread-state, but dealing with 
them can't really be factored out at all).

>>>  And
>>> DecimalContext doesn't fit in with generators.
>> It does fit actually - you simply have to remember to restore the original
>> context around any invocations of yield.
> 
> Feh!  "Fit" is to "can be made to work with a bit of effort, just
> don't forget to follow the rules" as Python is to C++.

I see what you mean. I'm sure we can rely on PJE and others to push the limits 
of with statements and provide feedback on improving their interaction with 
generators in Python 2.6 :)

(although before that we need to bring mwh's patch up to speed with the svn 
trunk. . .)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Jan 21 05:40:29 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Jan 2006 14:40:29 +1000
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
Message-ID: <43D1BB3D.8040005@gmail.com>

Phillip J. Eby wrote:
>> Thoughts?
> 
> If we have to have a syntax, "yield from sub_generator()" seems clearer 
> than "yieldthrough", and doesn't require a new keyword.

Andrew Koenig suggested the same phrasing last year [1], and I liked it then. 
I don't like it any more, though, as I think it is too inflexible, and we have 
a better option available (specifically, stealing "continue with an argument" 
from PEP 340). The following ramblings (try to) explain my reasoning :)

Guido does raise an interesting point. The introduction of "send" and "throw" 
means that the current simple loop approach does not easily allow values to be 
passed down to nested generators, nor does not it correctly terminate nested 
generators in response to an invocation of "throw". Because of the explicit 
for loop, the nested generator only gets cleaned up in response to GC - it 
never sees the exception that occurs in the body of the for loop (at the point 
of the yield expression).

The "yield from iterable" concept could be translated roughly as follows:

   itr = iter(iterable)
   try:
       send_input = itr.send  # Can input values be passed down?
   except AttributeError:
       send_input = None
   try:
       next = itr.next()      # Get the first output
   except StopIteration:
       pass
   else:
       while 1:
           try:
               input = yield next  # yield and get input
           except:
               try:
                  throw_exc = itr.throw  # Can exception be passed down?
               except AttributeError:
                   raise                     # Nope, reraise
               else:
                   throw_exc(sys.exc_info()) # Yep, pass it down
           else:
               try:
                   if send_input is None:
                       if input is not None:
                           raise TypeError("Cannot send input!")
                       next = itr.next()
                   else:
                       next = send_input(input) # Pass input down
               except StopIteration:
                   break

I'm not particularly happy with this, though, as not only is it horribly 
implicit and magical, it's trivial to accidentally break the chain - consider 
what happens if you naively do:
   yield from (x*x for x in sub_generator())

The chain has been broken - the sub generator no longer sees either passed in 
values or thrown exceptions, as the generator expression intercepts them 
without passing them down. Even worse, IMO, is that the syntax is entirely 
inflexible - we have no easy way to manipulate either the results sent from 
the generator, or the input values passed to it.

However, an idea from Guido's PEP 340 helps with the "send" part of the story, 
involving passing an argument to continue:

   def main_generator():
       ...
       for value in sub_generator():
           continue yield value

Here, sub_generator's "send" method would be invoked with the result of the 
call to yield value. Manipulation in either direction (input or output) is 
trivial:

   def main_generator():
       ...
       for value in sub_generator():
           input = yield value*value   # Square the output values
           continue input*input        # Square the input values, too

You could even do odd things like yield each value twice, and then pass down 
pairs of inputs:

   def main_generator():
       ...
       for value in sub_generator():
           continue (yield value), (yield value)

The need to use a "continue" statement eliminates the temptation to use a 
generator expression, and makes it far less likely the downwards connection 
between the main generator and the sub generator will be accidentally broken.

Exception propagation is a different story. What do you want to propagate? All 
exceptions from the body of the for loop? Or just those from the yield statement?

Well, isn't factoring out exception processing part of what PEP 343 is for?

   # Simply make sure the generator is closed promptly
   def main_generator():
       ...
       with closing(sub_generator()) as gen:
           for value in gen:
               continue yield value

   # Or throw the real exception to the nested generator
   class throw_to(object):
       def __init__(self, gen):
           self.gen = gen
       def __enter__(self):
           return self.gen
       def __exit__(self, exc_type, *exc_details):
           if exc_type is not None:
               try:
                   self.gen.throw(exc_type, *exc_details)
               except StopIteration:
                   pass

   def main_generator():
       ...
       with throw_to(sub_generator()) as gen:
           for value in gen:
               continue yield value

   # We can even limit the propagated exceptions to those
   # from the outside world and leave the rest alone
   def main_generator():
       ...
       gen = sub_generator()
       for value in gen:
           with throw_to(gen):
               input = yield value
           continue input


Thanks to Jason's other thread, I even have a hypothetical use case for all 
this. Specifically, "generator decorators", that manipulate the state that 
holds while executing the generator. Suppose I have a number of generators 
that I want to work in high precision decimal, but yield results with normal 
precision. Doing this dance manually in every generator would be a pain, so 
let's use a decorator:

      def precise_gen(gen_func):
         def wrapper(*args, **kwds):
             orig_ctx = decimal.getcontext()
             with orig_ctx as ctx:
                 ctx.prec = HIGH_PRECISION
                 gen = gen_func(*args, **kwds)
                 for val in gen:
                     with orig_ctx:
                         val = +val
                         try:
                             yield val
                         except:
                             gen.throw() # make it default to sys.exc_info()
         wrapper.__name__ = gen_func.__name__
         wrapper.__dict__ = gen_func.__dict__
         wrapper.__doc__ = gen_func.__doc__
         return wrapper

So far, so good (although currently, unlike a raise statement, gen.throw() 
doesn't automatically default to sys.exc_info()). Tough luck, however, if you 
want to use this on a generator that accepts input values - those inputs will 
get dropped on the floor by the wrapper, and the wrapped generator will never 
see them.

"yield from" wouldn't help in the least, because there are other things going 
on in the for loop containing the yield statement. 'continue with argument', 
however, would work just fine:

      def precise_gen(gen_func):
         def wrapper(*args, **kwds):
             orig_ctx = decimal.getcontext()
             with orig_ctx as ctx:
                 ctx.prec = HIGH_PRECISION
                 gen = gen_func(*args, **kwds)
                 for val in gen:
                     with orig_ctx:
                         val = +val
                         try:
                             continue yield val
                         except:
                             gen.throw() # make it default to sys.exc_info()
         wrapper.__name__ = gen_func.__name__
         wrapper.__dict__ = gen_func.__dict__
         wrapper.__doc__ = gen_func.__doc__
         return wrapper

Now an arbitrary generator can be decorated with "@precise_gen", and it's 
internal operations will be carried out with high precision, while any interim 
results will be yielded using the caller's precision and the caller's rounding 
rules.

Cheers,
Nick.

[1] http://mail.python.org/pipermail/python-dev/2005-October/057410.html

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ncoghlan at gmail.com  Sat Jan 21 05:55:19 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 21 Jan 2006 14:55:19 +1000
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <43D1BB3D.8040005@gmail.com>
References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
	<43D1BB3D.8040005@gmail.com>
Message-ID: <43D1BEB7.10202@gmail.com>

Nick Coghlan wrote:
> Exception propagation is a different story. What do you want to propagate? All 
> exceptions from the body of the for loop? Or just those from the yield statement?
> 
> Well, isn't factoring out exception processing part of what PEP 343 is for?

>    # We can even limit the propagated exceptions to those
>    # from the outside world and leave the rest alone
>    def main_generator():
>        ...
>        gen = sub_generator()
>        for value in gen:
>            with throw_to(gen):
>                input = yield value
>            continue input

A point I should have made here (but didn't, despite using it in a later 
example) is that the "with" versions don't allow the nested generator to 
suppress the exception.

A small refinement to gen.throw() that makes the first argument optional 
(similar to the raise statement itself) would make it straightforward to allow 
the generator to suppress passed in exceptions:

    def main_generator():
        ...
        gen = sub_generator()
        for value in gen:
            try:
                input = yield value
            except:
                gen.throw() # Default to sys.exc_info(), just like raise
            continue input

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From ark at acm.org  Sat Jan 21 06:55:05 2006
From: ark at acm.org (Andrew Koenig)
Date: Sat, 21 Jan 2006 00:55:05 -0500
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <ca471dc20601201017k1e14e0e3t34070e0576129b3@mail.gmail.com>
Message-ID: <000201c61e4f$3dafaa10$6402a8c0@arkdesktop>

> The discussion about PEP 343 reminds me of the following. Bram Cohen
> pointed out in private email that, before PEP 342, there wasn't a big
> need for a shortcut to pass control to a "sub-generator" because the
> following for-loop works well enough:

>  def main_generator():
>       ...
>       for value in sub_generator():
>           yield value

> but now that yield can return a value, that value might have to be
> passed into sub_generator(), not to mention of exceptions. I'm sure
> there's a way to write that (although I haven't found the time to
> figure it out) but I expect it to be cumbersome and subtle.

It looks to me like continuations are starting to rear their heads...





From aleaxit at gmail.com  Sat Jan 21 06:57:00 2006
From: aleaxit at gmail.com (Alex Martelli)
Date: Fri, 20 Jan 2006 21:57:00 -0800
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <5.1.1.6.0.20060120143818.00a981e0@mail.telecommunity.com>
References: <5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
	<5.1.1.6.0.20060120140021.02565a70@mail.telecommunity.com>
	<5.1.1.6.0.20060120143818.00a981e0@mail.telecommunity.com>
Message-ID: <048D2824-99D0-4E79-8C87-4361CA3EF3C0@gmail.com>


On Jan 20, 2006, at 1:39 PM, Phillip J. Eby wrote:

> At 11:19 AM 01/20/2006 -0800, Guido van Rossum wrote:
>> (There *are*other uses besides the trampoline,
>> right? :-)
>
> It's easy to come up with use cases where you feed data *into* a  
> generator
> (parsers and pipelines, for example).  I just don't know of any
> "simultaneous bidirectional" uses other than in association with a
> coroutine scheduler, or a dedicated pipelining tool.  In both the  
> coroutine

Hmm, what about an iterator of iterators, where the sub-iterators  
need to be advanced until some condition is satisfied, then processed  
from that point onwards (only those subiterators for which some item  
did satisfy the condition).  E.g.:

goodfiles = []
for afile in manyfiles:
   for aline in afile:
     if aline.startswith('*starthere*'):
       goodfiles.append(afile)
       break
for afile in goodfiles: ...

I'm sure we've all met this kind of issue (indeed, standard library  
module fileinput does support this case directly, by wrapping the  
double loop into one yielding lines, and still providing ways to get  
the current file and skipping to the next file).

It might be nice to wrap the general logic (double nested looping &c)  
in a generator leaving the specifics (exactly when to consider a  
subiterator 'good' and what to do in that case) out of it, sort of  
like fileinput does but with more generality than just files and  
lines thereof.  In Python 2.5 it seems that a .send call with a non- 
None argument might be a natural convention for the loop body to tell  
the generator "abandon this subiterator and yield it, then move to  
the next one".  E.g.:

goodfiles = []
allinput = inputfrom(manyfiles)
for aline in allinput:
   if aline.startswith('*starthere*'):
     goodfiles.append(allinput.send(True))
for afile in goodfiles: ...

Perhaps not the greatest of simplifications, but it might be  
generalized to (e.g.) recursive walks with the same client-logic as  
above, while the original (nested for-loops) is pretty rigid.  And  
the inputfrom generator doesn't look too complicated either:

def inputfrom(iterators):
   for iterator in iterators:
     for item in iterator:
       if (yield item) is not None:
         yield iterator
         break

Wouldn't this be considered a reasonable use for the new generator  
features?


Alex


From barry at python.org  Sat Jan 21 17:19:11 2006
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Jan 2006 11:19:11 -0500
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <r7761zxk.fsf@python.net>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<200601182031.54111.anthony@interlink.com.au>
	<871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <CEBF6256-29BE-403F-AD3E-2332607DAD68@python.org>

On Jan 18, 2006, at 8:24 AM, Stephen J. Turnbull wrote:

> The import of the Ghostscript case is that the FSF considers a
> Makefile stanza clearly intended to cause linkage to a GPL library,
> even if optional and supplied by the user, to create a work derived
> from that library.  A "GNU readline"-enabled Python is derived from
> GNU readline, and must be distributed under the GPL or not distributed
> at all.  I assume that the former is not acceptable to Python, and the
> latter is clearly undesirable.

In that case, what I think we ought to do is not add the DP paths  
(i.e. /opt/local) to setup.py specifically to get its readline, but  
instead to pick up any libraries that happen to be in DP in  
preference to those in OSX by default.  If that /happens/ to cause a  
different version of readline to be used as a side-effect, then oops!  
sorry, but so be it.

-Barry


From barry at python.org  Sat Jan 21 17:41:23 2006
From: barry at python.org (Barry Warsaw)
Date: Sat, 21 Jan 2006 11:41:23 -0500
Subject: [Python-Dev] Building on OS X 10.4 fails
In-Reply-To: <CEBF6256-29BE-403F-AD3E-2332607DAD68@python.org>
References: <r7761zxk.fsf@python.net>
	<200601181117.34549.anthony@interlink.com.au>
	<87slrm85w0.fsf@tleepslib.sk.tsukuba.ac.jp>
	<200601182031.54111.anthony@interlink.com.au>
	<871wz57job.fsf@tleepslib.sk.tsukuba.ac.jp>
	<CEBF6256-29BE-403F-AD3E-2332607DAD68@python.org>
Message-ID: <8E8FF935-4D7F-4542-A8EF-70279810099C@python.org>

On Jan 21, 2006, at 11:19 AM, Barry Warsaw wrote:

> In that case, what I think we ought to do is not add the DP paths
> (i.e. /opt/local) to setup.py specifically to get its readline, but
> instead to pick up any libraries that happen to be in DP in
> preference to those in OSX by default.  If that /happens/ to cause a
> different version of readline to be used as a side-effect, then oops!
> sorry, but so be it.

This patch (currently unassigned) does the trick for me:

https://sourceforge.net/tracker/index.php? 
func=detail&aid=1411588&group_id=5470&atid=305470

-Barry


From martin at v.loewis.de  Sat Jan 21 19:37:08 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Sat, 21 Jan 2006 19:37:08 +0100
Subject: [Python-Dev] [Python-checkins] r42116
	-	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <dqrk2s$8n0$1@sea.gmane.org>
References: <20060120175503.4C0C71E4002@bag.python.org>	<20060120204329.GZ18916@xs4all.nl>	<1137790386.8802.38.camel@geddy.wooz.org>
	<dqrk2s$8n0$1@sea.gmane.org>
Message-ID: <43D27F54.70004@v.loewis.de>

Georg Brandl wrote:
> For 2.5, this would be a good opportunity to add additional schemes
> to urlparse. Candidates?

The registered ones:

http://www.iana.org/assignments/uri-schemes

Not sure whether urn parsing would also be desirable:

http://www.iana.org/assignments/urn-namespaces

Regards,
Martin

From fdrake at acm.org  Sat Jan 21 20:23:32 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sat, 21 Jan 2006 14:23:32 -0500
Subject: [Python-Dev]
	=?iso-8859-1?q?=5BPython-checkins=5D_r42116_-=09pyth?=
	=?iso-8859-1?q?on/branches/release24-maint/Lib/unittest=2Epy?=
In-Reply-To: <43D27F54.70004@v.loewis.de>
References: <20060120175503.4C0C71E4002@bag.python.org>
	<dqrk2s$8n0$1@sea.gmane.org> <43D27F54.70004@v.loewis.de>
Message-ID: <200601211423.33402.fdrake@acm.org>

On Saturday 21 January 2006 13:37, Martin v. L?wis wrote:
 > The registered ones:
 >
 > http://www.iana.org/assignments/uri-schemes

I think that these should be supported.

 > Not sure whether urn parsing would also be desirable:
 >
 > http://www.iana.org/assignments/urn-namespaces

The "hdl" scheme already included in the urlparse module is a URN scheme from 
CNRI.  Either we should support URNs, or we should consider deprecating 
support for that one.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From g.brandl-nospam at gmx.net  Sat Jan 21 21:11:53 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sat, 21 Jan 2006 21:11:53 +0100
Subject: [Python-Dev] New Pythondoc by effbot
Message-ID: <dqu4i9$ar9$1@sea.gmane.org>

What Fredrik hacks together there (http://www.effbot.org/lib) is very 
impressive. I especially like the "permalinks" in this style:

http://effbot.org/lib/os.path.join

What I would suggest (for any new doc system) is a "split" view: on the left, 
the normal text, on the right, an area with only the headings, functions, 
example and "see also" links (which preferably stays in sight). This way, you 
always keep the outline in view.

Of course, I wouldn't say no to a user-commenting system, but this would have to 
be moderated.

What I'm also curious about regarding the current docs, why are optional 
arguments in function declarations not written in Python style?

regards,
Georg

PS: Fredrik, if I can help you, please tell me!


From g.brandl-nospam at gmx.net  Sat Jan 21 21:23:11 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sat, 21 Jan 2006 21:23:11 +0100
Subject: [Python-Dev] [Python-checkins] r42116 -
	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <200601211423.33402.fdrake@acm.org>
References: <20060120175503.4C0C71E4002@bag.python.org>	<dqrk2s$8n0$1@sea.gmane.org>
	<43D27F54.70004@v.loewis.de> <200601211423.33402.fdrake@acm.org>
Message-ID: <dqu57f$cfa$1@sea.gmane.org>

Fred L. Drake, Jr. wrote:
> On Saturday 21 January 2006 13:37, Martin v. L?wis wrote:
>  > The registered ones:
>  >
>  > http://www.iana.org/assignments/uri-schemes
> 
> I think that these should be supported.

That's okay, but it may be much work to find out which of them use relative 
paths, fragments, queries and parameters. Is there a list of these features 
somewhere?

Georg


From martin at v.loewis.de  Sat Jan 21 21:59:11 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 21 Jan 2006 21:59:11 +0100
Subject: [Python-Dev] [Python-checkins] r42116
	-	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: <dqu57f$cfa$1@sea.gmane.org>
References: <20060120175503.4C0C71E4002@bag.python.org>	<dqrk2s$8n0$1@sea.gmane.org>	<43D27F54.70004@v.loewis.de>
	<200601211423.33402.fdrake@acm.org> <dqu57f$cfa$1@sea.gmane.org>
Message-ID: <43D2A09F.5060101@v.loewis.de>

Georg Brandl wrote:
> That's okay, but it may be much work to find out which of them use relative 
> paths, fragments, queries and parameters. Is there a list of these features 
> somewhere?

Not that I know of. Collecting a list of missing schemes in the file
might be a good start already.

Regards,
Martin

From guido at python.org  Sun Jan 22 00:20:26 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Jan 2006 15:20:26 -0800
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqu4i9$ar9$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
Message-ID: <ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>

On 1/21/06, Georg Brandl <g.brandl-nospam at gmx.net> wrote:
> What Fredrik hacks together there (http://www.effbot.org/lib) is very
> impressive. I especially like the "permalinks" in this style:
>
> http://effbot.org/lib/os.path.join

Which (despite having "perma" in its name) evaporates and leaves
behind a link to os.path.html#join.

> What I would suggest (for any new doc system) is a "split" view: on the left,
> the normal text, on the right, an area with only the headings, functions,
> example and "see also" links (which preferably stays in sight). This way, you
> always keep the outline in view.

Can you mock that up a bit? I'm somewhat confused about what you're
requesting, and also worried that it would take up too much horizontal
space. (Despite that monitors are wider than tall, horizontal real
estate feels more scarce than vertical, because horizontal scrolling
is such a pain.)

> Of course, I wouldn't say no to a user-commenting system, but this would have to
> be moderated.

Why? If wikipedia can do without moderation (for most pages)  then why
couldn't the Python docs?

> What I'm also curious about regarding the current docs, why are optional
> arguments in function declarations not written in Python style?

I'm assuming you're asking why we use

  foo(a[, b[, c]])

instead of

  foo(a, b=1, c=2)

? I can see several reasons; arguments with default values aren't
necessarily usable keyword arguments (at least not if they
function/method is implemented in C); the default value isn't always
relevant (or is dynamic, or is a huge expression); and square brackets
are the standard way in computer manuals to indicate optional parts of
syntax.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From tjreedy at udel.edu  Sun Jan 22 01:15:14 2006
From: tjreedy at udel.edu (Terry Reedy)
Date: Sat, 21 Jan 2006 19:15:14 -0500
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
Message-ID: <dquiqj$jvh$1@sea.gmane.org>

>> http://effbot.org/lib/os.path.join

On this page, 8 of 30 entries have a 'new in' comment.  For anyone with no 
interest in the past, these constitute noise.  I wonder if for 3.0, the 
timer can be reset and the docs start clean again.  To keep them backwards 
compatible, they would also have to be littered with 'changed in 3.0' and 
'deleted in 3.0' entries.  Better, I think, to refer people to the last 2.x 
docs and a separate 2.x/3.0 changes doc.




From guido at python.org  Sun Jan 22 02:44:35 2006
From: guido at python.org (Guido van Rossum)
Date: Sat, 21 Jan 2006 17:44:35 -0800
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dquiqj$jvh$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dquiqj$jvh$1@sea.gmane.org>
Message-ID: <ca471dc20601211744t3ef965ebtf377483968c57150@mail.gmail.com>

On 1/21/06, Terry Reedy <tjreedy at udel.edu> wrote:
> On this page, 8 of 30 entries have a 'new in' comment.  For anyone with no
> interest in the past, these constitute noise.  I wonder if for 3.0, the
> timer can be reset and the docs start clean again.  To keep them backwards
> compatible, they would also have to be littered with 'changed in 3.0' and
> 'deleted in 3.0' entries.  Better, I think, to refer people to the last 2.x
> docs and a separate 2.x/3.0 changes doc.

(That has nothing to do with Fredrik's efforts of course -- he's just
faithfully copying the annotations from the existing docs.)

If Python 3.0 is indeed the major incompatible release that we
currently expect it will be, I agree that "new in 3.0" and "new in
2.x" annotations make little sense, since so much will be "new in 3.0"
(or "deleted in 3.0"). It's probably better to have a separate
document explaining the differences between 2.x and 3.0.

For the 2.x line, however (and again post-3.0) I think the "new in X" 
annotations are important -- lots of people have to deal with
different Python versions, and it's a lot easier to depend on "new in
X" notations than to manually compare two or more versions of the
docs. (Although you still need to do thorough testing -- like the rest
of the docs, these annotations can't be 100% perfect.)

As far as noise goes, "new in X" is minor compared to all the stuff
that's documented that the average user never needs... :-)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From janssen at parc.com  Sun Jan 22 02:58:14 2006
From: janssen at parc.com (Bill Janssen)
Date: Sat, 21 Jan 2006 17:58:14 PST
Subject: [Python-Dev] [Python-checkins] r42116 -
	python/branches/release24-maint/Lib/unittest.py
In-Reply-To: Your message of "Fri, 20 Jan 2006 13:18:20 PST."
	<dqrk2s$8n0$1@sea.gmane.org> 
Message-ID: <06Jan21.175820pst."58633"@synergy1.parc.xerox.com>

The "data:" scheme would be a good one.

Bill

> Barry Warsaw wrote:
> > On Fri, 2006-01-20 at 21:43 +0100, Thomas Wouters wrote:
> > 
> >> I don't believe this belongs in 2.4, since it can, actually, break code.
> >> Code that depends on the current situation, _TestCase__attributename.
> >> Fragile code, to be sure, but still. If there were a compelling reason to
> >> backport, I guess it could be hacked to work right-ish, but subclassing
> >> TestCase in this way, while convenient, isn't important enough to warrant
> >> this (IMHO).
> > 
> > Exactly right.
> 
> You're right. Next time I'm going to decide in favor of not backporting.
> I reverted both checkins, this and the urlparse one.
> 
> 
> For 2.5, this would be a good opportunity to add additional schemes
> to urlparse. Candidates?
> 
> Georg
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/janssen%40parc.com


From aahz at pythoncraft.com  Sun Jan 22 05:41:31 2006
From: aahz at pythoncraft.com (Aahz)
Date: Sat, 21 Jan 2006 20:41:31 -0800
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
Message-ID: <20060122044131.GA18077@panix.com>

On Sat, Jan 21, 2006, Guido van Rossum wrote:
>
> Why? If wikipedia can do without moderation (for most pages)  then why
> couldn't the Python docs?

If we're strictly talking about user comments, I won't disagree, but the
main docs do need to be "authoritative" IMO.

Aside to Georg: your messages are all getting /dev/null'd because you use
"spam" in your From: line.  I get too much spam to change that, and I'll
bet other people use that heuristic.  Please change your From: line for
this mailing list.
-- 
Aahz (aahz at pythoncraft.com)           <*>         http://www.pythoncraft.com/

"19. A language that doesn't affect the way you think about programming,
is not worth knowing."  --Alan Perlis

From g.brandl-nospam at gmx.net  Sun Jan 22 08:24:09 2006
From: g.brandl-nospam at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 08:24:09 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <ca471dc20601211744t3ef965ebtf377483968c57150@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dquiqj$jvh$1@sea.gmane.org>
	<ca471dc20601211744t3ef965ebtf377483968c57150@mail.gmail.com>
Message-ID: <dqvbup$9vi$1@sea.gmane.org>

Guido van Rossum wrote:
> On 1/21/06, Terry Reedy <tjreedy at udel.edu> wrote:
>> On this page, 8 of 30 entries have a 'new in' comment.  For anyone with no
>> interest in the past, these constitute noise.  I wonder if for 3.0, the
>> timer can be reset and the docs start clean again.  To keep them backwards
>> compatible, they would also have to be littered with 'changed in 3.0' and
>> 'deleted in 3.0' entries.  Better, I think, to refer people to the last 2.x
>> docs and a separate 2.x/3.0 changes doc.
> 
> (That has nothing to do with Fredrik's efforts of course -- he's just
> faithfully copying the annotations from the existing docs.)
> 
> If Python 3.0 is indeed the major incompatible release that we
> currently expect it will be, I agree that "new in 3.0" and "new in
> 2.x" annotations make little sense, since so much will be "new in 3.0"
> (or "deleted in 3.0"). It's probably better to have a separate
> document explaining the differences between 2.x and 3.0.
> 
> For the 2.x line, however (and again post-3.0) I think the "new in X" 
> annotations are important -- lots of people have to deal with
> different Python versions, and it's a lot easier to depend on "new in
> X" notations than to manually compare two or more versions of the
> docs. (Although you still need to do thorough testing -- like the rest
> of the docs, these annotations can't be 100% perfect.)
 >
> As far as noise goes, "new in X" is minor compared to all the stuff
> that's documented that the average user never needs... :-)

And, of course, the "new in 2.x" could be formatted less space-consuming,
perhaps to the right of the method name.

Georg


From ncoghlan at gmail.com  Sun Jan 22 08:32:40 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Jan 2006 17:32:40 +1000
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqvbup$9vi$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dquiqj$jvh$1@sea.gmane.org>	<ca471dc20601211744t3ef965ebtf377483968c57150@mail.gmail.com>
	<dqvbup$9vi$1@sea.gmane.org>
Message-ID: <43D33518.1090800@gmail.com>

Georg Brandl wrote:
> Guido van Rossum wrote:
>> As far as noise goes, "new in X" is minor compared to all the stuff
>> that's documented that the average user never needs... :-)
> 
> And, of course, the "new in 2.x" could be formatted less space-consuming,
> perhaps to the right of the method name.

That makes it worse by giving the information a prominence it doesn't really 
deserve. It also doesn't help when the change being noted was a signature or 
semantics change, rather than the addition of the function.

As with many other things, being able to clean out some of this history will 
be one of the benefits of Py3k.

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From g.brandl at gmx.net  Sun Jan 22 08:27:18 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 08:27:18 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <20060122044131.GA18077@panix.com>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<20060122044131.GA18077@panix.com>
Message-ID: <dqvc4m$9vi$2@sea.gmane.org>

Aahz wrote:
> On Sat, Jan 21, 2006, Guido van Rossum wrote:
>>
>> Why? If wikipedia can do without moderation (for most pages)  then why
>> couldn't the Python docs?
> 
> If we're strictly talking about user comments, I won't disagree, but the
> main docs do need to be "authoritative" IMO.
> 
> Aside to Georg: your messages are all getting /dev/null'd because you use
> "spam" in your From: line.  I get too much spam to change that, and I'll
> bet other people use that heuristic.  Please change your From: line for
> this mailing list.

I don't quite understand that. Why would a spam email have "spam" in the From?

For me, the "-nospam" suffix works relatively good to avoid spam, as most
harvesting programs will think this is a false address.

Of course, that may be more useful in Usenet, not in mailing lists.

Georg


From t-meyer at ihug.co.nz  Sun Jan 22 08:56:44 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Sun, 22 Jan 2006 20:56:44 +1300
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqvc4m$9vi$2@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<20060122044131.GA18077@panix.com> <dqvc4m$9vi$2@sea.gmane.org>
Message-ID: <17BD8931-BF5F-45B0-BD2C-6FE1FABA365B@ihug.co.nz>

> For me, the "-nospam" suffix works relatively good to avoid spam,  
> as most
> harvesting programs will think this is a false address.

http://spambayes.org works, too, without bothering others <0.5 wink>

=Tony.Meyer

From g.brandl at gmx.net  Sun Jan 22 10:23:24 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 10:23:24 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
Message-ID: <dqviud$n7b$1@sea.gmane.org>

Guido van Rossum wrote:
> On 1/21/06, Georg Brandl <g.brandl-nospam at gmx.net> wrote:
>> What Fredrik hacks together there (http://www.effbot.org/lib) is very
>> impressive. I especially like the "permalinks" in this style:
>>
>> http://effbot.org/lib/os.path.join
> 
> Which (despite having "perma" in its name) evaporates and leaves
> behind a link to os.path.html#join.

There may be other uses (e.g. marking a certain location in the docs with
a "permalink" marker so that one can point the user to /lib/marker.
Especially useful for the tutorial and recurring c.l.py questions ;)

>> What I would suggest (for any new doc system) is a "split" view: on the left,
>> the normal text, on the right, an area with only the headings, functions,
>> example and "see also" links (which preferably stays in sight). This way, you
>> always keep the outline in view.
> 
> Can you mock that up a bit? I'm somewhat confused about what you're
> requesting, and also worried that it would take up too much horizontal
> space. (Despite that monitors are wider than tall, horizontal real
> estate feels more scarce than vertical, because horizontal scrolling
> is such a pain.)

Something like

http://home.in.tum.de/~brandlg/zipfile.html

(It works this way (position: fixed) in most browsers, for IE one would have to
apply compatibility hacks.)

>> Of course, I wouldn't say no to a user-commenting system, but this would have to
>> be moderated.
> 
> Why? If wikipedia can do without moderation (for most pages)  then why
> couldn't the Python docs?

Well, why not... it's surely worth a try. Perhaps using a spam filter like most
modern weblogs would suffice.

>> What I'm also curious about regarding the current docs, why are optional
>> arguments in function declarations not written in Python style?
> 
> I'm assuming you're asking why we use
> 
>   foo(a[, b[, c]])
> 
> instead of
> 
>   foo(a, b=1, c=2)
> 
> ?

Yep.

> I can see several reasons; arguments with default values aren't
> necessarily usable keyword arguments (at least not if they
> function/method is implemented in C); the default value isn't always
> relevant (or is dynamic, or is a huge expression); and square brackets
> are the standard way in computer manuals to indicate optional parts of
> syntax.

Thanks.

Georg


From walter at livinglogic.de  Sun Jan 22 11:51:00 2006
From: walter at livinglogic.de (=?iso-8859-1?Q?Walter_D=F6rwald?=)
Date: Sun, 22 Jan 2006 11:51:00 +0100 (CET)
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqviud$n7b$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dqviud$n7b$1@sea.gmane.org>
Message-ID: <61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de>

Georg Brandl wrote:

>> [...]
>> Can you mock that up a bit? I'm somewhat confused about what you're requesting, and also worried that it would take up too
>> much horizontal space. (Despite that monitors are wider than tall, horizontal real estate feels more scarce than vertical,
>> because horizontal scrolling is such a pain.)
>
> Something like
>
> http://home.in.tum.de/~brandlg/zipfile.html
>
> (It works this way (position: fixed) in most browsers, for IE one would have to apply compatibility hacks.)

Vertical spacing is IMHO pretty bad on this page. Everything has the same distance from everything else. This makes
understanding the structure of the page pretty hard.
Bye,
   Walter D?rwald




From g.brandl at gmx.net  Sun Jan 22 12:07:16 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 12:07:16 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>
	<61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de>
Message-ID: <dqvp14$4nq$1@sea.gmane.org>

Walter D?rwald wrote:
> Georg Brandl wrote:
> 
>>> [...]
>>> Can you mock that up a bit? I'm somewhat confused about what you're requesting, and also worried that it would take up too
>>> much horizontal space. (Despite that monitors are wider than tall, horizontal real estate feels more scarce than vertical,
>>> because horizontal scrolling is such a pain.)
>>
>> Something like
>>
>> http://home.in.tum.de/~brandlg/zipfile.html
>>
>> (It works this way (position: fixed) in most browsers, for IE one would have to apply compatibility hacks.)
> 
> Vertical spacing is IMHO pretty bad on this page. Everything has the same distance from everything else. This makes
> understanding the structure of the page pretty hard.

The main content is the original zipfile.html from
http://www.effbot.org/lib/zipfile.html.

Only the sidebar is my addition.

Georg


From fredrik at pythonware.com  Sun Jan 22 12:58:14 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 22 Jan 2006 12:58:14 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
Message-ID: <dqvs0r$c27$1@sea.gmane.org>

Guido wrote:

> > What Fredrik hacks together there (http://www.effbot.org/lib) is very
> > impressive. I especially like the "permalinks" in this style:
> >
> > http://effbot.org/lib/os.path.join
>
> Which (despite having "perma" in its name) evaporates and leaves
> behind a link to os.path.html#join.

"permalink" was george's term; I'm just calling them "link URLs"

     http://online.effbot.org/2006_01_01_archive.htm#link-doc

to make them "real permalinks", we could:

- generate individual pages for all targets (on demand), with
  a "see this in context"
- return the same page and use some javascript trickery to
  emulate anchors

or use some other approach that I cannot think of right now

however, at this point, I'm more interested in getting the information
model to a point where this can be done.

> > Of course, I wouldn't say no to a user-commenting system, but this would have to
> > be moderated.
>
> Why? If wikipedia can do without moderation (for most pages)  then why
> couldn't the Python docs?

wikipedia has a large numbers of volunteers that constantly monitor all
updates to the site.  I'm not we have enough volunteers in python land,
and even if we have, I'm not sure site monitoring is the best way to use
the volunteer resources.

(this is the main motivator behind my documentation and site efforts; we

> > What I'm also curious about regarding the current docs, why are optional
> > arguments in function declarations not written in Python style?
>
> I'm assuming you're asking why we use
>
>   foo(a[, b[, c]])
>
> instead of
>
>   foo(a, b=1, c=2)
>
> ? I can see several reasons; arguments with default values aren't
> necessarily usable keyword arguments (at least not if they
> function/method is implemented in C); the default value isn't always
> relevant (or is dynamic, or is a huge expression); and square brackets
> are the standard way in computer manuals to indicate optional parts of
> syntax.

the [] syntax has its issues, including that it makes keyword arguments
look like positional arguments, and that descriptions that have lots of
optional arguments tend to look like either

    foo([a[,b[,c[,d[,e[,f]]]]]])

    foo([a][,b][,c][,d][,e][,f])

where the first one is ugly and hard to read, and the latter is, strictly
speaking, not correct.

there are ways to address all these problems with a more pythonic syntax,
but I'll have to get back to that later.

</F>




From fredrik at pythonware.com  Sun Jan 22 13:14:14 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 22 Jan 2006 13:14:14 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org><61073.85.74.64.105.1137927060.squirrel@isar.livinglogic.de>
	<dqvp14$4nq$1@sea.gmane.org>
Message-ID: <dqvsuo$eit$1@sea.gmane.org>

Georg Brandl wrote:

> > Vertical spacing is IMHO pretty bad on this page. Everything
> > has the same distance from everything else. This makes under-
> > standing the structure of the page pretty hard.
>
> The main content is the original zipfile.html from
> http://www.effbot.org/lib/zipfile.html.
>
> Only the sidebar is my addition.

the effbot.org/lib material is raw HTML; at this stage, I'm working on
getting the translation and the information model right, not polishing
the formatting.

formatting ideas are of course welcome (especially in CSS form), but
comments on the information structure are more useful at this time.

</F>




From fredrik at pythonware.com  Sun Jan 22 13:24:59 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 22 Jan 2006 13:24:59 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org><ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dqvs0r$c27$1@sea.gmane.org>
Message-ID: <dqvtis$g1r$1@sea.gmane.org>

> (this is the main motivator behind my documentation and site efforts; we

should learn not to press control-enter when we mean enter.

anyway, what I intended to say was that we should work harder on lowering the
threshold for drive-by contributions; it should be possible for anyone to notice a
problem, add a comment or correction, preview the correction properly rendered,
and submit it to a moderation queue, in less than a minute.

(I still think moderation is needed before corrections are made visible, but that
should be no harder, nor require any more tools, than making the correction in
the first place.  I'm sure we could get some turbogears/django hackers to build
us a nice moderation queue webapp in no time at all...)

</F>




From fredrik at pythonware.com  Sun Jan 22 13:30:41 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 22 Jan 2006 13:30:41 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org><ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com><dquiqj$jvh$1@sea.gmane.org>
	<ca471dc20601211744t3ef965ebtf377483968c57150@mail.gmail.com>
Message-ID: <dqvttj$gsu$1@sea.gmane.org>

Guido van Rossum wrote:

> > On this page, 8 of 30 entries have a 'new in' comment.  For anyone with no
> > interest in the past, these constitute noise.  I wonder if for 3.0, the
> > timer can be reset and the docs start clean again.  To keep them backwards
> > compatible, they would also have to be littered with 'changed in 3.0' and
> > 'deleted in 3.0' entries.  Better, I think, to refer people to the last 2.x
> > docs and a separate 2.x/3.0 changes doc.
>
> (That has nothing to do with Fredrik's efforts of course -- he's just
> faithfully copying the annotations from the existing docs.)
>
> If Python 3.0 is indeed the major incompatible release that we
> currently expect it will be, I agree that "new in 3.0" and "new in
> 2.x" annotations make little sense, since so much will be "new in 3.0"
> (or "deleted in 3.0"). It's probably better to have a separate
> document explaining the differences between 2.x and 3.0.

absolutely.

> For the 2.x line, however (and again post-3.0) I think the "new in X"
> annotations are important -- lots of people have to deal with
> different Python versions, and it's a lot easier to depend on "new in
> X" notations than to manually compare two or more versions of the
> docs. (Although you still need to do thorough testing -- like the rest
> of the docs, these annotations can't be 100% perfect.)

if we move over to a web-oriented (wiki-ish) maintenance model, we
probably have to accept that "X" will, sometimes, be a future release.

</F>




From ncoghlan at gmail.com  Sun Jan 22 14:24:42 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sun, 22 Jan 2006 23:24:42 +1000
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqvtis$g1r$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org><ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqvs0r$c27$1@sea.gmane.org>
	<dqvtis$g1r$1@sea.gmane.org>
Message-ID: <43D3879A.8090101@gmail.com>

Fredrik Lundh wrote:
>> (this is the main motivator behind my documentation and site efforts; we
> 
> should learn not to press control-enter when we mean enter.
> 
> anyway, what I intended to say was that we should work harder on lowering the
> threshold for drive-by contributions; it should be possible for anyone to notice a
> problem, add a comment or correction, preview the correction properly rendered,
> and submit it to a moderation queue, in less than a minute.
> 
> (I still think moderation is needed before corrections are made visible, but that
> should be no harder, nor require any more tools, than making the correction in
> the first place.  I'm sure we could get some turbogears/django hackers to build
> us a nice moderation queue webapp in no time at all...)

Or go low-tech and have a 'mailto' link that goes to a documentation 
discussion list ;)

Cheers,
Nick.

P.S. I do like the way this is coming together - particularly if we can find 
some cross-platform CSS like Georg's to keep the handy list of links on the right.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From guido at python.org  Sun Jan 22 16:19:13 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Jan 2006 07:19:13 -0800
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqvbup$9vi$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dquiqj$jvh$1@sea.gmane.org>
	<ca471dc20601211744t3ef965ebtf377483968c57150@mail.gmail.com>
	<dqvbup$9vi$1@sea.gmane.org>
Message-ID: <ca471dc20601220719k3c8c70d1ye4ca6d434c2681cb@mail.gmail.com>

On 1/21/06, Georg Brandl <g.brandl-nospam at gmx.net> wrote:
> And, of course, the "new in 2.x" could be formatted less space-consuming,
> perhaps to the right of the method name.

No, that would give it too much attention.

It's getting way too outstanding in effbot's sample, probably because
he didn't implement the formatter for @since yet. It should simply be
an addition to the text paragraph that precedes it, like in the
regular docs.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Jan 22 17:16:30 2006
From: guido at python.org (Guido van Rossum)
Date: Sun, 22 Jan 2006 08:16:30 -0800
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqviud$n7b$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dqviud$n7b$1@sea.gmane.org>
Message-ID: <ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>

(This is a bulk reply to several messages in this thread from Georg
and Fredrik.)

[Georg]
> >> http://effbot.org/lib/os.path.join

[Guido]
> > Which (despite having "perma" in its name) evaporates and leaves
> > behind a link to os.path.html#join.

[Georg]
> There may be other uses (e.g. marking a certain location in the docs with
> a "permalink" marker so that one can point the user to /lib/marker.
> Especially useful for the tutorial and recurring c.l.py questions ;)

Rather than allowing users to add new markers I'd like the document to
be sprinked generously with pregenerated markers to which one can
easily linked. I've seen sample docs (not for Python) with such a
feature.

[Fredrik, in a different message]
> "permalink" was george's term; I'm just calling them "link URLs"
>
>      http://online.effbot.org/2006_01_01_archive.htm#link-doc
>
> to make them "real permalinks", we could:
>
> - generate individual pages for all targets (on demand), with
>   a "see this in context"
> - return the same page and use some javascript trickery to
>   emulate anchors
>
> or use some other approach that I cannot think of right now

That sounds overkill. But if you could drop the redundant ".html", the
links in the source could look like like

  .../os.path#join

which is just fine with me (and solves the problem of what part is the
module, on which you spend altogether much time explaining how to
compute in the current version :-).

> however, at this point, I'm more interested in getting the information
> model to a point where this can be done.

Understood. What's still missing from the info model? It looks pretty
complete to me from a quick scan.

[Georg again]
> >> What I would suggest (for any new doc system) is a "split" view: on the left,
> >> the normal text, on the right, an area with only the headings, functions,
> >> example and "see also" links (which preferably stays in sight). This way, you
> >> always keep the outline in view.

[Guido]
> > Can you mock that up a bit? I'm somewhat confused about what you're
> > requesting, and also worried that it would take up too much horizontal
> > space. (Despite that monitors are wider than tall, horizontal real
> > estate feels more scarce than vertical, because horizontal scrolling
> > is such a pain.)

[Georg]
> Something like
>
> http://home.in.tum.de/~brandlg/zipfile.html
>
> (It works this way (position: fixed) in most browsers, for IE one would have to
> apply compatibility hacks.)

I believe there's a CSS trick (most often used for images) that can
makes the summary window "float" to the right so that below it the
main text resumes the full breadth of the window. If you can pull that
off I think this is a good addition!

> >> Of course, I wouldn't say no to a user-commenting system, but this would have to
> >> be moderated.
> >
> > Why? If wikipedia can do without moderation (for most pages)  then why
> > couldn't the Python docs?
>
> Well, why not... it's surely worth a try. Perhaps using a spam filter like most
> modern weblogs would suffice.

Have you studied wikipedia's approach? It's multi-layered and worth
learning from (start with their FAQ on editing).

(And by the way, I am *not* advocating writing the docs as one big
wikipedia -- only the user commentary.)

[Fredrik again]
> the [] syntax has its issues, including that it makes keyword arguments
> look like positional arguments,

(I think that is actually Python's fault for not distinguishing them
in the function definition. :-)

> and that descriptions that have lots of
> optional arguments tend to look like either
>
>     foo([a[,b[,c[,d[,e[,f]]]]]])
>
>     foo([a][,b][,c][,d][,e][,f])
>
> where the first one is ugly and hard to read, and the latter is, strictly
> speaking, not correct.

Or (my favorite, but still wrong)

  foo([a, b, c, d, e, f])

> there are ways to address all these problems with a more pythonic syntax,
> but I'll have to get back to that later.

Please do.

[Fredrik, in yet another message]
> anyway, what I intended to say was that we should work harder on lowering the
> threshold for drive-by contributions; it should be possible for anyone to notice a
> problem, add a comment or correction, preview the correction properly rendered,
> and submit it to a moderation queue, in less than a minute.

Given that it will take significantly longer to *process* the queue I
don't necessarily want to make the contribution process *too* easy --
we need to balance the quality of the contributions with the ease to
make them. For many simple corrections (spelling, grammar, inversions,
some factual mistakes)  the need to preview the correction properly
rendered is quite unnecessary; the important part is being able to
*point* at any word or words in the text and attach a comment.

> (I still think moderation is needed before corrections are made visible, but that
> should be no harder, nor require any more tools, than making the correction in
> the first place.  I'm sure we could get some turbogears/django hackers to build
> us a nice moderation queue webapp in no time at all...)

Agreed. But as I said above, I think there should also be a way to
simply add comments, without moderation (but with wikipedia-style
editing / spam-removal / moderation after-the-fact). And, probably, if
we don't get to the moderation queue etc., the comments wiki could
serve as a moderation queue as well. Being public, it would also
reduce the submission of duplicate corrections (especially when the
moderator is out of town for a while).

[Fredrik, later]
> if we move over to a web-oriented (wiki-ish) maintenance model, we
> probably have to accept that "X" will, sometimes, be a future release.

This makes me worry that you're thinking of abandoning having separate
docs per version. That would be a shame; I think the docs ought to be
maintained well enough to be authorative for a particular version
(modulo bugs in the docs). Perhaps we could add links that let you
navigate between versions quickly, and hopefully we can somehow
(optionally?) merge the wiki-style user comments across versions.
(Although I can also see the point of starting afresh for each release
-- at some point the sheer amount of (possibly outdated) user comments
reduces their usefulness.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From facundobatista at gmail.com  Sun Jan 22 17:32:39 2006
From: facundobatista at gmail.com (Facundo Batista)
Date: Sun, 22 Jan 2006 13:32:39 -0300
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <e04bdf310601220831x553cecccv@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dqviud$n7b$1@sea.gmane.org>
	<e04bdf310601220831x553cecccv@mail.gmail.com>
Message-ID: <e04bdf310601220832h44901c82r@mail.gmail.com>

2006/1/22, Georg Brandl <g.brandl at gmx.net>:

> Guido van Rossum wrote:
> > ...
> > Why? If wikipedia can do without moderation (for most pages)  then why
> > couldn't the Python docs?
>
> Well, why not... it's surely worth a try. Perhaps using a spam filter like most
> modern weblogs would suffice.

I can see three levels of permissiveness. Everybody (with or without
registration?) will be able to post whatever, but the annotation...

- will be posted always
- will go through an antispam
- will go through a posting-revision-comitee (that is, a human antispam, ;)).

I don't like the first one, I'm +1 with the second and +0 with the third.

In the last two, specially in the third one, you'll need human
workforce. I'm stepping forward for that, if you need it.

.    Facundo

Blog: http://www.taniquetil.com.ar/plog/
PyAr: http://www.python.org/ar/

From g.brandl at gmx.net  Sun Jan 22 17:34:49 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 17:34:49 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>
	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
Message-ID: <dr0c79$pco$1@sea.gmane.org>

Guido van Rossum wrote:

>> > Which (despite having "perma" in its name) evaporates and leaves
>> > behind a link to os.path.html#join.
> 
>> There may be other uses (e.g. marking a certain location in the docs with
>> a "permalink" marker so that one can point the user to /lib/marker.
>> Especially useful for the tutorial and recurring c.l.py questions ;)
> 
> Rather than allowing users to add new markers I'd like the document to
> be sprinked generously with pregenerated markers to which one can
> easily linked. I've seen sample docs (not for Python) with such a
> feature.

Exactly what I had in mind. And we can add markers by request of the
c.l.py "support" regulars.

>> Something like
>>
>> http://home.in.tum.de/~brandlg/zipfile.html
>>
>> (It works this way (position: fixed) in most browsers, for IE one would have to
>> apply compatibility hacks.)
> 
> I believe there's a CSS trick (most often used for images) that can
> makes the summary window "float" to the right so that below it the
> main text resumes the full breadth of the window. If you can pull that
> off I think this is a good addition!

Well, floating the summary is no problem, but it wouldn't stay at the side
if you scroll down then.

I updated the mockup a bit:

http://home.in.tum.de/~brandlg/zipfile.html

An alternative with floating summary:

http://home.in.tum.de/~brandlg/zipfile-float.html

>> >> Of course, I wouldn't say no to a user-commenting system, but this would have to
>> >> be moderated.
>> >
>> > Why? If wikipedia can do without moderation (for most pages)  then why
>> > couldn't the Python docs?
>>
>> Well, why not... it's surely worth a try. Perhaps using a spam filter like most
>> modern weblogs would suffice.
> 
> Have you studied wikipedia's approach? It's multi-layered and worth
> learning from (start with their FAQ on editing).
>
> (And by the way, I am *not* advocating writing the docs as one big
> wikipedia -- only the user commentary.)

Agreed. Something like a "discussion page" could be fruitful.

>> (I still think moderation is needed before corrections are made visible, but that
>> should be no harder, nor require any more tools, than making the correction in
>> the first place.  I'm sure we could get some turbogears/django hackers to build
>> us a nice moderation queue webapp in no time at all...)
> 
> Agreed. But as I said above, I think there should also be a way to
> simply add comments, without moderation (but with wikipedia-style
> editing / spam-removal / moderation after-the-fact). And, probably, if
> we don't get to the moderation queue etc., the comments wiki could
> serve as a moderation queue as well. Being public, it would also
> reduce the submission of duplicate corrections (especially when the
> moderator is out of town for a while).

+1.

> [Fredrik, later]
>> if we move over to a web-oriented (wiki-ish) maintenance model, we
>> probably have to accept that "X" will, sometimes, be a future release.
> 
> This makes me worry that you're thinking of abandoning having separate
> docs per version. That would be a shame; I think the docs ought to be
> maintained well enough to be authorative for a particular version
> (modulo bugs in the docs). Perhaps we could add links that let you
> navigate between versions quickly, and hopefully we can somehow
> (optionally?) merge the wiki-style user comments across versions.

+1.

> (Although I can also see the point of starting afresh for each release
> -- at some point the sheer amount of (possibly outdated) user comments
> reduces their usefulness.

That would have to be decided in situ.

regards,
Georg


From tim.peters at gmail.com  Sun Jan 22 19:16:56 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 22 Jan 2006 13:16:56 -0500
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dqvc4m$9vi$2@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<20060122044131.GA18077@panix.com> <dqvc4m$9vi$2@sea.gmane.org>
Message-ID: <1f7befae0601221016t24d1bb5dt1cde577cd2b75a82@mail.gmail.com>

[Aahz]
>> Aside to Georg: your messages are all getting /dev/null'd because you use
>> "spam" in your From: line.  I get too much spam to change that, and I'll
>> bet other people use that heuristic.  Please change your From: line for
>> this mailing list.

[Georg Brandl]
> I don't quite understand that. Why would a spam email have "spam" in
> the From?

In general it wouldn't.  Aahz didn't tell us everything, or he's
simply not making sense.

Because I still like ;-) the SpamBayes project, I don't do anything to
try to stop receiving spam, and have several email addresses that have
been visible for years.  Of the 4456 spam I received via Outlook in
the last 30 days, onlly 3 had "spam" in the From header:

From: alison at charlotte.dontspam.wsrcc.com
From: ".?.??? .?????" <moskowspam at BonBon.net>
From: spam <hasting at krause-taylor.com>

It was much more common among ham, including email from you, from the
spambayes mailing lists, from SORBS, ....  If:

    The `From:` header contains "spam" as a substring, case-insensitively.

were a feature SpamBayes scored, it would be a strong ham clue in my
training database.

> For me, the "-nospam" suffix works relatively good to avoid spam, as most
> harvesting programs will think this is a false address.
>
> Of course, that may be more useful in Usenet, not in mailing lists.

It's OK by me if you keep using it.

From tim at pollenation.net  Sun Jan 22 20:10:04 2006
From: tim at pollenation.net (Tim Parkin)
Date: Sun, 22 Jan 2006 19:10:04 +0000
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>
	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
Message-ID: <43D3D88C.6070001@pollenation.net>

Guido van Rossum wrote:
> I believe there's a CSS trick (most often used for images) that can
> makes the summary window "float" to the right so that below it the
> main text resumes the full breadth of the window. If you can pull that
> off I think this is a good addition!
> 
> 

Something like this...

http://beta.python.org/download/releases/2.4.1/

Tim

From tim at pollenation.net  Sun Jan 22 20:18:41 2006
From: tim at pollenation.net (Tim Parkin)
Date: Sun, 22 Jan 2006 19:18:41 +0000
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <43D3D88C.6070001@pollenation.net>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
	<43D3D88C.6070001@pollenation.net>
Message-ID: <43D3DA91.9070708@pollenation.net>

Tim Parkin wrote:
> Guido van Rossum wrote:
> 
>>I believe there's a CSS trick (most often used for images) that can
>>makes the summary window "float" to the right so that below it the
>>main text resumes the full breadth of the window. If you can pull that
>>off I think this is a good addition!
>>
>>
> 
> 
> Something like this...
> 
> http://beta.python.org/download/releases/2.4.1/
> 

It would also be simple to add an 'alternate stylesheet' that can hide
the left hand navigation. A small link could be added to switch styles
(which would include javascript) but all good browsers allow you to
choose which stylesheet you wish to use without the aid of javascript
(which I beleive should only be used for 'optional enhancements' to a
website).

The layout also positions the main navigation at the bottom of the html
source so the first items you see using a 'text browser' are the
breadcrumb and document level navigation.

Tim Parkin

From g.brandl at gmx.net  Sun Jan 22 20:40:58 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 20:40:58 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <e04bdf310601220832h44901c82r@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>	<e04bdf310601220831x553cecccv@mail.gmail.com>
	<e04bdf310601220832h44901c82r@mail.gmail.com>
Message-ID: <dr0n4a$p80$1@sea.gmane.org>

Facundo Batista wrote:
> 2006/1/22, Georg Brandl <g.brandl at gmx.net>:
> 
>> Guido van Rossum wrote:
>> > ...
>> > Why? If wikipedia can do without moderation (for most pages)  then why
>> > couldn't the Python docs?
>>
>> Well, why not... it's surely worth a try. Perhaps using a spam filter like most
>> modern weblogs would suffice.
> 
> I can see three levels of permissiveness. Everybody (with or without
> registration?) will be able to post whatever, but the annotation...
> 
> - will be posted always
> - will go through an antispam
> - will go through a posting-revision-comitee (that is, a human antispam, ;)).
> 
> I don't like the first one, I'm +1 with the second and +0 with the third.
> 
> In the last two, specially in the third one, you'll need human
> workforce. I'm stepping forward for that, if you need it.

Metoo, of course.

Georg


From fredrik at pythonware.com  Sun Jan 22 20:50:31 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun, 22 Jan 2006 20:50:31 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org><ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com><dqviud$n7b$1@sea.gmane.org>
	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
Message-ID: <dr0nm9$sc1$1@sea.gmane.org>

Guido wrote:

> Have you studied wikipedia's approach? It's multi-layered and worth
> learning from (start with their FAQ on editing).
>
> (And by the way, I am *not* advocating writing the docs as one big
> wikipedia -- only the user commentary.)

to clarify, I'm advocating maintaining the docs via a restricted wiki-like
system, and using a blog-style discussion board or a wiki to collect non-
specific user commentary.

but it's the combination that really interests me: the ability for users to
propose specific corrections (as patches, at least under the cover), and
for volunteers to apply such patches with a minimum of effort.

more later.

</F>




From g.brandl at gmx.net  Sun Jan 22 20:50:41 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Sun, 22 Jan 2006 20:50:41 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <43D3DA91.9070708@pollenation.net>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>	<43D3D88C.6070001@pollenation.net>
	<43D3DA91.9070708@pollenation.net>
Message-ID: <dr0nmh$rli$1@sea.gmane.org>

Tim Parkin wrote:
> Tim Parkin wrote:
>> Guido van Rossum wrote:
>> 
>>>I believe there's a CSS trick (most often used for images) that can
>>>makes the summary window "float" to the right so that below it the
>>>main text resumes the full breadth of the window. If you can pull that
>>>off I think this is a good addition!
>>>
>> Something like this...
>> 
>> http://beta.python.org/download/releases/2.4.1/

That's an ordinary float, I presume?

> It would also be simple to add an 'alternate stylesheet' that can hide
> the left hand navigation. A small link could be added to switch styles
> (which would include javascript) but all good browsers allow you to
> choose which stylesheet you wish to use without the aid of javascript
> (which I beleive should only be used for 'optional enhancements' to a
> website).

Good idea! Dito for options like "enlarge text" or such.

Georg


From tim at pollenation.net  Sun Jan 22 23:05:17 2006
From: tim at pollenation.net (Tim Parkin)
Date: Sun, 22 Jan 2006 22:05:17 +0000
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dr0nmh$rli$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>	<dqviud$n7b$1@sea.gmane.org>	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>	<43D3D88C.6070001@pollenation.net>	<43D3DA91.9070708@pollenation.net>
	<dr0nmh$rli$1@sea.gmane.org>
Message-ID: <43D4019D.8020707@pollenation.net>

Georg Brandl wrote:

>Tim Parkin wrote:
>
>>>Something like this...
>>>
>>>http://beta.python.org/download/releases/2.4.1/
>>>      
>>>
>
>That's an ordinary float, I presume?
>  
>
indeed it is.

>  
>
>>It would also be simple to add an 'alternate stylesheet' that can hide
>>the left hand navigation. A small link could be added to switch styles
>>(which would include javascript) but all good browsers allow you to
>>choose which stylesheet you wish to use without the aid of javascript
>>(which I beleive should only be used for 'optional enhancements' to a
>>website).
>>    
>>
>
>Good idea! Dito for options like "enlarge text" or such.
>  
>
It already has a 'large text' alternate style sheet, however it needs a
couple of tweaks since I've made some big style sheet changes. The style
sheet switching is well documented here with example code etc...

http://www.brothercake.com/site/resources/scripts/iotbs/

Tim


From brett at python.org  Mon Jan 23 01:48:42 2006
From: brett at python.org (Brett Cannon)
Date: Sun, 22 Jan 2006 16:48:42 -0800
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dqviud$n7b$1@sea.gmane.org>
	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
Message-ID: <bbaeab100601221648h23efc94eu19f7cb706bd5dde3@mail.gmail.com>

> [Fredrik, later]
> > if we move over to a web-oriented (wiki-ish) maintenance model, we
> > probably have to accept that "X" will, sometimes, be a future release.
>
> This makes me worry that you're thinking of abandoning having separate
> docs per version. That would be a shame; I think the docs ought to be
> maintained well enough to be authorative for a particular version
> (modulo bugs in the docs). Perhaps we could add links that let you
> navigate between versions quickly, and hopefully we can somehow
> (optionally?) merge the wiki-style user comments across versions.
> (Although I can also see the point of starting afresh for each release
> -- at some point the sheer amount of (possibly outdated) user comments
> reduces their usefulness.

It would probably be best to clear out the comments for each minor
release.  Otherwise they will just organically grow.

Problem is that useful examples might disappear.  This might suggest
that having possible example usage tagged on to a method or such could
be saved and, by default, made folded (ala folding in Ian Bicking's
Python doc comment system:
http://blog.ianbicking.org/python-doc-comments.html).  These examples
could also be verified using something like doctest so that
invalidated examples do not persist or submitted.

And to /F, kudos from me.  I have been randomly thinking about it and
I understand your desire for semantic markup now.  Hopefully something
can get hammered out so that at least the Python 3 docs can premiere
having been developed on by the whole community.

-Brett

From bjourne at gmail.com  Mon Jan 23 10:32:43 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Mon, 23 Jan 2006 10:32:43 +0100
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dr0nm9$sc1$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dqviud$n7b$1@sea.gmane.org>
	<ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
	<dr0nm9$sc1$1@sea.gmane.org>
Message-ID: <740c3aec0601230132n2ff31a22n1cd49f9517330cb1@mail.gmail.com>

> > Have you studied wikipedia's approach? It's multi-layered and worth
> > learning from (start with their FAQ on editing).
> >
> > (And by the way, I am *not* advocating writing the docs as one big
> > wikipedia -- only the user commentary.)
>
> to clarify, I'm advocating maintaining the docs via a restricted wiki-like
> system, and using a blog-style discussion board or a wiki to collect non-
> specific user commentary.

Why does it have to be "wiki-like"? Why can't it be a wiki? MediaWiki
seem to work pretty well for a lot of software projects that have put
their documentation in a wiki. Talk pages for commentary and primary
pages for reviewed content.

--
mvh Bj?rn

From fredrik at pythonware.com  Mon Jan 23 16:42:24 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 23 Jan 2006 16:42:24 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org><ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com><dqviud$n7b$1@sea.gmane.org><ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com><dr0nm9$sc1$1@sea.gmane.org>
	<740c3aec0601230132n2ff31a22n1cd49f9517330cb1@mail.gmail.com>
Message-ID: <dr2th0$6qb$1@sea.gmane.org>

BJörn Lindqvist wrote:

> > > Have you studied wikipedia's approach? It's multi-layered and worth
> > > learning from (start with their FAQ on editing).
> > >
> > > (And by the way, I am *not* advocating writing the docs as one big
> > > wikipedia -- only the user commentary.)
> >
> > to clarify, I'm advocating maintaining the docs via a restricted wiki-like
> > system, and using a blog-style discussion board or a wiki to collect non-
> > specific user commentary.
>
> Why does it have to be "wiki-like"? Why can't it be a wiki? MediaWiki
> seem to work pretty well for a lot of software projects that have put
> their documentation in a wiki. Talk pages for commentary and primary
> pages for reviewed content.

semantics, semantics, semantics (see earlier posts on this topic).

but sure, a wiki engine with support for custom source syntax (to support
python-oriented semantic markup) and section editing could work.

(moinmoin has the former, but not the latter, afaik).

</F> 




From nnorwitz at gmail.com  Mon Jan 23 08:01:36 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Sun, 22 Jan 2006 23:01:36 -0800
Subject: [Python-Dev] stabilizing builds
Message-ID: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>

I would really like to stabilize a bunch of the current rough edges in
2.5.  We are likely to have enough instability with the PEP 353
changes, so it would be better to fix some of these issues sooner
rather than later.  There are a bunch of tests that are not stable. 
It would really help to get people knowledgeable about a particular
subdomain to provide input into bugs/patches and produce patches too!

The areas that are known to have problems are listed here:
    http://wiki.python.org/moin/BuildBot

It would be nice to clean those issues up.  Does anyone have some time
to spend to resolve the issues with Berkeley DB?  That's the one that
annoys me most right now.  I have a patch that fixes at least one of
the problems, but no idea if it's the correct solution or not. 
http://python.org/sf/1407992

n
--
PS. In case you are interested in all the problems:

* BSD DB 4.1 and 3.2 fail
* test_pty is brittle on solaris 10, sometimes it works, sometimes not
* getaddrinfo() configure check sometimes fails on OS X 10.3

From nnorwitz at gmail.com  Mon Jan 23 09:36:36 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Mon, 23 Jan 2006 00:36:36 -0800
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <20060111223039.GC6642@activestate.com>
References: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>
	<dq3u93$s4g$1@sea.gmane.org> <20060111223039.GC6642@activestate.com>
Message-ID: <ee2a432c0601230036l61c79c4dp617d1521f514afd6@mail.gmail.com>

On 1/11/06, Trent Mick <trentm at activestate.com> wrote:
> > > Does that make sense?  We would just need /f's script in SVN.
> >
> > in python/Tools/something or sandbox/something ?
>
> python/Doc/tools/something?

Fredrik were you still working on that?  I can make the changes to the
bb master.  I thought Trent's suggested placement was good.

n

From thomas at xs4all.net  Mon Jan 23 22:22:10 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 23 Jan 2006 22:22:10 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
Message-ID: <20060123212210.GA18916@xs4all.nl>

On Sun, Jan 22, 2006 at 11:01:36PM -0800, Neal Norwitz wrote:

> * test_pty is brittle on solaris 10, sometimes it works, sometimes not

Do we have a Solaris 10 box to test on? I think I wrote most of test_pty,
and I can see if I can pin down the problem, but I don't have a Solaris 10
box myself.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From kbk at shore.net  Tue Jan 24 03:01:08 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon, 23 Jan 2006 21:01:08 -0500 (EST)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary
Message-ID: <200601240201.k0O218GE007855@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  391 open ( +7) /  3028 closed (+12) /  3419 total (+19)
Bugs    :  905 open ( -4) /  5519 closed (+19) /  6424 total (+15)
RFE     :  207 open ( -1) /   197 closed ( +1) /   404 total ( +0)

New / Reopened Patches
______________________

Patch for #1371247  (2006-01-14)
CLOSED http://python.org/sf/1406159  opened by  Peter van Kampen

Doc patch for bug #1396471  (2006-01-16)
CLOSED http://python.org/sf/1407021  opened by  Peter van Kampen

anonymous mmap  (2006-01-16)
       http://python.org/sf/1407135  opened by  Ralf Schmitt

ParenMatch: workaround for misinterpreting of closing parens  (2006-01-16)
       http://python.org/sf/1407280  opened by  Tal Einat

Patches Closed
______________

AF_NETLINK sockets basic support  (2005-01-15)
       http://python.org/sf/1103116  closed by  loewis

Further .vcproj cleanups  (2006-01-03)
       http://python.org/sf/1396093  closed by  loewis

bind() for netlink sockets  (2005-12-15)
       http://python.org/sf/1381398  closed by  loewis

Patch for #1371247  (2006-01-14)
       http://python.org/sf/1406159  closed by  birkenfeld

Doc patch for bug #1396471  (2006-01-16)
       http://python.org/sf/1407021  closed by  birkenfeld

New / Reopened Bugs
___________________

bz2module with no long long type support  (2006-01-16)
CLOSED http://python.org/sf/1407069  opened by  kbob_ru

urlparse does not know about sftp: urls  (2006-01-17)
CLOSED http://python.org/sf/1407902  opened by  Stuart Bishop

Bugs Closed
___________

segfault when using mmap(-1,...) [+PATCH]  (2006-01-10)
       http://python.org/sf/1402308  closed by  nnorwitz

2.3.5 RPM provides incompatibility  (2006-01-11)
       http://python.org/sf/1403225  closed by  jafo

locale.windows_locale   (2005-12-01)
       http://python.org/sf/1371247  closed by  birkenfeld

file.tell() returns illegal value on Windows  (2006-01-04)
       http://python.org/sf/1396471  closed by  birkenfeld

bz2module with no long long type support  (2006-01-15)
       http://python.org/sf/1407069  closed by  nnorwitz

TimedRotatingFileHandler at midnight rolls over at 01:00  (2006-01-03)
       http://python.org/sf/1396030  closed by  vsajip

TimedRotatingFileHandler does not recover from open error  (2006-01-03)
       http://python.org/sf/1396040  closed by  vsajip

urllib2 should be more robust for sloppy proxy URLs  (2004-02-22)
       http://python.org/sf/902075  closed by  birkenfeld

urlparse does not know about sftp: urls  (2006-01-17)
       http://python.org/sf/1407902  closed by  birkenfeld

in email can't get attachments' filenames using get_filename  (2006-01-11)
       http://python.org/sf/1403349  closed by  bwarsaw

RFE Closed
__________

python.org - add RFE tracker  (2006-01-05)
       http://python.org/sf/1397806  closed by  kbk


From kbk at shore.net  Tue Jan 24 03:30:08 2006
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon, 23 Jan 2006 21:30:08 -0500 (EST)
Subject: [Python-Dev] Weekly Python Patch/Bug Summary (Revised)
Message-ID: <200601240230.k0O2U8GE022475@bayview.thirdcreek.com>

Patch / Bug Summary
___________________

Patches :  391 open ( +7) /  3028 closed (+12) /  3419 total (+19)
Bugs    :  906 open ( -3) /  5519 closed (+19) /  6425 total (+16)
RFE     :  207 open ( -1) /   197 closed ( +1) /   404 total ( +0)

New / Reopened Patches
______________________

Patch for #1371247  (2006-01-14)
CLOSED http://python.org/sf/1406159  opened by  Peter van Kampen

Doc patch for bug #1396471  (2006-01-16)
CLOSED http://python.org/sf/1407021  opened by  Peter van Kampen

anonymous mmap  (2006-01-16)
       http://python.org/sf/1407135  opened by  Ralf Schmitt

ParenMatch: workaround for misinterpreting of closing parens  (2006-01-16)
       http://python.org/sf/1407280  opened by  Tal Einat

fix bsddb test associate problems w/bsddb 4.1  (2006-01-16)
       http://python.org/sf/1407992  opened by  Neal Norwitz

Corrupt Berkeley DB using Modify bsddb.dbtables  (2006-01-17)
       http://python.org/sf/1408584  opened by  jross

use computed goto's in ceval loop  (2006-01-18)
       http://python.org/sf/1408710  opened by  Simon David Burton

Add foldr and foldl to functional module  (2006-01-19)
CLOSED http://python.org/sf/1410119  opened by  Collin Winter

Add 'surgical editing' to ConfigParser  (2006-01-21)
       http://python.org/sf/1410680  opened by  Tony Meyer

Add notes to the manual about `is` and methods  (2006-01-20)
       http://python.org/sf/1410739  opened by  Collin Winter

Doc patch for classmethod and staticmethod  (2006-01-20)
CLOSED http://python.org/sf/1410783  opened by  Collin Winter

Remove mention of DOS from os.py's docstring  (2006-01-20)
CLOSED http://python.org/sf/1410998  opened by  Hamish Lawson

Add DarwinPorts directories to setup.py  (2006-01-21)
CLOSED http://python.org/sf/1411588  opened by  Barry A. Warsaw

More easily extensible logging module  (2006-01-22)
       http://python.org/sf/1412054  opened by  Daniele Varrazzo

Compile under mingw properly  (2006-01-23)
       http://python.org/sf/1412448  opened by  Alexandre Gir?o Bezerra de Mene

Fill out the functional module  (2006-01-22)
       http://python.org/sf/1412451  opened by  Collin Winter

Proper locking with asynchronous callbacks.  (2006-01-23)
       http://python.org/sf/1412632  opened by  uiltje

zipfile: use correct system type on unixy systems  (2006-01-23)
       http://python.org/sf/1412872  opened by  Ronald Oussoren

PyThreadState_Delete doesnt' clean autoTLSkey  (2006-01-23)
       http://python.org/sf/1413181  opened by  Gabriel Becedillas

Patches Closed
______________

AF_NETLINK sockets basic support  (2005-01-15)
       http://python.org/sf/1103116  closed by  loewis

Further .vcproj cleanups  (2006-01-03)
       http://python.org/sf/1396093  closed by  loewis

bind() for netlink sockets  (2005-12-15)
       http://python.org/sf/1381398  closed by  loewis

Patch for #1371247  (2006-01-14)
       http://python.org/sf/1406159  closed by  birkenfeld

Doc patch for bug #1396471  (2006-01-16)
       http://python.org/sf/1407021  closed by  birkenfeld

Add foldr and foldl to functional module  (2006-01-19)
       http://python.org/sf/1410119  closed by  collinwinter

Need warning that `dl` module can cause segfaults/crashes  (2006-01-11)
       http://python.org/sf/1402224  closed by  birkenfeld

Doc patch for classmethod and staticmethod  (2006-01-20)
       http://python.org/sf/1410783  closed by  birkenfeld

Make unittest.TestCase easier to subclass  (2005-12-22)
       http://python.org/sf/1388073  closed by  birkenfeld

Remove mention of DOS from os.py's docstring  (2006-01-20)
       http://python.org/sf/1410998  closed by  birkenfeld

fix UnixBrowswer failure when no browser running  (2005-12-03)
       http://python.org/sf/1372125  closed by  birkenfeld

Add DarwinPorts directories to setup.py  (2006-01-21)
       http://python.org/sf/1411588  closed by  bwarsaw

New / Reopened Bugs
___________________

bz2module with no long long type support  (2006-01-16)
CLOSED http://python.org/sf/1407069  opened by  kbob_ru

urlparse does not know about sftp: urls  (2006-01-17)
CLOSED http://python.org/sf/1407902  opened by  Stuart Bishop

email.Message should supress warning from uu.decode  (2006-01-18)
       http://python.org/sf/1409403  opened by  Mark Sapiro

frame->f_lasti not always correct  (2006-01-18)
       http://python.org/sf/1409443  opened by  John Ehresman

email.Message.set_payload followed by bad result get_payload  (2006-01-18)
       http://python.org/sf/1409455  opened by  Mark Sapiro

email.Utils.parseaddr() gives arcane result  (2006-01-18)
       http://python.org/sf/1409460  opened by  Mark Sapiro

email.Charset.py CODEC_MAP no longer requires 'japanese'  (2006-01-19)
       http://python.org/sf/1409538  opened by  Tokio Kikuchi

urllib2.urlopen() hangs due to use of socket._fileobject?  (2006-01-20)
       http://python.org/sf/1411097  opened by  John J Lee

"print statement" in libref index broken  (2006-01-21)
       http://python.org/sf/1411674  opened by  Skip Montanaro

XML.sax.saxutils.escape -- always escapes <, >, &,  (2006-01-21)
       http://python.org/sf/1411695  opened by  Mark Sandler

minor doc issues in os  (2006-01-22)
CLOSED http://python.org/sf/1412227  opened by  tiissa

locale.format gives wrong exception on some erroneous input  (2006-01-23)
       http://python.org/sf/1412580  opened by  Tim Diggins

__self - Watcom compiler reserved word  (2006-01-23)
       http://python.org/sf/1412837  opened by  kbob_ru

function usage not in #ifdef WITH_THREAD  #endif  (2006-01-23)
CLOSED http://python.org/sf/1412845  opened by  kbob_ru

bsddb: segfault on db.associate call with Txn and large data  (2006-01-23)
       http://python.org/sf/1413192  opened by  Alex Roitman

Bugs Closed
___________

segfault when using mmap(-1,...) [+PATCH]  (2006-01-10)
       http://python.org/sf/1402308  closed by  nnorwitz

2.3.5 RPM provides incompatibility  (2006-01-11)
       http://python.org/sf/1403225  closed by  jafo

locale.windows_locale   (2005-12-01)
       http://python.org/sf/1371247  closed by  birkenfeld

file.tell() returns illegal value on Windows  (2006-01-04)
       http://python.org/sf/1396471  closed by  birkenfeld

bz2module with no long long type support  (2006-01-15)
       http://python.org/sf/1407069  closed by  nnorwitz

TimedRotatingFileHandler at midnight rolls over at 01:00  (2006-01-03)
       http://python.org/sf/1396030  closed by  vsajip

TimedRotatingFileHandler does not recover from open error  (2006-01-03)
       http://python.org/sf/1396040  closed by  vsajip

urllib2 should be more robust for sloppy proxy URLs  (2004-02-22)
       http://python.org/sf/902075  closed by  birkenfeld

urlparse does not know about sftp: urls  (2006-01-17)
       http://python.org/sf/1407902  closed by  birkenfeld

in email can't get attachments' filenames using get_filename  (2006-01-11)
       http://python.org/sf/1403349  closed by  bwarsaw

email.Generators does not separates headers with "\r\n"  (2005-11-05)
       http://python.org/sf/1349106  closed by  bwarsaw

FeedParser does not comply with RFC2822  (2005-11-03)
       http://python.org/sf/1347874  closed by  bwarsaw

Odd warning behaviors  (2004-02-03)
       http://python.org/sf/890010  closed by  birkenfeld

Doc error on super(cls,self)  (2004-06-16)
       http://python.org/sf/973579  closed by  birkenfeld

CVS webbrowser.py (1.40) bugs  (2005-10-27)
       http://python.org/sf/1338995  closed by  birkenfeld

urllib2.HTTPBasicAuthHandler fails on non-default port  (2005-12-12)
       http://python.org/sf/1378679  closed by  birkenfeld

minor doc issues in os  (2006-01-22)
       http://python.org/sf/1412227  closed by  birkenfeld

function usage not in #ifdef WITH_THREAD  #endif  (2006-01-23)
       http://python.org/sf/1412845  closed by  mwh

RFE Closed
__________

python.org - add RFE tracker  (2006-01-05)
       http://python.org/sf/1397806  closed by  kbk


From fredrik at pythonware.com  Tue Jan 24 10:09:12 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 24 Jan 2006 10:09:12 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
References: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com><dq3u93$s4g$1@sea.gmane.org>
	<20060111223039.GC6642@activestate.com>
	<ee2a432c0601230036l61c79c4dp617d1521f514afd6@mail.gmail.com>
Message-ID: <dr4qrp$dhh$1@sea.gmane.org>

Neal Norwitz wrote:

> > > > Does that make sense?  We would just need /f's script in SVN.
> > >
> > > in python/Tools/something or sandbox/something ?
> >
> > python/Doc/tools/something?
>
> Fredrik were you still working on that?  I can make the changes to the
> bb master.  I thought Trent's suggested placement was good.

iirc, the script needed some minor tweaking (using os.path.splitext to
test for the module.so extension isn't a good idea), and I don't recall
if I've fixed that or not...

(probably not, since I never checked it in).

I'll take a look asap.

</F>




From fredrik at pythonware.com  Tue Jan 24 12:08:51 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 24 Jan 2006 12:08:51 +0100
Subject: [Python-Dev] New Pythondoc by effbot
References: <dqu4i9$ar9$1@sea.gmane.org><ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com><dqviud$n7b$1@sea.gmane.org><ca471dc20601220816j7fdc321lcf99db846dbf48fe@mail.gmail.com>
	<bbaeab100601221648h23efc94eu19f7cb706bd5dde3@mail.gmail.com>
Message-ID: <dr51s4$5q8$1@sea.gmane.org>

Brett Cannon wrote:

> And to /F, kudos from me.  I have been randomly thinking about it and
> I understand your desire for semantic markup now.

thanks.

> Hopefully something can get hammered out so that at least the Python
> 3 docs can premiere having been developed on by the whole community.

why wait for Python 3 ?

what's the current release plan for Python 2.5, btw?  I cannot find a
relevant PEP, and the "what's new" says "late 2005":

    http://www.python.org/dev/doc/devel/whatsnew/contents.html

</F>




From thomas at xs4all.net  Tue Jan 24 12:15:29 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 24 Jan 2006 12:15:29 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
Message-ID: <20060124111528.GB18916@xs4all.nl>

On Sun, Jan 22, 2006 at 11:01:36PM -0800, Neal Norwitz wrote:

> * test_pty is brittle on solaris 10, sometimes it works, sometimes not

FWIW, it's brittle on Solaris 9, too, and the SF compilefarm has two of
those. I don't know if it's the same problem, but on Solaris 9, the slave
part of the tty/pty pair sometimes isn't a TTY (according to os.isatty.) The
buildbot's log doesn't list the solaris 10 test_pty failure though, just the
test_logging failure.

It looks like a timing issue; the first run succeeds, all subsequent runs
fail, for a while, anyway. I'll do some googling and browsing other
tty/pty-using code to see if there's anything we're not doing we should be
doing, but it looks like a platform bug that we can't fix... Not without
re-implementing os.isatty anyway ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From abo at minkirri.apana.org.au  Tue Jan 24 14:05:34 2006
From: abo at minkirri.apana.org.au (Donovan Baarda)
Date: Tue, 24 Jan 2006 13:05:34 +0000
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <dquiqj$jvh$1@sea.gmane.org>
References: <dqu4i9$ar9$1@sea.gmane.org>
	<ca471dc20601211520h419be477y2ca0cb5e7d45b0ab@mail.gmail.com>
	<dquiqj$jvh$1@sea.gmane.org>
Message-ID: <1138107934.32075.47.camel@warna.dub.corp.google.com>

On Sat, 2006-01-21 at 19:15 -0500, Terry Reedy wrote:
> >> http://effbot.org/lib/os.path.join
> 
> On this page, 8 of 30 entries have a 'new in' comment.  For anyone with no 
> interest in the past, these constitute noise.  I wonder if for 3.0, the 

Even the past is relative... I find the "new in" doco absolutely
essential, because my "present" depends on what system I'm on, and some
of them are stuck in a serious time-warp. I do not have a time-machine
big enough to transport whole companies.

> timer can be reset and the docs start clean again.  To keep them backwards 
> compatible, they would also have to be littered with 'changed in 3.0' and 
> 'deleted in 3.0' entries.  Better, I think, to refer people to the last 2.x 
> docs and a separate 2.x/3.0 changes doc.

I also find "changed in" essential, but I don't mind not having "deleted
in"... it encourages developers stuck in those time-warps to avoid
features that get deleted in the future :-)

-- 
Donovan Baarda <abo at minkirri.apana.org.au>
http://minkirri.apana.org.au/~abo/


From fredrik at pythonware.com  Tue Jan 24 17:20:26 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue, 24 Jan 2006 17:20:26 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
References: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com><dq3u93$s4g$1@sea.gmane.org><20060111223039.GC6642@activestate.com><ee2a432c0601230036l61c79c4dp617d1521f514afd6@mail.gmail.com>
	<dr4qrp$dhh$1@sea.gmane.org>
Message-ID: <dr5k4c$agj$1@sea.gmane.org>

I wrote:

> Neal Norwitz wrote:
>
> > > > > Does that make sense?  We would just need /f's script in SVN.
> > > >
> > > > in python/Tools/something or sandbox/something ?
> > >
> > > python/Doc/tools/something?
> >
> > Fredrik were you still working on that?  I can make the changes to the
> > bb master.  I thought Trent's suggested placement was good.
>
> iirc, the script needed some minor tweaking (using os.path.splitext to
> test for the module.so extension isn't a good idea), and I don't recall
> if I've fixed that or not...
>
> (probably not, since I never checked it in).
>
> I'll take a look asap.

alright, I just checked in a

    Doc/tools/listmodules.py

which attempts to produce a sorted list of all modules available in
a given python build.  by default, it prints the list to stdout, which
should be good enough for a "catalog" buildbot step.

</F>




From skip at pobox.com  Tue Jan 24 19:27:43 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 24 Jan 2006 12:27:43 -0600
Subject: [Python-Dev] When will regex really go away?
Message-ID: <17366.29087.747411.639080@montanaro.dyndns.org>


I ran Fredrik's listmodules script in my current sandbox and got a
deprecation warning for the regex module.  According to PEP 4 it is already
obsolete.  I saw nothing there about the timeframe for actual removal.  Will
it ever go away?

Skip

From guido at python.org  Tue Jan 24 19:38:04 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jan 2006 10:38:04 -0800
Subject: [Python-Dev] When will regex really go away?
In-Reply-To: <17366.29087.747411.639080@montanaro.dyndns.org>
References: <17366.29087.747411.639080@montanaro.dyndns.org>
Message-ID: <ca471dc20601241038w3dd3af76p57da6a19935f2f86@mail.gmail.com>

On 1/24/06, skip at pobox.com <skip at pobox.com> wrote:
>
> I ran Fredrik's listmodules script in my current sandbox and got a
> deprecation warning for the regex module.  According to PEP 4 it is already
> obsolete.  I saw nothing there about the timeframe for actual removal.  Will
> it ever go away?

ASAP please!

PEP 4 lists these that were already obsolete in 2.0:

    addpack, cmp, cmpcache, codehack, dircmp, dump, fmt, lockfile,
    newdir, Para, poly, regex, regsub, tb, timing, util, whatsound,
    tzmod, find, grep, packmail, ni, rand, soundex, cl, sv

of these, regex, regsub, and timing are still importable in 2.4 (I
don't have a 2.5 handy here at Google, and my home machine seems
inaccessible). ISTM these last three can safely be dropped now.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From crutcher at gmail.com  Tue Jan 24 20:24:46 2006
From: crutcher at gmail.com (Crutcher Dunnavant)
Date: Tue, 24 Jan 2006 11:24:46 -0800
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused
	as global dictionaries
In-Reply-To: <43C6E25F.2040606@v.loewis.de>
References: <d49fe110601111603q336dc718v9381024c7a8a2237@mail.gmail.com>
	<001201c61756$cc2d8780$5916c797@oemcomputer>
	<d49fe110601121354u6628da58j10de76961514e463@mail.gmail.com>
	<43C6E25F.2040606@v.loewis.de>
Message-ID: <d49fe110601241124u6b6c8899r55f2302754ef9622@mail.gmail.com>

Okay, but is there any reason not to include this in 2.5? There
doesn't seem to be any noticeable performance impact, and it does add
consistancy (and opens some really, really cool options up).

Does anyone have objections to 1402289?

On 1/12/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Crutcher Dunnavant wrote:
> > I sorta disagree about it not being broken. Adding a feature which
> > works for eval but not for exec seems pretty broken.
>
> You "seem" to have a different notion of "to be broken", then.
>
> Python is broken, if and only if
> - the interpreter crashes, for any Python input
> - the implementation does not do what the documentation says
>   it would do
> - the BDFL pronounces it is broken
>
> In this specific change, the change did precisely what the requestor
> of the feature wanted it to do (that eval could accept non-exact
> dicts was a new feature back then also)
>
> > It's difficult to
> > reason about what will happen in the exec context, so I can't see what
> > fixing it would endanger; but I'd deffinately like to see it for 2.5.
>
> It would make Python code run which is currently rejected with an
> exception. Therefore, it is a new feature (a behaviour change).
>
> Applications relying on that feature would have to specify that
> they require "2.4.3"; people would find that code that runs fine
> in 2.4.3 fails in 2.4.2. This is not acceptable.
>
> Regards,
> Martin
>


--
Crutcher Dunnavant <crutcher at gmail.com>
monket.samedi-studios.com

From skip at pobox.com  Tue Jan 24 20:33:08 2006
From: skip at pobox.com (skip at pobox.com)
Date: Tue, 24 Jan 2006 13:33:08 -0600
Subject: [Python-Dev] When will regex really go away?
In-Reply-To: <ca471dc20601241038w3dd3af76p57da6a19935f2f86@mail.gmail.com>
References: <17366.29087.747411.639080@montanaro.dyndns.org>
	<ca471dc20601241038w3dd3af76p57da6a19935f2f86@mail.gmail.com>
Message-ID: <17366.33012.687343.786721@montanaro.dyndns.org>


    Guido> PEP 4 lists these that were already obsolete in 2.0:

    Guido>     addpack, cmp, cmpcache, codehack, dircmp, dump, fmt,
    Guido>     lockfile, newdir, Para, poly, regex, regsub, tb, timing,
    Guido>     util, whatsound, tzmod, find, grep, packmail, ni, rand,
    Guido>     soundex, cl, sv

    Guido> of these, regex, regsub, and timing are still importable in 2.4.

And in 2.5.  Regex and regsub both import with deprecation warnings.  Timing
imports without a peep.  None of the others import.

Skip

From bjourne at gmail.com  Tue Jan 24 21:22:01 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Tue, 24 Jan 2006 21:22:01 +0100
Subject: [Python-Dev] The path module PEP
Message-ID: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>

The last time this was discussed six months ago it seemed like most of
python-dev fancied Jason Orendorff's path module. But Guido wanted a
PEP and noone created one. So I decided to claim the fame and write
one since I also love the path module. :) Much of it is copy-pasted
from Peter Astrand's PEP 324, many thanks to him.


#################################################
PEP: XXX
Title: path - Object oriented handling of filesystem paths
Version: $Revision: XXXX $
Last-Modified: $Date: 2006-01-24 19:24:59 +0100 (Sat, 29 Jan 2005) $
Author: Bj?rn Lindqvist <bjourne at gmail.com>
Status: Dummo
Type: Standards Track (library)
Created: 24-Jan-2006
Content-Type: text/plain
Python-Version: 2.4


Abstract

    This PEP describes a new class for handling paths in an object
    oriented fashion.


Motivation

    Dealing with filesystem paths is a common task in any programming
    language, and very common in a high-level language like
    Python. Good support for this task is needed, because:

    - Almost every program uses paths to access files. It makes sense
      that a task, that is so often performed, should be as intuitive
      and as easy to perform as possible.

    - It makes Python an even better replacement language for
      over-complicated shell scripts.

    Currently, Python has a large number of different functions
    scattered over half a dozen modules for handling paths. This makes
    it hard for newbies and experienced developers to to choose the
    right method.

    The Path class provides the following enhancements over the
    current common practice:

    - One "unified" object provides all functionality from previous
      functions.

    - Subclassability - the Path object can be extended to support
      paths other than filesystem paths. The programmer does not need
      to learn a new API, but can reuse his or her knowledge of Path
      to deal with the extended class.

    - With all related functionality in one place, the right approach
      is easier to learn as one does not have to hunt through many
      different modules for the right functions.

    - Python is an object oriented language. Just like files,
      datetimes and sockets are objects so are paths, they are not
      merely strings to be passed to functions. Path objects are
      inherently a pythonic idea.

    - Path takes advantage of properties. Properties make for more
      readable code.

      if imgpath.ext == 'jpg':
          jpegdecode(imgpath)

      Is better than:

      if os.path.splitexit(imgpath)[1] == 'jpg':
          jpegdecode(imgpath)


Rationale

    The following points summarizes the design:

    - Path extends from string, therefore all code which expects
      string pathnames need not be modified and no existing code will
      break.

    - A Path object can be created either by using the classmethod
      Path.cwd, by instantiating the class with a string representing
      a path or by using the default constructor which is equivalent
      with Path(".").

    - The factory functions in popen2 have been removed, because I
      consider the class constructor equally easy to work with.

    - Path provides for common pathname manipulation, pattern
      expansion, pattern matching and other high-level file operations
      including copying. Basically everything path related except for
      manipulation of file contents which file objects are better
      suited for.

    - Platform incompatibilites are dealt with by not instantiating
      system specific methods.


Specification

    This class defines the following public methods:

        # Special Python methods.
        def __new__(cls, init = os.curdir): ...
        def __repr__(self): ...
        def __add__(self, more): ...
        def __radd__(self, other): ...
        def __div__(self, rel): ...
        def __truediv__(self, rel): ...

        # Alternative constructor.
        def cwd(cls): ...

        # Operations on path strings.
        def abspath(sef): ...
        def normcase(self): ...
        def normpath(self): ...
        def realpath(self): ...
        def expanduser(self): ...
        def expandvars(self): ...
        def dirname(self): ...
        def basename(self): ...
        def expand(self): ...
        def splitpath(self): ...
        def splitdrive(self): ...
        def splitext(self): ...
        def stripext(self): ...
        def splitunc(self): ... [1]
        def joinpath(self, *args): ...
        def splitall(self): ...
        def relpath(self): ...
        def relpathto(self, dest): ...

        # Properties about the path.
        parent, name, namebase, ext, drive, uncshare[1]

        # Operations that return lists of paths.
        def listdir(self, pattern = None): ...
        def dirs(self, pattern = None): ...
        def files(self, pattern = None): ...
        def walk(self, pattern = None): ...
        def walkdirs(self, pattern = None): ...
        def walkfiles(self, pattern = None): ...
        def match(self, pattern):
        def matchcase(self, pattern):
        def glob(self, pattern):

        # Methods for retrieving information about the filesystem
        # path.
        def exists(self): ...
        def isabs(self): ...
        def isdir(self): ...
        def isfile(self): ...
        def islink(self): ...
        def ismount(self): ...
        def samefile(self, other): ... [1]
        def getatime(self): ...
        def getmtime(self): ...
        def getctime(self): ...
        def getsize(self): ...
        def access(self, mode): ... [1]
        def stat(self): ...
        def lstat(self): ...
        def statvfs(self): ... [1]
        def pathconf(self, name): ... [1]
        def utime(self, times): ...
        def chmod(self, mode): ...
        def chown(self, uid, gid): ... [1]
        def rename(self, new): ...
        def renames(self, new):

        # Filesystem properties for path.
        atime, getmtime, getctime, size

        # Methods for manipulating information about the filesystem
        # path.
        def utime(self, times): ...
        def chmod(self, mode): ...
        def chown(self, uid, gid): ... [1]
        def rename(self, new): ...
        def renames(self, new): ...

        # Create/delete operations on directories
        def mkdir(self, mode = 0777): ...
        def makedirs(self, mode = 0777): ...
        def rmdir(self): ...
        def removedirs(self): ...

        # Modifying operations on files
        def touch(self): ...
        def remove(self): ...
        def unlink(self): ...

        # Modifying operations on links
        def link(self, newpath): ...
        def symlink(self, newlink): ...
        def readlink(self): ...
        def readlinkabs(self): ...

        # High-level functions from shutil
        def copyfile(self, dst): ...
        def copymode(self, dst): ...
        def copystat(self, dst): ...
        def copy(self, dst): ...
        def copy2(self, dst): ...
        def copytree(self, dst, symlinks = True): ...
        def move(self, dst): ...
        def rmtree(self, ignore_errors = False, onerror = None): ...

        # Special stuff from os
        def chroot(self): ... [1]
        def startfile(self): ... [1]

    [1] - Method is not availible on all platforms.


Replacing older functions with the Path class

    In this section, "a ==> b" means that b can be used as a
    replacement for a.

    In the following examples, we assume that the Path class is
    imported with "from path import Path".

    Replacing os.path.join
    ----------------------

    os.path.join(os.getcwd(), "foobar")
    ==>
    Path.cwd() / "foobar"


    Replacing os.path.splitext
    --------------------------

    os.path.splitext("Python2.4.tar.gz")[1]
    ==>
    Path("Python2.4.tar.gz").ext


    Replacing glob.glob
    -------------------

    glob.glob("/lib/*.so")
    ==>
    Path("/lib").glob("*.so")


Deprecations

    Introducing this module to the standard library introduces the
    need to deprecate a number of existing modules and functions. The
    table below explains which existing functionality that must be
    deprecated.

        PATH METHOD         DEPRECATES FUNCTION
        normcase()          os.path.normcase()
        normpath()          os.path.normpath()
        realpath()          os.path.realpath()
        expanduser()        os.path.expanduser()
        expandvars()        os.path.expandvars()
        dirname()           os.path.dirname()
        basename()          os.path.basename()
        splitpath()         os.path.split()
        splitdrive()        os.path.splitdrive()
        splitext()          os.path.splitext()
        splitunc()          os.path.splitunc()
        joinpath()          os.path.join()
        listdir()           os.listdir() [fnmatch.filter()]
        match()             fnmatch.fnmatch()
        matchcase()         fnmatch.fnmatchcase()
        glob()              glob.glob()
        exists()            os.path.exists()
        isabs()             os.path.isabs()
        isdir()             os.path.isdir()
        isfile()            os.path.isfile()
        islink()            os.path.islink()
        ismount()           os.path.ismount()
        samefile()          os.path.samefile()
        getatime()          os.path.getatime()
        getmtime()          os.path.getmtime()
        getsize()           os.path.getsize()
        cwd()               os.getcwd()
        access()            os.access()
        stat()              os.stat()
        lstat()             os.lstat()
        statvfs()           os.statvfs()
        pathconf()          os.pathconf()
        utime()             os.utime()
        chmod()             os.chmod()
        chown()             os.chown()
        rename()            os.rename()
        renames()           os.renames()
        mkdir()             os.mkdir()
        makedirs()          os.makedirs()
        rmdir()             os.rmdir()
        removedirs()        os.removedirs()
        remove()            os.remove()
        unlink()            os.unlink()
        link()              os.link()
        symlink()           os.symlink()
        readlink()          os.readlink()
        chroot()            os.chroot()
        startfile()         os.startfile()
        copyfile()          shutil.copyfile()
        copymode()          shutil.copymode()
        copystat()          shutil.copystat()
        copy()              shutil.copy()
        copy2()             shutil.copy2()
        copytree()          shutil.copytree()
        move()              shutil.move()
        rmtree()            shutil.rmtree()

    The Path class deprecates the whole of os.path, shutil, fnmatch
    and glob. A big chunk of os is also deprecated.


Open Issues

    Some functionality of Jason Orendorff's path module have been
    omitted:

    * Function for opening a path - better handled by the builtin
      open().

    * Functions for reading and writing a whole file - better handled
      by file objects read() and write() methods.

    * A chdir() function may be a worthy inclusion.

    * A deprecation schedule needs to be setup. How much functionality
      should Path implement? How much of existing functionality should
      it deprecate and when?

    * Where should the class be placed and what should it be called?

    The functions and modules that this new module is trying to
    replace (os.path, shutil, fnmatch, glob and parts of os are
    expected to be available in future Python versions for a long
    time, to preserve backwards compatibility.


Reference Implementation

    Currently, the Path class is implemented as a thin wrapper around
    the standard library modules sys, os, fnmatch, glob and
    shutil. The intention of this PEP is to move functionality from
    the aforementioned modules to Path while they are being
    deprecated.

    For more detail, and diverging implementations see:

        * http://www.jorendorff.com/articles/python/path/path.py
        * http://svn.python.org/projects/sandbox/trunk/path/path.py
        * http://cafepy.com/quixote_extras/rex/path/enhpath.py


Examples

    In this section, "a ==> b" means that b can be used as a
    replacement for a.

    1. Make all python files in the a directory executable:

        DIR = '/usr/home/guido/bin'
        for f in os.listdir(DIR):
            if f.endswith('.py'):
                path = os.path.join(DIR, f)
                os.chmod(path, 0755)
        ==>
        for f in Path('/usr/home/guido/bin'):
            f.chmod(0755)

    2. Delete emacs backup files:

        def delete_backups(arg, dirname, names):
            for name in names:
                if name.endswith('~'):
                    os.remove(os.path.join(dirname, name))
        ==>
        d = Path(os.environ['HOME'])
        for f in d.walkfiles('*~'):
            f.remove()

    3. Finding the relative path to a file:

        b = Path('/users/peter/')
        a = Path('/users/peter/synergy/tiki.txt')
        a.relpathto(b)

    4. Splitting a path into directory and filename:

        os.path.split("/path/to/foo/bar.txt")
        ==>
        Path("/path/to/foo/bar.txt").splitpath()

    5. List all Python scripts in the current directory tree:

        list(Path().walkfiles("*.py"))

    6. Create directory paths:

        os.path.join("foo", "bar", "baz")
        ==>
        Path("foo") / "bar" / "baz"


Copyright

    This document has been placed in the public domain.


Local Variables:
mode: indented-text
indent-tabs-mode: nil
sentence-end-double-space: t
fill-column: 70
End:

--
mvh Bj?rn

From greg at electricrain.com  Tue Jan 24 21:22:38 2006
From: greg at electricrain.com (Gregory P. Smith)
Date: Tue, 24 Jan 2006 12:22:38 -0800
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
Message-ID: <20060124202238.GM17240@zot.electricrain.com>

On Sun, Jan 22, 2006 at 11:01:36PM -0800, Neal Norwitz wrote:
> rather than later.  There are a bunch of tests that are not stable. 
> It would really help to get people knowledgeable about a particular
> subdomain to provide input into bugs/patches and produce patches too!
> 
> The areas that are known to have problems are listed here:
>     http://wiki.python.org/moin/BuildBot
> 
> It would be nice to clean those issues up.  Does anyone have some time
> to spend to resolve the issues with Berkeley DB?  That's the one that
> annoys me most right now.  I have a patch that fixes at least one of
> the problems, but no idea if it's the correct solution or not. 
> http://python.org/sf/1407992

Nice!  It does fix the associate test problem on BerkeleyDB 3.3-4.1
and 4.2-4.4 continue to pass.  I committed it.  Good to have tests
passing again when using "old" but still common BerkeleyDBs.

> * BSD DB 4.1 and 3.2 fail

Using BerkeleyDB 3.2 often segfaults for me; using 3.3 often hangs in
the test suite.  Both are so old I don't see much motivation to track
the issues down.


From python at discworld.dyndns.org  Tue Jan 24 21:56:51 2006
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Tue, 24 Jan 2006 14:56:51 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <20060124205651.GA28435@discworld.dyndns.org>

BJ?rn Lindqvist <bjourne at gmail.com> wrote:
> 
>     1. Make all python files in the a directory executable:
[...]
>         ==>
>         for f in Path('/usr/home/guido/bin'):
>             f.chmod(0755)

Iterating over a path string to read the contents of the directory possibly
pointed to by that string seems like "magic implicit" behaviour.  Perhaps
making it a method explicitly returning an iterator would by more Pythonic?

    for f in Path(...).readDir():

>     4. Splitting a path into directory and filename:
[...]
>         Path("/path/to/foo/bar.txt").splitpath()

Good.  But the opposite isn't done similarly:

>     6. Create directory paths:
[...]
>         Path("foo") / "bar" / "baz"

Using "/" as "path concatenation operator" seems like un-Pythonic magic as
well (while "+" would be an improvement, it's still not a large one).  I would
think 

    Path('foo').appendparts('bar', 'baz')

or similar would be more readable and obvious.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python at discworld.dyndns.org>
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From eric.nieuwland at xs4all.nl  Tue Jan 24 22:18:52 2006
From: eric.nieuwland at xs4all.nl (Eric Nieuwland)
Date: Tue, 24 Jan 2006 22:18:52 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <22e4d10c08d514bbe2f8db0dd186e015@xs4all.nl>

Good stuff. Some suggestions:

>         def joinpath(self, *args): ...
I suggest append() or extend() as join*() sort of suggest join() as 
provided by strings, which does something quite different

>         def splitall(self): ...
and this may renamed split(), as it is quite similar to split() as 
provided by strings

>         # Properties about the path.
>         parent, name, namebase, ext, drive, uncshare[1]
so we can drop basename(), dirname(), splitdrive(), and splitext()

>         def dirs(self, pattern = None): ...
>         def files(self, pattern = None): ...
can we add others()? (sockets, pipes, block and character devices)

--eric


From jason.orendorff at gmail.com  Tue Jan 24 22:18:30 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Tue, 24 Jan 2006 16:18:30 -0500
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>

Thanks for doing this.  I'm not sure anyone that matters here is
actually keen on path, but I guess we'll see.  A few comments:

On 1/24/06, BJ?rn Lindqvist <bjourne at gmail.com> wrote:
>     The following points summarizes the design:
>
>     - Path extends from string, therefore all code which expects
>       string pathnames need not be modified and no existing code will
>       break.

Actually, I would prefer a Path that *didn't* subclass string, and a
new "%p" format-thingy in PyArg_ParseTuple().  %p would expect either
a Path object or a string.  Stdlib C functions that take paths would
be changed from using %s or %u to %p.  This would pretty much
eliminate the need for path objects to act like strings (except where
__cmp__, __hash__, and common sense dictate).

The only reason I didn't do this in path.py is that I don't have the
required write access to the Python source tree. ;)  Subclassing
str/unicode seemed like the next best thing.


>     [...]omitted:
>     * Function for opening a path - better handled by the builtin
>       open().

Aside:  I added this to support a few people who liked the idea of
"openable objects", meaning anything that has .open(), analogous to
"writeable objects" being anything with .write().  I don't use it
personally.

Examples 1 and 2 have errors.  In example 1, the "after" code should be:

  d = path('/usr/home/guido/bin')
  for f in d.files('*.py'):
      f.chmod(0755)

In example 2, the "before" code is missing a line -- the call to
os.path.walk().  (Actually it should probably use os.walk(), which
looks much nicer.)

I suspect you'll be asked to change the PEP to remove __div__ for
starters, in which case I propose using the Path constructor as the
replacement for os.path.join().  In that case, Path.joinpath can be
dropped.

-j

From ianb at colorstudy.com  Tue Jan 24 22:28:05 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 24 Jan 2006 15:28:05 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <20060124205651.GA28435@discworld.dyndns.org>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<20060124205651.GA28435@discworld.dyndns.org>
Message-ID: <43D69BE5.2030809@colorstudy.com>

Charles Cazabon wrote:
> BJ?rn Lindqvist <bjourne at gmail.com> wrote:
> 
>>    1. Make all python files in the a directory executable:
> 
> [...]
> 
>>        ==>
>>        for f in Path('/usr/home/guido/bin'):
>>            f.chmod(0755)
> 
> 
> Iterating over a path string to read the contents of the directory possibly
> pointed to by that string seems like "magic implicit" behaviour.  Perhaps
> making it a method explicitly returning an iterator would by more Pythonic?
> 
>     for f in Path(...).readDir():

I believe .listdir() exists already as a method alternative.  I'm -0 on 
iteration as listdir.  Doing iteration like strings (over the 
characters) would be evil.

>>    4. Splitting a path into directory and filename:
> 
> [...]
> 
>>        Path("/path/to/foo/bar.txt").splitpath()
> 
> 
> Good.  But the opposite isn't done similarly:
> 
> 
>>    6. Create directory paths:
> 
> [...]
> 
>>        Path("foo") / "bar" / "baz"
> 
> 
> Using "/" as "path concatenation operator" seems like un-Pythonic magic as
> well (while "+" would be an improvement, it's still not a large one).  I would
> think 
> 
>     Path('foo').appendparts('bar', 'baz')
> 
> or similar would be more readable and obvious.

.joinpath() does this.  Though .join() does something else entirely that 
it inherits from strings, something evil to do with a path, and I think 
that method should raise NotImplementedError.  + should not be 
overridden, because strings define that.

Some other str methods are harmless but pointless: center, expandtabs, 
ljust, zfill; title, capitalize, and istitle are iffy.  Also, are there 
any particular cases where string methods on a path produce an actual 
str, not another Path object?


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Tue Jan 24 22:37:46 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 24 Jan 2006 15:37:46 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
Message-ID: <43D69E2A.6050205@colorstudy.com>

Jason Orendorff wrote:
>>    [...]omitted:
>>    * Function for opening a path - better handled by the builtin
>>      open().
> 
> 
> Aside:  I added this to support a few people who liked the idea of
> "openable objects", meaning anything that has .open(), analogous to
> "writeable objects" being anything with .write().  I don't use it
> personally.

Losing .open() would make it much harder for anyone wanting to write, 
say, a URI library that implements the Path API.

> I suspect you'll be asked to change the PEP to remove __div__ for
> starters, in which case I propose using the Path constructor as the
> replacement for os.path.join().  In that case, Path.joinpath can be
> dropped.

I'm -1 on this too.  This means people will be hardcoding the specific 
class they expect, so you can't pass in other classes.  E.g., this will 
fail:

   def read_config(home_dir):
       f = open(Path(home_dir, '.config_file'))
       c = f.read()
       f.close()
       return c

   read_config(URI('http://localhost/foo'))

I'm personally +1 on /.  I think people who think it is confusing are 
giving a knee-jerk reaction.  It's very easy to tell the difference 
between file-related code and math-related code, and / is currently only 
used for math.  In contrast, + is used for concatenation and addition, 
and these are far more ambiguous from context -- but still it doesn't 
cause that many problems.  But barring /, .joinpath() should remain. 
Too bad there isn't a shorter name, except .join() which is taken.

I would also, though, note that there are some security issues.  When 
you use open() or other path-related functions, you *know* you are doing 
filesystem operations.  You can't be getting some weird object that does 
who-knows-what.  For the most part if you can get an arbitrary object 
into the system then the system is just broken, but I can imagine cases 
where this encourages people to do bad things.  I only bring this up 
because it reminds me of PHP allowed over-the-net includes, which always 
seemed like a bad idea.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Tue Jan 24 22:58:36 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 24 Jan 2006 15:58:36 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <43D6A30C.30205@colorstudy.com>

BJ?rn Lindqvist wrote:
>     * Functions for reading and writing a whole file - better handled
>       by file objects read() and write() methods.

I would be disappointed to see this left out, because I get really tired 
of this little dance:

   f = open(filename)
   c = f.read()
   f.close()
   return c

Then you can put a try:finally: in there too for extra correctness. 
Anyway, I write this code all the time, and it's really tiresome. 
open(filename).read() also works, but relies on Python's reference 
counting to be really reliable; maybe I shouldn't worry about that and 
use just that form.  But that bothers me too.  The same occurs during 
writing.

The original Path object has a bytes and text method (I think), which 
nicely distinguishes between the two cases.  This helps suggest better 
and more explicit handling of unicode, something that Python should work 
harder at making clear.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ncoghlan at gmail.com  Tue Jan 24 23:10:15 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Wed, 25 Jan 2006 08:10:15 +1000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D6A30C.30205@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6A30C.30205@colorstudy.com>
Message-ID: <43D6A5C7.6080008@gmail.com>

Ian Bicking wrote:
> BJ?rn Lindqvist wrote:
>>     * Functions for reading and writing a whole file - better handled
>>       by file objects read() and write() methods.
> 
> I would be disappointed to see this left out, because I get really tired 
> of this little dance:
> 
>    f = open(filename)
>    c = f.read()
>    f.close()
>    return c

Python 2.5 (well, once someone finds time to update mwh's patch):

   with open(filename) as f:
     return f.read()

Behaviour guaranteed by language definition ;)

Cheers,
Nick.

P.S. I too would really like to see this happen for 2.5.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Tue Jan 24 23:22:21 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 24 Jan 2006 17:22:21 -0500
Subject: [Python-Dev] Long-time shy failure in test_socket_ssl
Message-ID: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com>

Has anyone else noticed this?  For a long time (possibly years), I see
an infrequent error in test_socket_ssl, like so (this is on WinXP
Pro):

test_socket_ssl
test test_socket_ssl crashed -- exceptions.TypeError: 'NoneType'
object is not callable

I haven't been able to provoke it by running test_socket_ssl in a
loop, and I don't have a guess about what's going wrong by eyeballing
the code.

test_rude_shutdown() is dicey, relying on a sleep() instead of proper
synchronization to make it probable that the `listener` thread goes
away before the main thread tries to connect, but while that race may
account for bogus TestFailed deaths, it doesn't seem possible that it
could account for the kind of failure above.

From martin at v.loewis.de  Tue Jan 24 23:52:48 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Jan 2006 23:52:48 +0100
Subject: [Python-Dev] [PATCH] Fix dictionary subclass semantics whenused
 as global dictionaries
In-Reply-To: <d49fe110601241124u6b6c8899r55f2302754ef9622@mail.gmail.com>
References: <d49fe110601111603q336dc718v9381024c7a8a2237@mail.gmail.com>	
	<001201c61756$cc2d8780$5916c797@oemcomputer>	
	<d49fe110601121354u6628da58j10de76961514e463@mail.gmail.com>	
	<43C6E25F.2040606@v.loewis.de>
	<d49fe110601241124u6b6c8899r55f2302754ef9622@mail.gmail.com>
Message-ID: <43D6AFC0.5080602@v.loewis.de>

Crutcher Dunnavant wrote:
> Okay, but is there any reason not to include this in 2.5? There
> doesn't seem to be any noticeable performance impact, and it does add
> consistancy (and opens some really, really cool options up).

I see no reason, except perhaps the lack of volunteers to actually
patch the repository (along with the accompanying work).

Regards,
Martin


From martin at v.loewis.de  Tue Jan 24 23:52:52 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 24 Jan 2006 23:52:52 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <20060124111528.GB18916@xs4all.nl>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl>
Message-ID: <43D6AFC4.3030200@v.loewis.de>

Thomas Wouters wrote:
> FWIW, it's brittle on Solaris 9, too, and the SF compilefarm has two of
> those. I don't know if it's the same problem, but on Solaris 9, the slave
> part of the tty/pty pair sometimes isn't a TTY (according to os.isatty.) The
> buildbot's log doesn't list the solaris 10 test_pty failure though, just the
> test_logging failure.

It occasionally failed in test_pty, too, with the very same error
message (that /dev/tty isn't a tty).

> It looks like a timing issue; the first run succeeds, all subsequent runs
> fail, for a while, anyway. I'll do some googling and browsing other
> tty/pty-using code to see if there's anything we're not doing we should be
> doing, but it looks like a platform bug that we can't fix... Not without
> re-implementing os.isatty anyway ;P

Couldn't there be a bug in openpty instead? Perhaps it is trying to
allocate the same device again, but fails to do so correctly, and fails
to recognize that it should use a different one instead.

Anyway, if you still think you need an OS 10 account, please let me
know, and I can give you an account to the machine the buildbot
runs on.

Regards,
Martin


From phd at mail2.phd.pp.ru  Tue Jan 24 23:24:18 2006
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Wed, 25 Jan 2006 01:24:18 +0300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <20060124222418.GA1409@phd.pp.ru>

On Tue, Jan 24, 2006 at 09:22:01PM +0100, BJ?rn Lindqvist wrote:
>         Path("foo") / "bar" / "baz"

   I really love this! But I am afraid it's too much a Unixism. (-:

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From gjc at inescporto.pt  Tue Jan 24 22:34:28 2006
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Tue, 24 Jan 2006 21:34:28 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <1138138468.6383.3.camel@localhost.localdomain>

On Tue, 2006-01-24 at 21:22 +0100, BJ?rn Lindqvist wrote:
[...]

>         # Operations on path strings.
>         def abspath(sef): ...
>         def normcase(self): ...
>         def normpath(self): ...
>         def realpath(self): ...
>         def expanduser(self): ...
>         def expandvars(self): ...
>         def dirname(self): ...
>         def basename(self): ...
>         def expand(self): ...
>         def splitpath(self): ...
>         def splitdrive(self): ...
>         def splitext(self): ...
>         def stripext(self): ...
>         def splitunc(self): ... [1]
>         def joinpath(self, *args): ...
>         def splitall(self): ...
>         def relpath(self): ...
>         def relpathto(self, dest): ...
[...etc...]

  If we wanted to take PEP 8 seriously, those method names should be
changed to words_separated_by_underscores.

  And BTW, what does splitunc do?  It really should have a more
descriptive name.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic


From ianb at colorstudy.com  Wed Jan 25 00:03:37 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 24 Jan 2006 17:03:37 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D69E2A.6050205@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
	<43D69E2A.6050205@colorstudy.com>
Message-ID: <43D6B249.8090603@colorstudy.com>

Ian Bicking wrote:
> I'm -1 on this too.  This means people will be hardcoding the specific 
> class they expect, so you can't pass in other classes.  E.g., this will 
> fail:
> 
>    def read_config(home_dir):
>        f = open(Path(home_dir, '.config_file'))
>        c = f.read()
>        f.close()
>        return c
> 
>    read_config(URI('http://localhost/foo'))

It occurs to me that it might be hopeless to expect substitution to work 
generally (at least without a specific thought on the matter) because I 
expect this form will be typical:

   def read_config(path):
       # convert string input to a path (has no effect on Path objects):
       path = Path(path)
       content = path.text()

Since people will be passing strings in to file-related functions for 
the forseeable future, so people will coerce that input to paths 
explicitly whenever they accept a path from a public function.

Now, if there were a way to make sure that "Path(x) is x" is true when x 
is already a Path, and maybe a way to coerce strings to a Path without 
coercing Path-like objects into Path objects, that would help resolve 
the problem.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From trentm at ActiveState.com  Wed Jan 25 00:03:59 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Tue, 24 Jan 2006 15:03:59 -0800
Subject: [Python-Dev] The path module PEP
In-Reply-To: <1138138468.6383.3.camel@localhost.localdomain>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<1138138468.6383.3.camel@localhost.localdomain>
Message-ID: <20060124230359.GB32313@activestate.com>

[Gustavo J. A. M. Carneiro wrote]
>   And BTW, what does splitunc do? 

http://en.wikipedia.org/wiki/Path_%28computing%29#Universal_Naming_Convention

> It really should have a more descriptive name.

No more that should "urllib" or "splitext".

Trent

-- 
Trent Mick
TrentM at ActiveState.com

From t-meyer at ihug.co.nz  Tue Jan 24 23:59:50 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed, 25 Jan 2006 11:59:50 +1300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <EA38A386-44BE-4DA6-BE7B-0DA189C1C9E9@ihug.co.nz>

> The last time this was discussed six months ago it seemed like most of
> python-dev fancied Jason Orendorff's path module. But Guido wanted a
> PEP and noone created one. So I decided to claim the fame and write
> one since I also love the path module. :) Much of it is copy-pasted
> from Peter Astrand's PEP 324, many thanks to him.

It would be great (and save a lot of rehashing) if you could go over  
the python-dev discussion and add the relevant parts (for example,  
whether to include the __div__ hack) to the PEP:

<http://mail.python.org/pipermail/python-dev/2005-June/054439.html>

=Tony.Meyer

From fredrik at pythonware.com  Wed Jan 25 00:25:21 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed, 25 Jan 2006 00:25:21 +0100
Subject: [Python-Dev] The path module PEP
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<1138138468.6383.3.camel@localhost.localdomain>
Message-ID: <dr6d14$i2n$1@sea.gmane.org>

Gustavo J. A. M. Carneiro wrote:

> >         # Operations on path strings.
> >         def abspath(sef): ...
> >         def normcase(self): ...
> >         def normpath(self): ...
> >         def realpath(self): ...
> >         def expanduser(self): ...
> >         def expandvars(self): ...
> >         def dirname(self): ...
> >         def basename(self): ...
> >         def expand(self): ...
> >         def splitpath(self): ...
> >         def splitdrive(self): ...
> >         def splitext(self): ...
> >         def stripext(self): ...
> >         def splitunc(self): ... [1]
> >         def joinpath(self, *args): ...
> >         def splitall(self): ...
> >         def relpath(self): ...
> >         def relpathto(self, dest): ...
> [...etc...]
>
>   If we wanted to take PEP 8 seriously, those method names should be
> changed to words_separated_by_underscores.
>   And BTW, what does splitunc do?  It really should have a more
> descriptive name.

when did you last use the os.path module ?

</F>




From ianb at colorstudy.com  Wed Jan 25 00:36:36 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 24 Jan 2006 17:36:36 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <1138138468.6383.3.camel@localhost.localdomain>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<1138138468.6383.3.camel@localhost.localdomain>
Message-ID: <43D6BA04.8010409@colorstudy.com>

Gustavo J. A. M. Carneiro wrote:.
>>        def splitall(self): ...
>>        def relpath(self): ...
>>        def relpathto(self, dest): ...
> 
> [...etc...]
> 
>   If we wanted to take PEP 8 seriously, those method names should be
> changed to words_separated_by_underscores.

There's a (unspecified?) convention that many standard/core objects or 
objects in the standard library use squishedwords for methods.  has_key 
is an anomoly, not the norm.

Also, many of these are direct translations of methods from os.path, and 
so the names offer familiarity.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Wed Jan 25 00:38:45 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Tue, 24 Jan 2006 17:38:45 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <43D6BA85.8070007@colorstudy.com>

There's kind of a lot of methods in here, which is a little bothersome. 
  It also points towards the motivation for the class -- too many 
options in too many places in the stdlib.  But throwing them *all* in 
one class consolidates but doesn't simplify, especially with duplicate 
functionality.

I'm not strongly advocating any of the methods below be removed, but at 
least it seems like it should be discussed.  By not removing any it is 
easier to translate the os.path (and other) forms.  I imagine it will be 
a long time before those forms go away, though, so I don't know how 
useful it is to plan for a speedy and seamless transition.

BJ?rn Lindqvist wrote:
>     This class defines the following public methods:
> 
>         # Special Python methods.
>         def __new__(cls, init = os.curdir): ...
>         def __repr__(self): ...
>         def __add__(self, more): ...
>         def __radd__(self, other): ...
>         def __div__(self, rel): ...
>         def __truediv__(self, rel): ...
> 
>         # Alternative constructor.
>         def cwd(cls): ...
> 
>         # Operations on path strings.
>         def abspath(sef): ...
>         def normcase(self): ...
>         def normpath(self): ...
>         def realpath(self): ...
>         def expanduser(self): ...
>         def expandvars(self): ...

This is equivalent to 
p.__class__(string.Template(p).safe_substitute(os.environ)).  Obviously 
that form is a lot longer, but maybe usefully more explicit.  Well, it 
is a *lot* longer.  But if string.Template's functionality becomes a 
method on str (is that the plan?) then this won't be so bad.  Also if 
string.Template returns an object of the same class as is passed in. 
Then maybe it'd just be p.safe_substitute(os.environ), which isn't bad 
at all.

Maybe if this used Windows conventions on that platform -- of %VAR% -- 
it would seem more useful.  Though I think $VAR should still work on 
both platforms regardless (just like / does).

>         def dirname(self): ...
>         def basename(self): ...

These are duplicated as properties.  basename and namebase are confusing 
alternatives to each other.

>         def expand(self): ...

I see this is a combination of normpath, expanduser, and expandvars. 
Useful, certainly.  But there's also a lot of forms, and no one applies 
these operations consistently it seems.

>         def splitpath(self): ...
>         def splitdrive(self): ...
>         def splitext(self): ...
>         def stripext(self): ...

This is another new method, equivalent to .splitext()[0].  I'm not sure 
it's that important.

>         def splitunc(self): ... [1]
>         def joinpath(self, *args): ...
>         def splitall(self): ...

And there's just so many splitting functions.  Could these somehow be 
combined?  Maybe returning a tuple/struct?  Or maybe just relying on 
properties.

>         def relpath(self): ...
>         def relpathto(self, dest): ...

These don't feel compellingly different according to the name.  I find 
the cwd fragile too, so maybe the first form offends me from that 
perspective too.  Just the explicit form feels sufficient to me, and 
unambiguous as both a reader and writer of code.

>         # Properties about the path.
>         parent, name, namebase, ext, drive, uncshare[1]

Actually, I see namebase is actually the name without an extension.  It 
seems ambiguous to me just from the names, and I'd rather both weren't 
there.  Though ext somehow seems useful and unambiguous in a way 
namebase isn't.  Not sure why.

It's unclear which of these should be Paths.  Of course parent should. 
None of the others?  When methods return paths and when they return 
strings is an important part of the spec.

>         # Operations that return lists of paths.
>         def listdir(self, pattern = None): ...
>         def dirs(self, pattern = None): ...
>         def files(self, pattern = None): ...
>         def walk(self, pattern = None): ...
>         def walkdirs(self, pattern = None): ...
>         def walkfiles(self, pattern = None): ...

Notably these aren't like os.path.walk, I assume.  Which is fine by me.

>         def match(self, pattern):
>         def matchcase(self, pattern):

I don't see these methods in the path class, and I'm not sure what 
they'd do.

>         def glob(self, pattern):
> 
>         # Methods for retrieving information about the filesystem
>         # path.
>         def exists(self): ...
>         def isabs(self): ...
>         def isdir(self): ...
>         def isfile(self): ...
>         def islink(self): ...
>         def ismount(self): ...
>         def samefile(self, other): ... [1]
>         def getatime(self): ...
>         def getmtime(self): ...
>         def getctime(self): ...
>         def getsize(self): ...
>         def access(self, mode): ... [1]
>         def stat(self): ...
>         def lstat(self): ...
>         def statvfs(self): ... [1]

The stat and get* functions overlap too.  I.e., p.getmtime() and 
p.stat().st_mtime are the same.  Too bad about the st_* names on stat 
objects, otherwise I don't see any problem with using that directly.  It 
still seems clearer.

>         def pathconf(self, name): ... [1]

I can't figure out what this does, even from the docs.  Some of these 
seem obscure enough they could be left in os.

>         def utime(self, times): ...
>         def chmod(self, mode): ...
>         def chown(self, uid, gid): ... [1]
>         def rename(self, new): ...
>         def renames(self, new):
> 
>         # Filesystem properties for path.
>         atime, getmtime, getctime, size

Mmm... then these show up yet again.

>         # Methods for manipulating information about the filesystem
>         # path.
>         def utime(self, times): ...
>         def chmod(self, mode): ...
>         def chown(self, uid, gid): ... [1]
>         def rename(self, new): ...
>         def renames(self, new): ...

Dups in the spec.

>         # Create/delete operations on directories
>         def mkdir(self, mode = 0777): ...
>         def makedirs(self, mode = 0777): ...
>         def rmdir(self): ...
>         def removedirs(self): ...
> 
>         # Modifying operations on files
>         def touch(self): ...
>         def remove(self): ...
>         def unlink(self): ...
> 
>         # Modifying operations on links
>         def link(self, newpath): ...
>         def symlink(self, newlink): ...
>         def readlink(self): ...
>         def readlinkabs(self): ...
 >
>         # High-level functions from shutil
>         def copyfile(self, dst): ...
>         def copymode(self, dst): ...
>         def copystat(self, dst): ...
>         def copy(self, dst): ...
>         def copy2(self, dst): ...
>         def copytree(self, dst, symlinks = True): ...
>         def move(self, dst): ...
>         def rmtree(self, ignore_errors = False, onerror = None): ...
> 
>         # Special stuff from os
>         def chroot(self): ... [1]
>         def startfile(self): ... [1]

Like pathconf, maybe these don't need to be moved into the module, and 
can be left in os.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From tim.peters at gmail.com  Wed Jan 25 00:53:36 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 24 Jan 2006 18:53:36 -0500
Subject: [Python-Dev] Long-time shy failure in test_socket_ssl
In-Reply-To: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com>
References: <1f7befae0601241422sd783c37u5d676893de2f6055@mail.gmail.com>
Message-ID: <1f7befae0601241553v10f7dd41i364e92b98f9de4a2@mail.gmail.com>

[Tim Peters]
> ...
> test_rude_shutdown() is dicey, relying on a sleep() instead of proper
> synchronization to make it probable that the `listener` thread goes
> away before the main thread tries to connect, but while that race may
> account for bogus TestFailed deaths, it doesn't seem possible that it
> could account for the kind of failure above.

Well, since it's silly to try to guess about one weird failure when a
clear cause for another kind of weird failure is known, I checked in
changes to do "proper" thread synchronization and termination in that
test.  Hasn't failed here since, but that's not surprising (it was
always a "once in a light blue moon" kind of thing).

I'm not sure how/whether this test is supposed to work with Jython --
perhaps the `thread.exit()` I removed could be important there.  The
test relies on that a socket gets closed when a socket object becomes
trash & is reclaimed; in CPython that's easy to control; I don't know
why the test didn't/doesn't simply do an explicit s.close() instead.

From guido at python.org  Wed Jan 25 01:01:29 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 24 Jan 2006 16:01:29 -0800
Subject: [Python-Dev] Know anyone interested in a Google internship?
Message-ID: <ca471dc20601241601w5bb08fd7i9870408fa09f5ca3@mail.gmail.com>

Sorry for the plug.

Google is looking to fill an unprecedented number of student intern
positions this summer, at several US locations (Mountain View, Santa
Monica, Kirkland (Wash.), and New York). If you're interested or know
someone interested, please see
http://www.google.com/jobs/intern.html. Candidates from outside the US
are welcome. The perks are incredible (just ask Brett Cannon, who's
coming back for the second year this summer :-).

We're not just looking for software development interns -- there are
also product management positions, and UI design and usability analyst
positions.

I can't guarantee that I'll be mentoring you, but if you end up using
Python in Mountain View, I'm sure we'll be interacting quite a bit!

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From thomas at xs4all.net  Wed Jan 25 01:59:18 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Wed, 25 Jan 2006 01:59:18 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <43D6AFC4.3030200@v.loewis.de>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
Message-ID: <20060125005918.GC18916@xs4all.nl>

On Tue, Jan 24, 2006 at 11:52:52PM +0100, "Martin v. L?wis" wrote:

> > It looks like a timing issue; the first run succeeds, all subsequent runs
> > fail, for a while, anyway. I'll do some googling and browsing other
> > tty/pty-using code to see if there's anything we're not doing we should be
> > doing, but it looks like a platform bug that we can't fix... Not without
> > re-implementing os.isatty anyway ;P

> Couldn't there be a bug in openpty instead? Perhaps it is trying to
> allocate the same device again, but fails to do so correctly, and fails
> to recognize that it should use a different one instead.

Well, a bug in openpty is what I started with. Python's posix.openpty() that
is, as Solaris doesn't have openpty. Openpty is emulated using code taken
almost verbatim from Solaris' pts(7D) manpage:

     fdm = open("/dev/ptmx", O_RDWR);  /* open master */
     grantpt(fdm);                     /* change permission of   slave */
     unlockpt(fdm);                    /* unlock slave */
     slavename = ptsname(fdm);         /* get name of slave */
     fds = open(slavename, O_RDWR);    /* open slave */
     ioctl(fds, I_PUSH, "ptem");       /* push ptem */
     ioctl(fds, I_PUSH, "ldterm");     /* push ldterm*/

(That's the manpage code.) This is also what openssh does (as far as I can
tell). Screen does it slightly differently; it does the ptsname() call
before the grantpt/unlockpt calls, but the open(slavename) after. If I make
posixmodule do that, it still fails on Solaris. Mucking with it more just
breaks it more.

The thing is, disabling the check that fails, whether the slave-tty returned
by os.openpty() is a tty, shows that the later test for the same thing
succeeds. The later test is done in a child created by pty.fork(). Disabling
the ptmx code on Solaris is probably the most reliable way to fix the
failing test; the pty module will fall back to old-style BSD pty code and
that works fine. But it's a bit of a shame not to use /dev/ptmx just because
the slave fd, when used directly (rather than in a child process) sometimes
doesn't seem to be a tty. They're still connected, it still seems to work
fine.

> Anyway, if you still think you need an OS 10 account, please let me
> know, and I can give you an account to the machine the buildbot
> runs on.

I think I've seen enough confusing situations for a while... I'm sure the
bug is the same on Solaris 10 ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From mike at skew.org  Wed Jan 25 04:25:10 2006
From: mike at skew.org (Mike Brown)
Date: Tue, 24 Jan 2006 20:25:10 -0700 (MST)
Subject: [Python-Dev] New Pythondoc by effbot
In-Reply-To: <740c3aec0601230132n2ff31a22n1cd49f9517330cb1@mail.gmail.com>
Message-ID: <200601250325.k0P3PAWO020866@chilled.skew.org>

BJ> Why does it have to be "wiki-like"? Why can't it be a wiki? MediaWiki
> seem to work pretty well for a lot of software projects that have put
> their documentation in a wiki. Talk pages for commentary and primary
> pages for reviewed content.

And inconsistent formatting from article to article, limitations in indexing
options, no way to browse a set of documentation specific to a particular
Python release...

I personally like the PHP docs - http://www.php.net/manual/en/

They're not versioned, and users can't modify the indexes or API docs, but
they do get to add annotations. Annotations that reflect errors or major 
omissions from the docs can be reviewed by editors and folded into the docs
as needed. *shrug*

From jeremy at alum.mit.edu  Wed Jan 25 04:46:17 2006
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue, 24 Jan 2006 22:46:17 -0500
Subject: [Python-Dev] Know anyone interested in a Google internship?
In-Reply-To: <ca471dc20601241601w5bb08fd7i9870408fa09f5ca3@mail.gmail.com>
References: <ca471dc20601241601w5bb08fd7i9870408fa09f5ca3@mail.gmail.com>
Message-ID: <e8bf7a530601241946y30b349cdub1e465d5c8ed9da6@mail.gmail.com>

On 1/24/06, Guido van Rossum <guido at python.org> wrote:
> Sorry for the plug.
>
> Google is looking to fill an unprecedented number of student intern
> positions this summer, at several US locations (Mountain View, Santa
> Monica, Kirkland (Wash.), and New York). If you're interested or know
> someone interested, please see
> http://www.google.com/jobs/intern.html. Candidates from outside the US
> are welcome. The perks are incredible (just ask Brett Cannon, who's
> coming back for the second year this summer :-).
>
> We're not just looking for software development interns -- there are
> also product management positions, and UI design and usability analyst
> positions.
>
> I can't guarantee that I'll be mentoring you, but if you end up using
> Python in Mountain View, I'm sure we'll be interacting quite a bit!

And I'd be happy to work with folks in New York!

Jeremy

From mwh at python.net  Wed Jan 25 10:34:45 2006
From: mwh at python.net (Michael Hudson)
Date: Wed, 25 Jan 2006 09:34:45 +0000
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <20060124202238.GM17240@zot.electricrain.com> (Gregory P.
	Smith's message of "Tue, 24 Jan 2006 12:22:38 -0800")
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124202238.GM17240@zot.electricrain.com>
Message-ID: <2mlkx4lkga.fsf@starship.python.net>

"Gregory P. Smith" <greg at electricrain.com> writes:

> Using BerkeleyDB 3.2 often segfaults for me; using 3.3 often hangs in
> the test suite.  Both are so old I don't see much motivation to track
> the issues down.

My goal is to not have http://www.python.org/dev/buildbot/ go red
randomly because of erratic bsddb tests, so I'd definitely prefer it
if we didn't build bsddb3 when a not-supported bsddb is found.

Things seem much better today after your recent changes though, so
thanks for that :)

Cheers,
mwh

-- 
  at any rate, I'm satisfied that not only do they know which end of
  the pointy thing to hold, but where to poke it for maximum effect.
                                  -- Eric The Read, asr, on google.com

From hoffman at ebi.ac.uk  Wed Jan 25 12:36:15 2006
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Wed, 25 Jan 2006 11:36:15 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D69E2A.6050205@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
	<43D69E2A.6050205@colorstudy.com>
Message-ID: <Pine.LNX.4.64.0601251134530.27932@qnzvnan.rov.np.hx>

[Ian Bickling]

> I'm personally +1 on /.  I think people who think it is confusing are
> giving a knee-jerk reaction.  It's very easy to tell the difference
> between file-related code and math-related code, and / is currently only
> used for math.  In contrast, + is used for concatenation and addition,
> and these are far more ambiguous from context -- but still it doesn't
> cause that many problems.

I was initially -0 on / but I have found it quite useful and a lot
simpler than a lot of joinpath()s over time.

So +1 on / for me.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From hoffman at ebi.ac.uk  Wed Jan 25 12:39:46 2006
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Wed, 25 Jan 2006 11:39:46 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <EA38A386-44BE-4DA6-BE7B-0DA189C1C9E9@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<EA38A386-44BE-4DA6-BE7B-0DA189C1C9E9@ihug.co.nz>
Message-ID: <Pine.LNX.4.64.0601251138100.27932@qnzvnan.rov.np.hx>

[Tony Meyer]

> It would be great (and save a lot of rehashing) if you could go over
> the python-dev discussion and add the relevant parts (for example,
> whether to include the __div__ hack) to the PEP:
>
> <http://mail.python.org/pipermail/python-dev/2005-June/054439.html>

In particular the points about Path being able to be a drop-in
replacement for str/unicode are useful ones, and explain the use of
joinpath() etc. It is really useful that I can use a Path anywhere I
might have used an str and not have to worry about the conversions.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From skip at pobox.com  Wed Jan 25 13:24:55 2006
From: skip at pobox.com (skip at pobox.com)
Date: Wed, 25 Jan 2006 06:24:55 -0600
Subject: [Python-Dev] Know anyone interested in a Google internship?
In-Reply-To: <ca471dc20601241601w5bb08fd7i9870408fa09f5ca3@mail.gmail.com>
References: <ca471dc20601241601w5bb08fd7i9870408fa09f5ca3@mail.gmail.com>
Message-ID: <17367.28183.39730.537523@montanaro.dyndns.org>


    Guido> Google is looking to fill an unprecedented number of student
    Guido> intern positions this summer, at several US locations ....

Maybe a Python Job Board post would be in order...

Skip

From tdickenson at devmail.geminidataloggers.co.uk  Wed Jan 25 13:30:06 2006
From: tdickenson at devmail.geminidataloggers.co.uk (Toby Dickenson)
Date: Wed, 25 Jan 2006 12:30:06 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <200601251230.06230.tdickenson@devmail.geminidataloggers.co.uk>

On Tuesday 24 January 2006 20:22, BJ?rn Lindqvist wrote:

>     Replacing glob.glob
>     -------------------
> 
>     glob.glob("/lib/*.so")
>     ==>
>     Path("/lib").glob("*.so")

This definition seems confusing because it splits the glob pattern string in 
two ('/lib', and '*.so'). Unless there is an intention to change the behavior 
of the current glob module, why not make the glob method parameterless:

    glob.glob("/lib/*.so")
    ==>
    Path("/lib/*.so").glob()


Possible confusion with the one parameter version:

Does glob matching happen on the first half too? That is, does 
Path('*').glob('*.so') match files in any directory, or only directories 
whose name is an asterisk.

What behavior can I expect from Path('/foo/').glob(bar), where bar is some 
arbitrary string? It could be reasonable to expect that it would only match 
filenames inside the foo directory. However it could also be reasonable to 
expect to use bar=='/etc/*'


-- 
Toby Dickenson

From jason.orendorff at gmail.com  Wed Jan 25 17:12:00 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 25 Jan 2006 11:12:00 -0500
Subject: [Python-Dev] The path module PEP
In-Reply-To: <200601251230.06230.tdickenson@devmail.geminidataloggers.co.uk>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<200601251230.06230.tdickenson@devmail.geminidataloggers.co.uk>
Message-ID: <bb8868b90601250812o437dddc6t32fb28cf1509130b@mail.gmail.com>

On 1/25/06, Toby Dickenson <tdickenson at devmail.geminidataloggers.co.uk> wrote:
> On Tuesday 24 January 2006 20:22, BJ?rn Lindqvist wrote:
> >     #Replacing glob.glob
> >     glob.glob("/lib/*.so")
> >     ==>
> >     Path("/lib").glob("*.so")
>
> This definition seems confusing because it splits the glob pattern string in
> two ('/lib', and '*.so'). [...]

Well, let's make this look more like real code:

    #line 1
    LIB_DIR = "/lib"
    ==>
    LIB_DIR = Path("/lib")

    #line 296
    libs = glob.glob(os.path.join(LIB_DIR, "*.so"))
    ==>
    libs = LIB_DIR.files("*.so")

Clearer?  In d.files(pattern), d is simply the root directory for the
search.  The same is true of all the searching methods: dirs(),
walkfiles(), walkdirs(), etc.

I actually never use path.glob().  For example, here files() is
actually more accurate, and the word "files" is surely clearer than
"glob".  Given files(), dirs(), and listdir(), I have never found a
real use case for glob().

-j

From jason.orendorff at gmail.com  Wed Jan 25 17:39:03 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Wed, 25 Jan 2006 11:39:03 -0500
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D6BA85.8070007@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
Message-ID: <bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>

On 1/24/06, Ian Bicking <ianb at colorstudy.com> wrote:
> There's kind of a lot of methods in here, which is a little bothersome.
> It also points towards the motivation for the class -- too many
> options in too many places in the stdlib.  But throwing them *all* in
> one class consolidates but doesn't simplify, especially with duplicate
> functionality.

I agree.

Let me explain why there's so much cruft in path.py.  The design is
heavily skewed toward people already very familiar with the existing
stdlib equivalents, because that is the market for third-party
modules.  I think my users want to type p.methodname() for whatever
method name they already know, and have it *just work*.   A sloppy API
you've already learned is easier to pick up than a clean API you've
never seen before.  Ergo, cruft.

A stdlib Path should have different design goals.  It should have less
redundancy, fewer methods overall, and PEP-8-compliant names.

-j

From bjourne at gmail.com  Wed Jan 25 21:37:04 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Wed, 25 Jan 2006 21:37:04 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
Message-ID: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>

My comments on the issues. It was easier this way than trying to reply
on every message individually.

Inheritance from string (Jason)

This issue has been brought up before when people were discussing the
path module. I think the consensus is that, while the inheritance
isn't pure, practicality beats purity. It would require to big changes
to Python and would break to much existing code to not extend string.

I'll add this to Resolved Issues if nobody minds.

* http://mail.python.org/pipermail/python-dev/2001-August/016663.html
* http://mail.python.org/pipermail/python-dev/2001-August/016665.html
* http://mail.python.org/pipermail/python-list/2005-July/291091.html
* http://mail.python.org/pipermail/python-list/2005-July/291152.html


Remove __div__ (Ian, Jason, Michael, Oleg)

This is one of those where everyone (me too) says "I don't care either
way." If that is so, then I see no reason to change it unless someone
can show a scenario in which it hurts readability. Plus, a few people
have said that they like the shortcut.

* http://mail.python.org/pipermail/python-list/2005-July/292251.html
* http://mail.python.org/pipermail/python-dev/2005-June/054496.html
* http://mail.python.org/pipermail/python-list/2005-July/291628.html
* http://mail.python.org/pipermail/python-list/2005-July/291621.html


Remove redundant methods (Eric, Ian, Jason, Ronald, Toby)

I think it is a good idea to take out some of Path's
methods. Especially the ones that exist as both methods and
properties. I have updated the pep and dumped basename(), dirname(),
splitdrive() and splitext().  I think that glob() should also go away
because I can't of the top of my head think of a scenario where it
would be suitable over listdir(), dirs() or files(). Plus, for
contrived examples; like glob.glob("/*bin/*foo*") the Path way doesn't
look so good: Path("/").glob("*bin/*foo*").


Renaming methods because of PEP 8 (Gustavo, Ian, Jason)

I'm personally not keen on that. I like most of the names as they
are. abspath(), joinpath(), realpath() and splitall() looks so much
better than abs_path(), join_path(), real_path() and split_all() in my
eyes. If someone like the underscores I'll add it to Open Issues.


Removing open() and methods that manipulate the whole file at once
(Ian, Jason)

I think this is settled with the addition of the with statement? My
idea when scrubbing these methods was that it would make it easier to
get the PEP accepted. However, even with with, these methods save some
typing.

* http://mail.python.org/pipermail/python-dev/2005-June/054439.html
* http://mail.python.org/pipermail/python-list/2005-July/291435.html


?time properties and get?time() methods

Clearly, Path has either the properties or the methods, not both at
once. Yet another "I don't care either way."

* http://mail.python.org/pipermail/python-dev/2005-June/054439.html
* http://mail.python.org/pipermail/python-list/2005-July/291424.html
* http://mail.python.org/pipermail/python-list/2005-July/291460.html


I have also the corrected the copy-paste errors I inadvertedly
introduced. Path should *not* have an __iter__. :)

* match() and matchcase() wraps the fnmatch.fnmatch() and
  fnmatch.fnmatchcase() functions. I believe that the renaming is
  uncontroversial and that the introduction of matchcase() makes it so
  the whole fnmatch module can be deprecated.

I have created an implementation of Path that corresponds to the
specification in the PEP (which I hope will appear on
www.python.org/peps soon). It is based on Reinhold's (Georg Brandl)
implementation from pre-PEP threads in c.l.p last summer. But I have
no place to upload it. I would also like if some people wants to
co-author this PEP with me - it's really neither my PEP nor my module.

--
mvh Bj?rn

From phd at mail2.phd.pp.ru  Wed Jan 25 21:59:26 2006
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Wed, 25 Jan 2006 23:59:26 +0300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <20060125205926.GA25426@phd.pp.ru>

On Wed, Jan 25, 2006 at 09:37:04PM +0100, BJ?rn Lindqvist wrote:
> Remove __div__ (Ian, Jason, Michael, Oleg)

   I didn't say "remove". Exactly opposite - I am enamoured by the beauty
of the syntax! (-:

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From ianb at colorstudy.com  Wed Jan 25 22:03:44 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 15:03:44 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <43D7E7B0.8070209@colorstudy.com>

BJ?rn Lindqvist wrote:
> * match() and matchcase() wraps the fnmatch.fnmatch() and
>   fnmatch.fnmatchcase() functions. I believe that the renaming is
>   uncontroversial and that the introduction of matchcase() makes it so
>   the whole fnmatch module can be deprecated.

The renaming is fine with me.  I generally use the fnmatch module for 
wildcard matching, not necessarily against path names.  Path.match 
doesn't replace that functionality.  Though fnmatch.translate isn't even 
publically documented, which is the function I actually tend to use.

Though it seems a little confusing to me that glob treats separators 
specially, and that's not implemented at the fnmatch level.  So 
Path('/a/b/d/c').match('a/*/d') is true, but Path('/').walk('a/*/d') 
won't return Path('/a/b/c/d').  I think .match() should be fixed.  But I 
don't think fnmatch should be changed.

I'm actually finding myself a little confused by the glob arguments (if 
the glob contains '/'), now that I really think about them.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From ianb at colorstudy.com  Wed Jan 25 22:05:33 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 15:05:33 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <Pine.LNX.4.58.0601252055020.6232@alice>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
	<43D69E2A.6050205@colorstudy.com>
	<Pine.LNX.4.58.0601252055020.6232@alice>
Message-ID: <43D7E81D.8090007@colorstudy.com>

John J Lee wrote:
> On Tue, 24 Jan 2006, Ian Bicking wrote:
> [...]
> 
>>Losing .open() would make it much harder for anyone wanting to write, 
>>say, a URI library that implements the Path API.
> 
> [...]
> 
> Why?  Could you expand a bit?
> 
> What's wrong with urlopen(filesystem_path_instance) ?

My example shows this more clearly I think:

   def read_config(path):
       text = path.open().read()
       ... do something ...

If I implement a URI object with an .open() method, then I can use it 
with this function, even though read_config() was written with file 
paths in mind.  But without it that won't work:

   def read_config(path):
       text = open(path).read()
       ...

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From jjl at pobox.com  Wed Jan 25 22:30:02 2006
From: jjl at pobox.com (John J Lee)
Date: Wed, 25 Jan 2006 21:30:02 +0000 (UTC)
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D7E81D.8090007@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
	<43D69E2A.6050205@colorstudy.com>
	<Pine.LNX.4.58.0601252055020.6232@alice>
	<43D7E81D.8090007@colorstudy.com>
Message-ID: <Pine.LNX.4.58.0601252118240.6232@alice>

[Ian Bicking]
>Losing .open() would make it much harder for anyone wanting to write, 
>say, a URI library that implements the Path API.

[John]
> Why?  Could you expand a bit?
> 
> What's wrong with urlopen(filesystem_path_instance) ?

[Ian]
>    def read_config(path):
>        text = path.open().read()
>        ... do something ...

I should have expected that answer, but couldn't believe that you think
it's a good idea to implement that obese filesystem path API for URLs ;-)

Shouldn't we instead have:

 a) .open()-able objects blessed in the stdlib & stdlib docs, as a
separate interface from the path interface (I guess that would be an
argument in favour of path implementing that one-method interface, as long
as it's not tied too tightly to the fat path interface)

 b) a path object with a thinner interface (I know you've already
expressed that preference yourself...)?


John

From jjl at pobox.com  Wed Jan 25 22:01:21 2006
From: jjl at pobox.com (John J Lee)
Date: Wed, 25 Jan 2006 21:01:21 +0000 (UTC)
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D69E2A.6050205@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<bb8868b90601241318i3f21f484gebe1f5241a9a8526@mail.gmail.com>
	<43D69E2A.6050205@colorstudy.com>
Message-ID: <Pine.LNX.4.58.0601252055020.6232@alice>

On Tue, 24 Jan 2006, Ian Bicking wrote:
[...]
> Losing .open() would make it much harder for anyone wanting to write, 
> say, a URI library that implements the Path API.
[...]

Why?  Could you expand a bit?

What's wrong with urlopen(filesystem_path_instance) ?


John

From t-meyer at ihug.co.nz  Wed Jan 25 23:25:57 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 26 Jan 2006 11:25:57 +1300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>

> Remove __div__ (Ian, Jason, Michael, Oleg)
>
> This is one of those where everyone (me too) says "I don't care either
> way." If that is so, then I see no reason to change it unless someone
> can show a scenario in which it hurts readability. Plus, a few people
> have said that they like the shortcut.
>
> * http://mail.python.org/pipermail/python-list/2005-July/292251.html
> * http://mail.python.org/pipermail/python-dev/2005-June/054496.html
> * http://mail.python.org/pipermail/python-list/2005-July/291628.html
> * http://mail.python.org/pipermail/python-list/2005-July/291621.html

Well, if you include the much larger discussion on python-list,  
people (including me) have said that removing __div__ is a good  
idea.  If it's included in the PEP, please at least include a  
justification and cover the problems with it.  The vast majority of  
people (at least at the time) were either +0 or -0, not +1.  +0's are  
not justification for including something.

Against it:

  * Zen: Beautiful is better than ugly. Explicit is better than  
implicit. Readability counts. There should be one-- and preferably  
only one --obvious way to do it.

  * Not every platform that Python supports has '/' as the path  
separator.  Windows, a pretty major one, has '\'.  I have no idea  
what various portable devices use, but there's a reasonable chance  
it's not '/'.

  * It's being used to mean "join", which is the exact opposite  
of /'s other meaning ("divide").

  * Python's not Perl.  We like using functions and not symbols.

> Renaming methods because of PEP 8 (Gustavo, Ian, Jason)
>
> I'm personally not keen on that. I like most of the names as they
> are. abspath(), joinpath(), realpath() and splitall() looks so much
> better than abs_path(), join_path(), real_path() and split_all() in my
> eyes. If someone like the underscores I'll add it to Open Issues.

+1 to following PEP 8.  These aren't built-ins, it's a library  
module.  In addition to the PEP, underscores make it much easier to  
read, especially for those for whom English is not their first language.

=Tony.Meyer

From pje at telecommunity.com  Wed Jan 25 23:54:04 2006
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed, 25 Jan 2006 17:54:04 -0500
Subject: [Python-Dev] The path module PEP
In-Reply-To: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
References: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com>

At 11:25 AM 1/26/2006 +1300, Tony Meyer wrote:
>Against it:
>
>   * Zen: Beautiful is better than ugly. Explicit is better than
>implicit. Readability counts. There should be one-- and preferably
>only one --obvious way to do it.
>
>   * Not every platform that Python supports has '/' as the path
>separator.  Windows, a pretty major one, has '\'.

"/" also works on Windows, and the Python distutils already set the 
precedent of requiring /-separated paths on *all* platforms, converting 
them to os.sep behind the scenes.

I'd also note that using the / operator seems to me to be a big win on 
"beautiful is better than ugly".  Path-joining code is mighty ugly without 
it, and / is more readable as well.

It'd be nice to see the urllib modules grow a URL type supporting this 
operator, among other path operators.

I would also suggest that as with the individual posixpath, ntpath, etc. 
libraries today, we should be able to import NTPath and PosixPath classes 
directly from those modules, for code that needs to manipulate a path for 
some system other than the one it's running on.


From ianb at colorstudy.com  Thu Jan 26 00:13:37 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 17:13:37 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
Message-ID: <43D80621.1080300@colorstudy.com>

Tony Meyer wrote:
>>Remove __div__ (Ian, Jason, Michael, Oleg)
>>
>>This is one of those where everyone (me too) says "I don't care either
>>way." If that is so, then I see no reason to change it unless someone
>>can show a scenario in which it hurts readability. Plus, a few people
>>have said that they like the shortcut.
>>
>>* http://mail.python.org/pipermail/python-list/2005-July/292251.html
>>* http://mail.python.org/pipermail/python-dev/2005-June/054496.html
>>* http://mail.python.org/pipermail/python-list/2005-July/291628.html
>>* http://mail.python.org/pipermail/python-list/2005-July/291621.html
> 
> 
> Well, if you include the much larger discussion on python-list,  
> people (including me) have said that removing __div__ is a good  
> idea.  If it's included in the PEP, please at least include a  
> justification and cover the problems with it.  The vast majority of  
> people (at least at the time) were either +0 or -0, not +1.  +0's are  
> not justification for including something.

If it were possible to use .join() for joining paths, I think I wouldn't 
mind so much.  But reusing a string method for something very different 
seems like a bad idea.  So we're left with .joinpath().  Still better 
than os.path.join() I guess, but only a little.  I guess that's why I'm 
+1 on /.

> Against it:
> 
>   * Zen: Beautiful is better than ugly. Explicit is better than  
> implicit. Readability counts. There should be one-- and preferably  
> only one --obvious way to do it.

I think / is pretty.  I think it reads well.  There's already some 
inevitable redundancy in this interface.  I use os.path.join so much 
that I know anything I use will feel readable quickly, but I also think 
I'll find / more appealing.

>   * Not every platform that Python supports has '/' as the path  
> separator.  Windows, a pretty major one, has '\'.  I have no idea  
> what various portable devices use, but there's a reasonable chance  
> it's not '/'.

I believe all platforms support /; at least Windows and Mac do, in 
addition to their native separators.  I assume any platform that 
supports filesystem access will support / in Python.

If anything, a good shortcut for .joinpath() will at least encourage 
people to use it, thus discouraging hardcoding of path separators.  I 
expect it would encourage portable paths.

Though Path('/foo') / '/bar' == Path('/bar'), which is *not* intuitive, 
though in the context of "join" it's not as surprising.  So that is a 
problem.  If / meant "under this path" then that could be a useful 
operator (in that I'd really like such an operator or method).  Either 
paths would be forced to be under the original path, or it would be an 
error if they somehow escaped.  Currently there's no quick-and-easy way 
to ensure this, except to join the paths, do abspath(), then confirm 
that the new path starts with the old path.

>   * It's being used to mean "join", which is the exact opposite  
> of /'s other meaning ("divide").
> 
>   * Python's not Perl.  We like using functions and not symbols.

A little too heavy on the truisms.  Python isn't the anti-Perl.

>>Renaming methods because of PEP 8 (Gustavo, Ian, Jason)
>>
>>I'm personally not keen on that. I like most of the names as they
>>are. abspath(), joinpath(), realpath() and splitall() looks so much
>>better than abs_path(), join_path(), real_path() and split_all() in my
>>eyes. If someone like the underscores I'll add it to Open Issues.
> 
> 
> +1 to following PEP 8.  These aren't built-ins, it's a library  
> module.  In addition to the PEP, underscores make it much easier to  
> read, especially for those for whom English is not their first language.

I don't find abs_path() much easier to read than abspath() -- neither is 
a full name.  absolute_path() perhaps, but that is somewhat redundant; 
absolute()...?  Eh.

Precedence in naming means something, and in this case all the names 
have existed for a very long time (as long as Python?)  PEP 8 encourages 
following naming precedence.  While I don't see a need to match every 
existing function with a method, to the degree they do match I see no 
reason why we shouldn't keep the names.  And I see reasons why the names 
shouldn't be changed.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From python at discworld.dyndns.org  Thu Jan 26 00:34:35 2006
From: python at discworld.dyndns.org (Charles Cazabon)
Date: Wed, 25 Jan 2006 17:34:35 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com>
References: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com>
Message-ID: <20060125233435.GD742@discworld.dyndns.org>

Phillip J. Eby <pje at telecommunity.com> wrote:
> 
> I'd also note that using the / operator seems to me to be a big win on 
> "beautiful is better than ugly".

It screams "magic" in a very un-Pythonic (and possibly very Perl-like) way.
I'm not aware of any other part of the standard library grossly abusing
standard operators in this way.  As others have noted, "/" is being used here
to mean precisely the opposite of what it means in every other use in Python,
which alone should be justification for getting rid of it.

Charles
-- 
-----------------------------------------------------------------------
Charles Cazabon                           <python at discworld.dyndns.org>
GPL'ed software available at:               http://pyropus.ca/software/
-----------------------------------------------------------------------

From jjl at pobox.com  Thu Jan 26 00:48:12 2006
From: jjl at pobox.com (John J Lee)
Date: Wed, 25 Jan 2006 23:48:12 +0000 (UTC)
Subject: [Python-Dev] / as path join operator (was: Re: The path module PEP)
In-Reply-To: <63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
Message-ID: <Pine.LNX.4.58.0601252331300.6232@alice>

On Thu, 26 Jan 2006, Tony Meyer wrote:
[...]
> Well, if you include the much larger discussion on python-list,  
> people (including me) have said that removing __div__ is a good  
> idea.  If it's included in the PEP, please at least include a  
> justification and cover the problems with it.  The vast majority of  
> people (at least at the time) were either +0 or -0, not +1.  +0's are  
> not justification for including something.

<bikeshed>

FWLIW, I'm definitely +1 on using / as a path join operator.


>   * It's being used to mean "join", which is the exact opposite  
> of /'s other meaning ("divide").

But it's a very readable way to write a common operation.  Perhaps one
reason the discrepancy you point out doesn't bother me is that division is
the least-used of the +-*/ arithmetic operations.

Also, &, | and ^ seem like some sort of precedent, to my brain (if they
don't to yours, that's fine and I believe you ;-).


>   * Python's not Perl.  We like using functions and not symbols.

I think this is a tasteful, if not parsimonious, use of a symbol.

</bikeshed>


John

From ianb at colorstudy.com  Thu Jan 26 00:49:51 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 17:49:51 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <43D80E9F.4070009@colorstudy.com>

BJ?rn Lindqvist wrote:
> Remove __div__ (Ian, Jason, Michael, Oleg)
> 
> This is one of those where everyone (me too) says "I don't care either
> way." If that is so, then I see no reason to change it unless someone
> can show a scenario in which it hurts readability. Plus, a few people
> have said that they like the shortcut.
> 
> * http://mail.python.org/pipermail/python-list/2005-July/292251.html
> * http://mail.python.org/pipermail/python-dev/2005-June/054496.html
> * http://mail.python.org/pipermail/python-list/2005-July/291628.html
> * http://mail.python.org/pipermail/python-list/2005-July/291621.html

Curious how often I use os.path.join and division, I searched a project 
of mine, and in 12k lines there were 34 uses of join, and 1 use of 
division.  In smaller scripts os.path.join tends to show up a lot more 
(per line).  I'm sure there's people who use division far more than I, 
and os.path.join less, but I'm guessing the majority of users are more 
like me.

That's not necessarily a justification of / for paths, but at least this 
use for "/" wouldn't be obscure or mysterious after you get a little 
experience seeing code that uses it.


-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From t-meyer at ihug.co.nz  Thu Jan 26 00:42:13 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 26 Jan 2006 12:42:13 +1300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D80621.1080300@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
Message-ID: <C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>

[Ian Bicking]
> If it were possible to use .join() for joining paths, I think I  
> wouldn't mind so much.  But reusing a string method for something  
> very different seems like a bad idea.  So we're left with .joinpath 
> ().  Still better than os.path.join() I guess, but only a little.   
> I guess that's why I'm +1 on /.

Why does reusing a string method for something very different seem  
like a bad idea, but reusing a mathematical operator for something  
very different seem like a good idea?  Path's aren't strings, so join 
() seems the logical choice.  (There are also alternatives to  
"joinpath" if the name is the thing: add(), for example).

[Tony Meyer]
>> Against it:
>>   * Zen: Beautiful is better than ugly. Explicit is better than   
>> implicit. Readability counts. There should be one-- and  
>> preferably  only one --obvious way to do it.
>
> I think / is pretty.  I think it reads well.

I suppose that the only beholder's eye that matters here is Guido's.   
(It still violates explicit/implicit and only-one-way.  Not rules, of  
course, but good guidelines).

>   There's already some inevitable redundancy in this interface.

That's hardly a reason to encourage *more*.  If anything, it's a  
reason to try for less, where possible.

>>   * Not every platform that Python supports has '/' as the path   
>> separator.  Windows, a pretty major one, has '\'.  I have no idea   
>> what various portable devices use, but there's a reasonable  
>> chance  it's not '/'.
>
> I believe all platforms support /; at least Windows and Mac do, in  
> addition to their native separators.

This is not strictly true.  Using '/' can lead to strange results  
with Windows, where it gets interpreted as a flag instead.  It's not  
reliable, it's not the path separator that Windows users/developers  
understand, and it's not the correct (i.e. according to Microsoft)  
path separator.  If by Mac you mean OS X, then that's just another  
*nix based OS.  I'm pretty sure that pre OS X (which isn't supported  
any more anyway, right?) '/' was not, in fact, supported, and that  
":" was required.  I also believe it's important to remember that  
Windows and *nix descendants are not "all platforms".

> If anything, a good shortcut for .joinpath() will at least  
> encourage people to use it, thus discouraging hardcoding of path  
> separators.  I expect it would encourage portable paths.

I'm not sure that I believe that the reason that people don't type  
"os.path.join('a', 'b')" is because they are too lazy to type it.   
However, I don't have any evidence either way, so it could be true.

[re: PEP8 following]
> Precedence in naming means something, and in this case all the  
> names have existed for a very long time (as long as Python?)  PEP 8  
> encourages following naming precedence.  While I don't see a need  
> to match every existing function with a method, to the degree they  
> do match I see no reason why we shouldn't keep the names.  And I  
> see reasons why the names shouldn't be changed.

PEP 8 encourages following naming precedence within a module, doesn't  
it?  Guido has said that he'd like to have the standard library  
tidied up, at least somewhat (e.g. StringIO.StringIO ->  
stringio.StringIO) for Python 3000.  It would make it less painful if  
new additions already followed the plan.

=Tony.Meyer

From t-meyer at ihug.co.nz  Thu Jan 26 00:51:04 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 26 Jan 2006 12:51:04 +1300
Subject: [Python-Dev] / as path join operator (was: Re: The path module
	PEP)
In-Reply-To: <Pine.LNX.4.58.0601252331300.6232@alice>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
Message-ID: <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz>

[John J Lee]
> But it's a very readable way to write a common operation.  Perhaps one
> reason the discrepancy you point out doesn't bother me is that  
> division is
> the least-used of the +-*/ arithmetic operations.

Do you have evidence to back that up?  It seems a strange claim.   
Outside of doing 'maths-y' work, I would think I'd use + most (but  
for strings), then / (for percentages).

> Also, &, | and ^ seem like some sort of precedent, to my brain (if  
> they
> don't to yours, that's fine and I believe you ;-).

I don't follow this, sorry.  You're referring to the bitwise operations?

[Ian Bicking]
> Curious how often I use os.path.join and division, I searched a  
> project
> of mine, and in 12k lines there were 34 uses of join, and 1 use of
> division.  In smaller scripts os.path.join tends to show up a lot more
> (per line).  I'm sure there's people who use division far more than I,
> and os.path.join less, but I'm guessing the majority of users are more
> like me.

The problem with these sorts of guesses is that there's no evidence.   
(Maybe the suggestion that Brett's PhD should collect a corpus of  
Python scripts was a good one <wink>).  Are mathematicians that under  
represented?  Is file processing that highly represented?  I have no  
idea.

=Tony.Meyer

From bob at redivi.com  Thu Jan 26 01:02:20 2006
From: bob at redivi.com (Bob Ippolito)
Date: Wed, 25 Jan 2006 16:02:20 -0800
Subject: [Python-Dev] The path module PEP
In-Reply-To: <C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
Message-ID: <8B530061-63C5-46F6-8081-F56ECF2EEC06@redivi.com>

On Jan 25, 2006, at 3:42 PM, Tony Meyer wrote:

> [Ian Bicking]
>> If it were possible to use .join() for joining paths, I think I
>> wouldn't mind so much.  But reusing a string method for something
>> very different seems like a bad idea.  So we're left with .joinpath
>> ().  Still better than os.path.join() I guess, but only a little.
>> I guess that's why I'm +1 on /.
>
> Why does reusing a string method for something very different seem
> like a bad idea, but reusing a mathematical operator for something
> very different seem like a good idea?  Path's aren't strings, so join
> () seems the logical choice.  (There are also alternatives to
> "joinpath" if the name is the thing: add(), for example).

join() is already defined for strings, division is not.  Different  
namespace... just like + is concatenation for list+list, tuple+tuple,  
basestring+basestring, but it's addition for numbers...

>>>   * Not every platform that Python supports has '/' as the path
>>> separator.  Windows, a pretty major one, has '\'.  I have no idea
>>> what various portable devices use, but there's a reasonable
>>> chance  it's not '/'.
>>
>> I believe all platforms support /; at least Windows and Mac do, in
>> addition to their native separators.
>
> This is not strictly true.  Using '/' can lead to strange results
> with Windows, where it gets interpreted as a flag instead.  It's not
> reliable, it's not the path separator that Windows users/developers
> understand, and it's not the correct (i.e. according to Microsoft)
> path separator.  If by Mac you mean OS X, then that's just another
> *nix based OS.  I'm pretty sure that pre OS X (which isn't supported
> any more anyway, right?) '/' was not, in fact, supported, and that
> ":" was required.  I also believe it's important to remember that
> Windows and *nix descendants are not "all platforms".

Mac OS X understands '/' as the separator at the POSIX layer, but ':'  
as the path separator at the Carbon API (which is only used in  
obscure places from Python).  Earlier versions of Mac OS are no  
longer supported, and you're right -- they only understood ':' as a  
path separator.

>> If anything, a good shortcut for .joinpath() will at least
>> encourage people to use it, thus discouraging hardcoding of path
>> separators.  I expect it would encourage portable paths.
>
> I'm not sure that I believe that the reason that people don't type
> "os.path.join('a', 'b')" is because they are too lazy to type it.
> However, I don't have any evidence either way, so it could be true.

In many cases, when I know I only care about *nix, I will use 'a/b'  
instead of os.path.join because it's just so much more concise and  
more obvious.

The only times I use os.path.join are when I don't know if there will  
be a trailing slash or not, or if I'm actively trying to make  
something cross-platform.

-bob


From jjl at pobox.com  Thu Jan 26 01:02:33 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 26 Jan 2006 00:02:33 +0000 (UTC)
Subject: [Python-Dev] The path module PEP
In-Reply-To: <C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
Message-ID: <Pine.LNX.4.58.0601252357040.6232@alice>

On Thu, 26 Jan 2006, Tony Meyer wrote:
[...]
> Why does reusing a string method for something very different seem  
> like a bad idea, but reusing a mathematical operator for something  
> very different seem like a good idea?
[...]

That's easy -- it's because, if you're going to use a name, people expect
(with some level of trust) that you'll pick a good one.  But people
understand that there are only a few operators to use, so the meaning of
operators is naturally more overloaded than that of method names.


John

From ianb at colorstudy.com  Thu Jan 26 01:10:27 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 18:10:27 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
Message-ID: <43D81373.8030201@colorstudy.com>

Tony Meyer wrote:
> [Ian Bicking]
> 
>> If it were possible to use .join() for joining paths, I think I  
>> wouldn't mind so much.  But reusing a string method for something  
>> very different seems like a bad idea.  So we're left with .joinpath 
>> ().  Still better than os.path.join() I guess, but only a little.   I 
>> guess that's why I'm +1 on /.
> 
> 
> Why does reusing a string method for something very different seem  like 
> a bad idea, but reusing a mathematical operator for something  very 
> different seem like a good idea?  Path's aren't strings, so join () 
> seems the logical choice.  (There are also alternatives to  "joinpath" 
> if the name is the thing: add(), for example).

Paths are strings, that's in the PEP.

As an aside, I think it should be specified what (if any) string methods 
won't be inherited by Path (or will be specifically disabled by making 
them throw some exception).  I think .join() and __iter__ at least 
should be disabled.

>> Precedence in naming means something, and in this case all the  names 
>> have existed for a very long time (as long as Python?)  PEP 8  
>> encourages following naming precedence.  While I don't see a need  to 
>> match every existing function with a method, to the degree they  do 
>> match I see no reason why we shouldn't keep the names.  And I  see 
>> reasons why the names shouldn't be changed.
> 
> 
> PEP 8 encourages following naming precedence within a module, doesn't  
> it?  Guido has said that he'd like to have the standard library  tidied 
> up, at least somewhat (e.g. StringIO.StringIO ->  stringio.StringIO) for 
> Python 3000.  It would make it less painful if  new additions already 
> followed the plan.

I think the use of underscores or squished words isn't as bit a deal as 
the case of modules.  It's often rather ambiguous what a "word" really 
is.  At least in English word combinations slowly and ambiguously float 
towards being combined.  So abspath and abs_path both feel sufficiently 
inside the scope of PEP 8 that precedence is worth maintaining. 
rfc822's getallmatchingheaders method was going too far, but a little 
squishing doesn't bother me, if it is consistent (and it's actually 
easier to be consistent about squishing than underscores).

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From hoffman at ebi.ac.uk  Thu Jan 26 01:22:36 2006
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Thu, 26 Jan 2006 00:22:36 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <20060125233435.GD742@discworld.dyndns.org>
References: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<5.1.1.6.0.20060125174748.04df9540@mail.telecommunity.com>
	<20060125233435.GD742@discworld.dyndns.org>
Message-ID: <Pine.LNX.4.64.0601260021430.27862@qnzvnan.rov.np.hx>

[Charles Cazabon]

> It screams "magic" in a very un-Pythonic (and possibly very
> Perl-like) way.  I'm not aware of any other part of the standard
> library grossly abusing standard operators in this way.

I think the use of the modulo operator for string substitution is
pretty comparable, despite it being in the interpreter rather than in
the stdlib. And some of us have come to love that, too.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From gjc at inescporto.pt  Thu Jan 26 01:30:58 2006
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Thu, 26 Jan 2006 00:30:58 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <1138235459.6668.14.camel@localhost.localdomain>

On Wed, 2006-01-25 at 21:37 +0100, BJ?rn Lindqvist wrote:

> Renaming methods because of PEP 8 (Gustavo, Ian, Jason)
> 
> I'm personally not keen on that. I like most of the names as they
> are. abspath(), joinpath(), realpath() and splitall() looks so much
> better than abs_path(), join_path(), real_path() and split_all() in my
> eyes. If someone like the underscores I'll add it to Open Issues.

<PEP8>
 Function Names

      Function names should be lowercase, with words separated by underscores
      as necessary to improve readability.

      mixedCase is allowed only in contexts where that's already the
      prevailing style (e.g. threading.py), to retain backwards compatibility.

 Method Names and Instance Variables

      Use the function naming rules: lowercase with words separated by
      underscores as necessary to improve readability.
</PEP8>

  It is very clear.  Whether you agree with PEP 8 or not is not relevant
to this discussion.  Since this is a completely new module, it should be
correctly named from the start.  The "familiarity with os.path argument"
is a very weak one, IMHO.

  Plus, the names are full of redundancy.  Why abspath(), joinpath(),
realpath(), splitall()?  Why not instead: absolute(), join(), real(),
split() ?  Remember that they are all methods of a Path class, you don't
need to keep repeating 'path' all over the place[1].

  On a slightly different subject, regarding path / path, I think it
feels much more natural path + path.  Path.join is really just a string
concatenation, except that it adds a path separator in the middle if
necessary, if I'm not mistaken.

  Best regards.

[1]  Yes, I'm the kind of guy who hates struct timeval having tv_sec and
tv_usec field members instead of sec and usec.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic


From thomas at xs4all.net  Thu Jan 26 01:41:13 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 26 Jan 2006 01:41:13 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <20060126004113.GD18916@xs4all.nl>

On Wed, Jan 25, 2006 at 09:37:04PM +0100, BJ?rn Lindqvist wrote:

> Inheritance from string (Jason)

> This issue has been brought up before when people were discussing the
> path module. I think the consensus is that, while the inheritance
> isn't pure, practicality beats purity. It would require to big changes
> to Python and would break to much existing code to not extend string.

This is my only problem with the PEP. It's all very nice that subclassing
from string makes it easier not to break things, but subclassing implies a
certain relationship. That relationship doesn't exist, in this case. Having
the string subclass behave differently than strings (consider the __iter__
and join methods) is a bad idea. I can dish up a half dozen contrived
problem cases, but the main reason I don't like it is that it feels wrong.

If the reason to subclass string is that it's too hard to make an object
'string-like' at a low enough level for the C API, I suggest fixing that,
instead. If that means Path has to wait until Python 2.6, then that's too
bad. The inability to feed C functions/types open() non-string objects has
troubled me before, and though I haven't invested a lot of time in it, I
don't quite understand why it isn't possible. Fixing it in a
backward-compatible manner may require a new __hook__, although I think
using __str__ or __unicode__ shouldn't be too problematic.

Even if fixing the "%es"/"%et" etc formats to the arg-parsing methods is
infeasible, I would prefer a new format code for 'string-alike as C string'
over the unpythonic inappropriate subclassing. Although, even if the
subclassing was completely appropriate, I would much rather improve builtin
support for ducktyping than showcase subclassing ;)

But perhaps I've missed that single, all-convincing argument that it has use
subclassing... In that case, could it be added to the PEP? :)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From jjl at pobox.com  Thu Jan 26 01:43:29 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 26 Jan 2006 00:43:29 +0000 (UTC)
Subject: [Python-Dev] / as path join operator (was: Re: The path module
 PEP)
In-Reply-To: <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz>
Message-ID: <Pine.LNX.4.58.0601260016350.6232@alice>

[John J Lee]
> But it's a very readable way to write a common operation.  Perhaps one
> reason the discrepancy you point out doesn't bother me is that  
> division is the least-used of the +-*/ arithmetic operations.

[Tony Meyer]
> 
> Do you have evidence to back that up? 

No. :)


[Ian Bicking]
> of mine, and in 12k lines there were 34 uses of join, and 1 use of
> division.  In smaller scripts os.path.join tends to show up a lot more

[Tony]
> The problem with these sorts of guesses is that there's no evidence.   
> (Maybe the suggestion that Brett's PhD should collect a corpus of  
> Python scripts was a good one <wink>).  Are mathematicians that under  
> represented?  Is file processing that highly represented?  I have no  
> idea.

A second data point: I looked at ~10k lines of physical data analysis code
I have lying around -- presumably a relatively rare and extreme example as
the Python-world in general goes.  Result:

  140 occurences of os.path.join

  170 physical lines (as opposed to syntactical lines) containing / as a
      division operator (very few lines contained > 1 use of '/', so you
      can multiply 170 by 1.25 to get an upper bound of 213 uses in total)

(To get the second number, I used find and grep heavily but very
cautiously, and final manual count of stubborn lines of grep output with
no use of '/' as division operator)

The fact that even in this extreme case os.path.join is close on the tail
of '/' strongly backs up Ian's guess that, in most Python code, / as
division is rare compared to path joining.

Should we deprecate use of '/' and '//' for division in Python 3.0?

is-he-joking?-ly y'rs


John

From steven.bethard at gmail.com  Thu Jan 26 01:45:20 2006
From: steven.bethard at gmail.com (Steven Bethard)
Date: Wed, 25 Jan 2006 17:45:20 -0700
Subject: [Python-Dev] / as path join operator (was: Re: The path module
	PEP)
In-Reply-To: <Pine.LNX.4.58.0601252331300.6232@alice>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
Message-ID: <d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>

John J Lee wrote:
> On Thu, 26 Jan 2006, Tony Meyer wrote:
> [...]
> > Well, if you include the much larger discussion on python-list,
> > people (including me) have said that removing __div__ is a good
> > idea.  If it's included in the PEP, please at least include a
> > justification and cover the problems with it.  The vast majority of
> > people (at least at the time) were either +0 or -0, not +1.  +0's are
> > not justification for including something.
>
> <bikeshed>
>
> FWLIW, I'm definitely +1 on using / as a path join operator.

My only fear with the / operator is that we'll end up with the same
problems we have for using % in string formatting -- the order of
operations might not be what users expect.  Since join is conceptually
an addition-like operator, I would expect:

    Path('home') / 'a' * 5

to give me:

    home/aaaaa

If I understand it right, it would actually give me something like:

    home/ahome/ahome/ahome/ahome/a

I don't want to claim this is the most common use case, but I've
certainly seen auto-generated paths that look like 'a' * 20, and it
would be a pity if using the / operator for Path objects did the wrong
thing by default here...

STeVe
--
You can wordify anything if you just verb it.
        --- Bucky Katt, Get Fuzzy

From t-meyer at ihug.co.nz  Thu Jan 26 02:22:16 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 26 Jan 2006 14:22:16 +1300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D81373.8030201@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
	<43D81373.8030201@colorstudy.com>
Message-ID: <B1B046F7-5C63-4C65-9CD7-92026F31111D@ihug.co.nz>

[Ian Bicking]
> Paths are strings, that's in the PEP.

No, the PEP says that Path is a *subclass* of string ("Path extends  
from string").  In addition, it's a disputed part of the PEP (see  
elsewhere in this thread).

=Tony.Meyer

From ncoghlan at gmail.com  Thu Jan 26 02:40:15 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Thu, 26 Jan 2006 11:40:15 +1000
Subject: [Python-Dev] / as path join operator
In-Reply-To: <d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
Message-ID: <43D8287F.70209@gmail.com>

Steven Bethard wrote:
> John J Lee wrote:
>> On Thu, 26 Jan 2006, Tony Meyer wrote:
>> [...]
>>> Well, if you include the much larger discussion on python-list,
>>> people (including me) have said that removing __div__ is a good
>>> idea.  If it's included in the PEP, please at least include a
>>> justification and cover the problems with it.  The vast majority of
>>> people (at least at the time) were either +0 or -0, not +1.  +0's are
>>> not justification for including something.
>> <bikeshed>
>>
>> FWLIW, I'm definitely +1 on using / as a path join operator.
> 
> My only fear with the / operator is that we'll end up with the same
> problems we have for using % in string formatting -- the order of
> operations might not be what users expect.  Since join is conceptually
> an addition-like operator, I would expect:
> 
>     Path('home') / 'a' * 5
> 
> to give me:
> 
>     home/aaaaa
> 
> If I understand it right, it would actually give me something like:
> 
>     home/ahome/ahome/ahome/ahome/a
> 
> I don't want to claim this is the most common use case, but I've
> certainly seen auto-generated paths that look like 'a' * 20, and it
> would be a pity if using the / operator for Path objects did the wrong
> thing by default here...

What if we used "subpath" as the name instead of joinpath?

The main appeal to me of the division operation is that it allows multiple 
path elements to be joined on a single line, but the joining method accepts an 
arbitrary number of arguments, which helps with that just as much, and doesn't 
raise precedence and readability questions.

The above example would be:

   Path('home').subpath('a'*5)

An example of retrieving a config file's full name:

Current:   os.path.join(HOME_DIR, APP_DIR, CONFIG_FILE)
Division:  HOME_DIR / APP_DIR / CONFIG_FILE
Subpath:   HOME_DIR.subpath(APP_DIR, CONFIG_FILE)

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From tim.peters at gmail.com  Thu Jan 26 02:35:17 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 25 Jan 2006 20:35:17 -0500
Subject: [Python-Dev] [Python-checkins] r42185 -
	python/trunk/Lib/test/test_socket_ssl.py
In-Reply-To: <20060125083940.E45321E4002@bag.python.org>
References: <20060125083940.E45321E4002@bag.python.org>
Message-ID: <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com>

[neal.norwitz]
> Modified:
>    python/trunk/Lib/test/test_socket_ssl.py
> Log:
> There was a race condition where the connector would try to connect
> before the listener was ready (on gentoo x86 buildslave).  This
> caused the listener to not exit normally since nobody connected to it
> (waited in accept()).  The exception was raised in the other thread
> and the test failed.

Good catch!  Thank you.

> This fix doesn't completely eliminate the race, but should make it
> near impossible to trigger.  Hopefully it's good enough.

Which race do you have in mind?  The server socket doesn't need to do
.accept() before a client socket can connect -- the server socket only
needs to have done .listen() for a connection to succeed.

> +    listener_ready = threading.Event()

...

[in the server]
>          s = socket.socket()
>          s.bind(('', PORT))
>          s.listen(5)
> +        listener_ready.set()
>          s.accept()

...

[in the client]
>      def connector():
> +        listener_ready.wait()
>          s = socket.socket()
>          s.connect(('localhost', PORT))

Because the server doesn't set listener_ready until after the server
has done listen(),  and the client waits for that event, it "should
be" 100% reliable that the client's connect() succeeds.

Or do you have some other race in mind?

From nnorwitz at gmail.com  Thu Jan 26 02:54:05 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Wed, 25 Jan 2006 17:54:05 -0800
Subject: [Python-Dev] [Python-checkins] r42185 -
	python/trunk/Lib/test/test_socket_ssl.py
In-Reply-To: <1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com>
References: <20060125083940.E45321E4002@bag.python.org>
	<1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com>
Message-ID: <ee2a432c0601251754l5d4ec3e0w28c06694511cf088@mail.gmail.com>

On 1/25/06, Tim Peters <tim.peters at gmail.com> wrote:
>
> Because the server doesn't set listener_ready until after the server
> has done listen(),  and the client waits for that event, it "should
> be" 100% reliable that the client's connect() succeeds.
>
> Or do you have some other race in mind?

That's what I was thinking of.  I thought you had to be accept()ing
prior to connect() working.  I thought listen() only sets the # of
outstanding connections allowed (basically internal buffer).  But if
the listen() is sufficient, I agree there is no race.

n

From tim.peters at gmail.com  Thu Jan 26 03:11:42 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Wed, 25 Jan 2006 21:11:42 -0500
Subject: [Python-Dev] [Python-checkins] r42185 -
	python/trunk/Lib/test/test_socket_ssl.py
In-Reply-To: <ee2a432c0601251754l5d4ec3e0w28c06694511cf088@mail.gmail.com>
References: <20060125083940.E45321E4002@bag.python.org>
	<1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com>
	<ee2a432c0601251754l5d4ec3e0w28c06694511cf088@mail.gmail.com>
Message-ID: <1f7befae0601251811r5d76e908ge0441f9313e3ce1b@mail.gmail.com>

[Neal Norwitz]
> That's what I was thinking of.  I thought you had to be accept()ing
> prior to connect() working.  I thought listen() only sets the # of
> outstanding connections allowed (basically internal buffer).  But if
> the listen() is sufficient, I agree there is no race.

listen() "should be" sufficient -- it allocates space for a connection
queue, and marks the socket as accepting connections.  accept() merely
does a blocking "pop" on the queue listen() created.

For fun ;-), run it in slow motion, by hand, on that box:  open Python
in one window, and create a server socket without doing accept().  In
another window, try to connect to the first socket's address.  The
connect should succeed at once.

Now socket docs are typically clear as mud, and it's possible that box
has a bad implementation.  "Thought experiments" can help:  if an
accept() were necessary before a connection could succeed, why would
listen() have a backlog argument at all?  Well, if everything else
were sane, it wouldn't ;-)

From trentm at ActiveState.com  Thu Jan 26 03:39:44 2006
From: trentm at ActiveState.com (Trent Mick)
Date: Wed, 25 Jan 2006 18:39:44 -0800
Subject: [Python-Dev] / as path join operator
In-Reply-To: <43D8287F.70209@gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<43D8287F.70209@gmail.com>
Message-ID: <20060126023944.GC11993@activestate.com>

[Nick Coghlan wrote]
> What if we used "subpath" as the name instead of joinpath?

"append"?

not-a-big-fan-of-joinpath-either-ly yours,
Trent

-- 
Trent Mick
TrentM at ActiveState.com

From xavier.morel at masklinn.net  Fri Jan 20 20:14:22 2006
From: xavier.morel at masklinn.net (Morel Xavier)
Date: Fri, 20 Jan 2006 20:14:22 +0100
Subject: [Python-Dev] yield back-and-forth?
In-Reply-To: <E1F00wg-0006V6-8c@swing.co.at>
References: <E1F00wg-0006V6-8c@swing.co.at>
Message-ID: <43D1368E.1090001@masklinn.net>

Christian Tanzer wrote:
> How about:
> 
>   def main_generator():
>       ...
>       yield * sub_generator()
> 
> Ducking-ly yrs,
> 

I like that one, but I'd stick the star to the generator (e.g. yield 
*sub_generator), the meaning being to "unpack" the generator into the 
yield, same as unpacking a sequence passed as function parameters.

As an extension, the syntax may even (though I'm not sure it'd be a good 
idea to do so) work with any iterable, behaving the same way (yielding 
the successive values of the iterable object)

From gabriel.becedillas at corest.com  Tue Jan 24 17:15:23 2006
From: gabriel.becedillas at corest.com (Gabriel Becedillas)
Date: Tue, 24 Jan 2006 13:15:23 -0300
Subject: [Python-Dev] pystate.c changes for Python 2.4.2
In-Reply-To: <43C68823.3000105@corest.com>
References: <43C68823.3000105@corest.com>
Message-ID: <43D6529B.8040909@corest.com>

Thanks to everyone for the great support.
I've submitted a patch for this: 
http://sourceforge.net/tracker/index.php?func=detail&aid=1413181&group_id=5470&atid=305470.


From ianb at colorstudy.com  Thu Jan 26 04:02:07 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 21:02:07 -0600
Subject: [Python-Dev] / as path join operator
In-Reply-To: <d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
Message-ID: <43D83BAF.20401@colorstudy.com>

Steven Bethard wrote:
> My only fear with the / operator is that we'll end up with the same
> problems we have for using % in string formatting -- the order of
> operations might not be what users expect.  Since join is conceptually
> an addition-like operator, I would expect:
> 
>     Path('home') / 'a' * 5
> 
> to give me:
> 
>     home/aaaaa
> 
> If I understand it right, it would actually give me something like:
> 
>     home/ahome/ahome/ahome/ahome/a

Both of these examples are rather silly, of course ;)  There's two 
operators currently used commonly with strings (that I assume Path would 
inherit): + and %.  Both actually make sense with paths too.

   filename_template = '%(USER)s.conf'
   p = Path('/conf') / filename_template % os.environ
which means:
   p = (Path('/conf') / filename_template) % os.environ

But probably the opposite is intended.  Still, it will usually be 
harmless.  Which is sometimes worse than usually harmful.

+ seems completely innocuous, though:

   ext = '.jpg'
   name = fields['name']
   image = Path('/images') / name + ext

It doesn't really matter what order it happens in there.  Assuming 
concatenation results in a new Path object, not a str.

-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From barry at python.org  Thu Jan 26 04:51:08 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Jan 2006 22:51:08 -0500
Subject: [Python-Dev] / as path join operator (was: Re: The path
	module	PEP)
In-Reply-To: <034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz>
Message-ID: <1138247468.30252.11.camel@geddy.wooz.org>

On Thu, 2006-01-26 at 12:51 +1300, Tony Meyer wrote:
> [John J Lee]
> > But it's a very readable way to write a common operation.  Perhaps one
> > reason the discrepancy you point out doesn't bother me is that  
> > division is
> > the least-used of the +-*/ arithmetic operations.
> 
> Do you have evidence to back that up?  It seems a strange claim.   
> Outside of doing 'maths-y' work, I would think I'd use + most (but  
> for strings), then / (for percentages).

I haven't followed the entire thread (I'll try to find time to catch up)
but while I think using __div__ to mean path concatenation is cute, I'm
not sure I'd like to see it all over the place.  It does seem awfully
"FAST" ("facinating and stomach turning" to use a term from years ago).

What I don't like about os.path.join() having to import os and having to
type all those characters over and over again.  What I /like/ about
os.path.join is that you can give it a bunch of path components and have
it return the correctly joined path, e.g. os.path.join('a, 'b', 'c').
That seems more efficient than having to create a bunch of intermediate
objects.

All in all, I'd have to say I'm -0 on __div__ for path concatenation.

-Barry


-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/763847fb/attachment.pgp 

From t-meyer at ihug.co.nz  Thu Jan 26 05:03:00 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Thu, 26 Jan 2006 17:03:00 +1300
Subject: [Python-Dev] The path module PEP
In-Reply-To: <1138235459.6668.14.camel@localhost.localdomain>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<1138235459.6668.14.camel@localhost.localdomain>
Message-ID: <79484C59-6A15-40F4-998B-0DFBA742D272@ihug.co.nz>

[Gustavo J. A. M. Carneiro]
> Plus, the names are full of redundancy.  Why abspath(), joinpath(),
> realpath(), splitall()?  Why not instead: absolute(), join(), real(),
> split() ?  Remember that they are all methods of a Path class, you  
> don't
> need to keep repeating 'path' all over the place.

+1 for all of those that aren't also string methods.

+0.9 for those that are string methods, although I suppose this  
depends on how much like a string a Path ends up like.

Other than join() (covered in the __div__ discussion), split() is an  
interesting case, since the default split-on-whitespace (str.split)  
doesn't make a whole lot of sense with a Path, but split-on-pathsep  
(os.path.split) does.  Does it make sense to be able to split a path  
on something else (like str.split), or should people just convert to/ 
from a string?  Should there be a maxsplit argument?

=Tony.Meyer

From barry at python.org  Thu Jan 26 05:17:43 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Jan 2006 23:17:43 -0500
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D81373.8030201@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
	<43D81373.8030201@colorstudy.com>
Message-ID: <1138249063.30246.24.camel@geddy.wooz.org>

On Wed, 2006-01-25 at 18:10 -0600, Ian Bicking wrote:

> Paths are strings, that's in the PEP.
> 
> As an aside, I think it should be specified what (if any) string methods 
> won't be inherited by Path (or will be specifically disabled by making 
> them throw some exception).  I think .join() and __iter__ at least 
> should be disabled.

Whenever I see derived classes deliberately disabling base class
methods, I see red flags that something in the design of the hierarchy
isn't right.  While I understand that you want to be able to use Path
instances anywhere a string is currently used, I'm not sure that
deriving from str is the right thing.  Maybe deriving from basestring
would be better, but even then I'm not sure.  Is it possible that we
don't need Path objects to interchangeable with strings, but just that
we can get away with expanding a few critical functions (such as
open())?

> I think the use of underscores or squished words isn't as bit a deal as 
> the case of modules.  It's often rather ambiguous what a "word" really 
> is.  At least in English word combinations slowly and ambiguously float 
> towards being combined.  So abspath and abs_path both feel sufficiently 
> inside the scope of PEP 8 that precedence is worth maintaining. 
> rfc822's getallmatchingheaders method was going too far, but a little 
> squishing doesn't bother me, if it is consistent (and it's actually 
> easier to be consistent about squishing than underscores).

For something like "abspath" which is really an abbreviation + word, I
suppose squishing them isn't so bad.  The alternative is absolute_path()
which is certainly more readable if a bit of a pain to write.  It's a
trade-off that should be made for practical purposes.  I've definitely
come to prefer spellings like is_absolute over isabsolute, and in
general dislike squish words.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/00fb0c1a/attachment.pgp 

From stephen at xemacs.org  Thu Jan 26 05:21:10 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Thu, 26 Jan 2006 13:21:10 +0900
Subject: [Python-Dev] / as path join operator
In-Reply-To: <d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	(Steven Bethard's message of "Wed, 25 Jan 2006 17:45:20 -0700")
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
Message-ID: <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Steven" == Steven Bethard <steven.bethard at gmail.com> writes:

    Steven> My only fear with the / operator is that we'll end up with
    Steven> the same problems we have for using % in string formatting
    Steven> -- the order of operations might not be what users expect.

Besides STeVe's example, (1) I think it's logical to expect that

    Path('home') / 'and/or'

points to a file named "and/or" in directory "home", not to a file
named "or" in directory "home/and".

(2) Note that '/' is also the path separator used by URIs, which RFC
2396 gives different semantics from Unix.  Most of my Python usage to
date has been heavily web-oriented, and I'd have little use for /
unless it follows RFC 2396.  By that, I mean that I would want the /
operator to treat its rhs as a relative reference, so the result would
be computed by the algorithm in section 5.2 of that RFC.  But this is
not correct (realpath) semantics on Unix.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From barry at python.org  Thu Jan 26 05:25:03 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Jan 2006 23:25:03 -0500
Subject: [Python-Dev] / as path join operator
In-Reply-To: <43D8287F.70209@gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<43D8287F.70209@gmail.com>
Message-ID: <1138249503.30245.26.camel@geddy.wooz.org>

On Thu, 2006-01-26 at 11:40 +1000, Nick Coghlan wrote:

> The main appeal to me of the division operation is that it allows multiple 
> path elements to be joined on a single line, but the joining method accepts an 
> arbitrary number of arguments, which helps with that just as much, and doesn't 
> raise precedence and readability questions.

Or the need to employ ugliness like backslashes when you have to split
the join across multiple lines.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/c251763e/attachment.pgp 

From ianb at colorstudy.com  Thu Jan 26 05:30:40 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 22:30:40 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <1138249063.30246.24.camel@geddy.wooz.org>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	
	<43D6BA85.8070007@colorstudy.com>	
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>	
	<43D80621.1080300@colorstudy.com>	
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>	
	<43D81373.8030201@colorstudy.com>
	<1138249063.30246.24.camel@geddy.wooz.org>
Message-ID: <43D85070.1010809@colorstudy.com>

Barry Warsaw wrote:
> On Wed, 2006-01-25 at 18:10 -0600, Ian Bicking wrote:
> 
> 
>>Paths are strings, that's in the PEP.
>>
>>As an aside, I think it should be specified what (if any) string methods 
>>won't be inherited by Path (or will be specifically disabled by making 
>>them throw some exception).  I think .join() and __iter__ at least 
>>should be disabled.
> 
> 
> Whenever I see derived classes deliberately disabling base class
> methods, I see red flags that something in the design of the hierarchy
> isn't right.

IMHO the hierarchy problem is a misdesign of strings; iterating over 
strings is usually a bug, not a deliberately used feature.  And it's a 
particularly annoying bug, leading to weird results.

In this case a Path is not a container for characters.  Strings aren't 
containers for characters either -- apparently they are containers for 
smaller strings, which in turn contain themselves.  Paths might be seen 
as a container for other subpaths, but I think everyone agrees this is 
too ambigous and implicit.  So there's nothing sensible that __iter__ 
can do, and having it do something not sensible (just to fill it in with 
something) does not seem very Pythonic.

join is also a funny method that most people wouldn't expect on strings 
anyway.  But putting that aside, the real issue I see is that it is a 
miscognate for os.path.join, to which it has no relation.  And I can't 
possibly imagine what you'd use it for in the context of a path.


-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From ianb at colorstudy.com  Thu Jan 26 05:35:23 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Wed, 25 Jan 2006 22:35:23 -0600
Subject: [Python-Dev] The path module PEP
In-Reply-To: <1138235459.6668.14.camel@localhost.localdomain>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<1138235459.6668.14.camel@localhost.localdomain>
Message-ID: <43D8518B.70900@colorstudy.com>

Gustavo J. A. M. Carneiro wrote:
>   On a slightly different subject, regarding path / path, I think it
> feels much more natural path + path.  Path.join is really just a string
> concatenation, except that it adds a path separator in the middle if
> necessary, if I'm not mistaken.

No, it isn't, which maybe is why / is bad.  os.path.join(a, b) basically 
returns the path as though b is interpreted to be relative to a.  I.e., 
os.path.join('/foo', '/bar') == '/bar'.  Not much like concatenation at 
all.  Plus string concatenation is quite useful with paths, e.g., to add 
an extension.

If a URI class implemented the same methods, it would be something of a 
question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz' 
(and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does). 
I assume it would be be the latter, and urljoin would be a different 
method, maybe something novel like "urljoin".


-- 
Ian Bicking  |  ianb at colorstudy.com  |  http://blog.ianbicking.org

From barry at python.org  Thu Jan 26 05:41:23 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Jan 2006 23:41:23 -0500
Subject: [Python-Dev] / as path join operator
In-Reply-To: <43D83BAF.20401@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<43D83BAF.20401@colorstudy.com>
Message-ID: <1138250483.30252.38.camel@geddy.wooz.org>

On Wed, 2006-01-25 at 21:02 -0600, Ian Bicking wrote:

>    ext = '.jpg'
>    name = fields['name']
>    image = Path('/images') / name + ext

Here's a good example of why I ultimately don't like __div__.  The last
line seems quite non-obvious to me.  It's actually jarring enough that I
have to stop and think about what it means because it /looks/ like
there's math going on.

OTOH, something like:

image = Path('', 'images', name) + ext

or even better

image = Path.join('', 'images', name) + ext

where .join is a staticmethod, seems much clearer.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/4244e51a/attachment.pgp 

From barry at python.org  Thu Jan 26 05:48:52 2006
From: barry at python.org (Barry Warsaw)
Date: Wed, 25 Jan 2006 23:48:52 -0500
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D85070.1010809@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<43D80621.1080300@colorstudy.com>
	<C082AF44-0C9D-4CBD-9985-E9028B6E26FF@ihug.co.nz>
	<43D81373.8030201@colorstudy.com>
	<1138249063.30246.24.camel@geddy.wooz.org>
	<43D85070.1010809@colorstudy.com>
Message-ID: <1138250932.30252.45.camel@geddy.wooz.org>

On Wed, 2006-01-25 at 22:30 -0600, Ian Bicking wrote:

> IMHO the hierarchy problem is a misdesign of strings; iterating over 
> strings is usually a bug, not a deliberately used feature.  And it's a 
> particularly annoying bug, leading to weird results.

Agreed.  I've written iteration code that has to special case
basestrings before and that's particularly ugly.

> In this case a Path is not a container for characters.  Strings aren't 
> containers for characters either -- apparently they are containers for 
> smaller strings, which in turn contain themselves.  Paths might be seen 
> as a container for other subpaths, but I think everyone agrees this is 
> too ambigous and implicit.  So there's nothing sensible that __iter__ 
> can do, and having it do something not sensible (just to fill it in with 
> something) does not seem very Pythonic.
> 
> join is also a funny method that most people wouldn't expect on strings 
> anyway.  But putting that aside, the real issue I see is that it is a 
> miscognate for os.path.join, to which it has no relation.  And I can't 
> possibly imagine what you'd use it for in the context of a path.

Good points, but to me that argues against having any inheritance
relationship between strings and Paths rather than having such a
relationship and disabling certain methods.  Thomas's post seemed more
on-target for me and I'd like to see that idea fleshed out in more
detail.  If it's proved to be an impossible (or merely an extremely
infeasible) task, then I think we can discuss the shortcut of deriving
from strings.  It just seems gross so I'd like to be sure there's no
better way.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060125/ce83f8f2/attachment-0001.pgp 

From martin at v.loewis.de  Thu Jan 26 06:57:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jan 2006 06:57:13 +0100
Subject: [Python-Dev] [Python-checkins] r42185
	-	python/trunk/Lib/test/test_socket_ssl.py
In-Reply-To: <ee2a432c0601251754l5d4ec3e0w28c06694511cf088@mail.gmail.com>
References: <20060125083940.E45321E4002@bag.python.org>	<1f7befae0601251735k3b000b97u6ba0f9a03af80ad4@mail.gmail.com>
	<ee2a432c0601251754l5d4ec3e0w28c06694511cf088@mail.gmail.com>
Message-ID: <43D864B9.8000302@v.loewis.de>

Neal Norwitz wrote:
> That's what I was thinking of.  I thought you had to be accept()ing
> prior to connect() working.  I thought listen() only sets the # of
> outstanding connections allowed (basically internal buffer).  But if
> the listen() is sufficient, I agree there is no race.

Actually, that's the whole point of listen(). Even if you initially do
accept() quickly, it might be that another connection request (SYN)
arrives between return of accept() and the next call to accept();
this is essentially the same as connection requests arriving before
the first accept() call.

The kernel will acknowledge the SYN packet immediately, and do that for
at most "number-of-backlog" incoming connections. Then, subsequent
accept() calls will immediately return.

Regards,
Martin

From tony.meyer at gmail.com  Thu Jan 26 05:31:26 2006
From: tony.meyer at gmail.com (Tony Meyer)
Date: Thu, 26 Jan 2006 17:31:26 +1300
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
Message-ID: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>

Here's the draft for the first January summary.  If anyone has time to
send me/Steve comments/suggestions/corrections/additions, those will
be most welcome.  The Py_ssize_t threads were particularly over my
head.  As always, many thanks :)

=============
Announcements
=============

----------------------------
QOTF: Quote of the Fortnight
----------------------------

Guido, on the crashability of Python:

    I'm not saying it's uncrashable. I'm saying that if you crash it,
it's a bug unless proven harebrained.

Contributing thread:

- `Include ctypes into core Python?
<http://mail.python.org/pipermail/python-dev/2006-January/059621.html>`_

[SJB]

------------------------------------
Brett Cannon looking for Ph.D. ideas
------------------------------------

Brett Cannon is looking for Python-related ideas to form the subject
of his PhD dissertation, under Eric Wohlstadter at the University of
British Columbia.  He has three areas in which he has possible funding
(XML integration, game scripting support, and AOP); he is open to
grants from anyone else interested in particular Python development. 
If anyone has suggestions for topics, Brett is listening!

So far:

* Phillip J. Eby mentioned that he has been doing some research on
implementing a non-traditional form of AOP in Python.
* Bill Janssen suggested a system to compile Python to AJAX.
* Steve Holden suggested a Python learning system.
* Ian Bicking suggested Boxer_ implemented for Python.
* Armin Rigo suggested PyPy related work.
* Jason Orendorff suggested collecting a downloadable corpus of Python
source code, and performing various analyses on it.
* Dennis Allison suggested interpreter structures for Python-like
languages that do not use a global interpreter lock, as well as
various other topics.

 .. _Boxer: http://dewey.soe.berkeley.edu/boxer.html

Contributing thread:

 - `Ph.D. dissertation ideas?
<http://mail.python.org/pipermail/python-dev/2006-January/059702.html>`__

[TAM]

---------------------------------
2.4 Documentation with Commentary
---------------------------------

Ian Bicking put an instance_ of his commenting system, Commentary_, up
against the full 2.4 documentation. If you'd like to see how the
Python docs look with user-added comments, now's your chance!

.. _instance: http://pythonpaste.org/comment/python24/
.. _Commentary: http://pythonpaste.org/commentary/

Contributing thread:

 - `[Doc-SIG] that library reference, again
<http://mail.python.org/pipermail/python-dev/2006-January/059353.html>`__

[SJB]

---------------------------------------
Builds of the Development Documentation
---------------------------------------

Be sure to keep yourself up to date with the newest Python
documentation - Neal Norwitz has arranged it so that the current
Python development documentation is now automatically generated every
12 hours and placed at http://docs.python.org/dev/.

Contributing thread:

 - `automated builds and tests
<http://mail.python.org/pipermail/python-dev/2006-January/059362.html>`__

[SJB]

--------------------------------
PEPs now automatically installed
--------------------------------

Martin v. L?wis has arranged for an automatic installation of PEPs_:
they will now get published in the subversion post-commit.

Contributing thread:

 - `Installing PEPs
<http://mail.python.org/pipermail/python-dev/2006-January/059745.html>`__

 .. _PEPs: http://www.python.org/peps

[TAM]

=========
Summaries
=========

---------------------------------------------------
Using buildbot to automate testing of Python builds
---------------------------------------------------

A buildbot_ has been setup to facilitate `automated testing of the
Python SVN`_.  There are currently five slaves setup:

* Linux x86 (gentoo 2.6, glibc 2.3)
* Linux AMD64 (gentoo 2.6, glibc 2.3)
* Solaris10 sparc
* OS X 10.3 G5
* OS X 10.4 G4

Ideally, all supported platforms will be added (including OS, version,
hardware, and various configurations); the slaves are testing the
trunk and the 2.4 branch.  Note that Window buildbot slaves have to
have an appropriate compiler (MS VC 7.1), otherwise it can't compile
the code; the Windows Python would also like to build several other
packages (like bz2 and Tcl/Tk).  A periodic "clean build" may also be
added; there was discussion whether this should be done via a patch to
the buildbot master, via an IRC command, or with a separate scheduler.

Martin v. L?wis would rather buildbot produced static pages than act
as a web server (for security reasons).  Brian Warner explained that
it once did, but was changed to make it easier to implement features
like the "Force Build" button.  An explanation of the security was
given, and it seems satisfactory at the moment.

The buildbot testing has already produced results: a compiler warning
on OS X 10.3 was noticed.  A discussion took place as to whether this
should be fixed (it should) and who could do it (Ronald Oussoren
volunteered, with help from Skip Montanaro).

Fredrik Lundh had been playing with "automatic module discovery" in an
attempt to figure out how complete the library reference really is,
tracking down missing reference pages and generating stubs for pages
that "should exist".  He suggested that the buildbot could also run
this script; at the time of this summary this was in progress.

 .. _automated testing of the Python SVN: http://www.python.org/dev/buildbot/
 .. _buildbot: http://buildbot.sourceforge.net/

Contributing threads:

 - `buildbot <http://mail.python.org/pipermail/python-dev/2006-January/059359.html>`__
 - `Buildbot questions
<http://mail.python.org/pipermail/python-dev/2006-January/059429.html>`__
 - `[Buildbot-devel] Re: buildbot
<http://mail.python.org/pipermail/python-dev/2006-January/059455.html>`__
 - `Automated Python testing
<http://mail.python.org/pipermail/python-dev/2006-January/059460.html>`__
 - `building a module catalogue with buildbot
<http://mail.python.org/pipermail/python-dev/2006-January/059623.html>`__
 - `Buildbot: doing occasional full builds
<http://mail.python.org/pipermail/python-dev/2006-January/059643.html>`__

[TAM]

----------------------
A "Rejected Ideas" PEP
----------------------

Alexander Kozlovsky submitted a pre-PEP for the oft-repeated request
that Python do away with its explicit "self" parameter.  Guido
responded with a definitive "This won't change", but didn't want to
allow the PEP to be created just to be rejected.  A number of people
felt that the mailing lists weren't a sufficiently visible place for
this response, especially considering the huge number of times the
request has been presented.  Thomas Wouters proposed a "Rejected
Ideas" PEP that would list some of the more commonly raised proposals
that weren't really even worthy of a PEP, with references to the
appropriate python-list or python-dev discussions.  Tim Peters
nominated the alternate title "Don't Bother: PEPs Rejected Before
Being Written" which perhaps better summarizes the intent.  People
seemed generally in favor of the idea, and Thomas Wouters offered to
write up a first draft indicating three major rejected ideas:

* removing the explicit "self" parameter
* upgrading the Python parser beyond a simple LL(1)
* allowing parentheses to be omitted in function calls

At the time of this summary, no PEP was yet available.

Contributing threads:

 - `Draft proposal: Implicit self in Python 3.0
<http://mail.python.org/pipermail/python-dev/2006-January/059446.html>`__
 - `Rejected ideas PEP (was re: Draft proposal: Implicit self in
Python 3.0) <http://mail.python.org/pipermail/python-dev/2006-January/059514.html>`__

[SJB]

-------------------------------
Using ssize_t as the index type
-------------------------------

Continuing from `last week`_, more process was made towards `PEP 353`_.

Tim Peters, Neal Norwitz, and Martin v. L?wis discussed replacing '#'
as a format character, and how this could be down in a backwards
compatible way; they also discussed how Py_ssize_t could be output
correctly in a printf-style statement.  Martin suggested using %zd,
but this isn't in VC 7.1; Tim suggested that a macro be added to
pyport.h, which would use "z" where possible, and define something
else where not (e.g. Windows).  Tim also suggested that PyErr_Format()
and PyString_Format() should be taught to recognize the "z" qualifier,
with the macros hidden inside the implementation of those functions.

M.-A. Lemburg pointed out other changes of output parameter types
which will cause problems in extensions, and worried that authors
wouldn't be able to get these converted in time for Python 2.5. 
Martin disagreed.  While code will continue to work with 32-bit
systems with little work, more needs to be done for 64-bit systems. 
Martin suggested that users could continue to use 32-bit Python (since
nothing will be gained by using 64-bit anyway) until the required
changes are made.  For more details, see the "Open Issues" section of
the PEP.

 .. _last week:
http://www.python.org/dev/summary/2005-12-16_2005-12-31.html#using-ssize-t-as-the-index-type
 .. _PEP 353: http://www.python.org/peps/pep-0353.html

Contributing threads:

 - `[Python-checkins] commit of r41906 -
python/branches/ssize_t/Objects/obmalloc.c
<http://mail.python.org/pipermail/python-dev/2006-January/059381.html>`__
 - `New PEP: Using ssize_t as the index type
<http://mail.python.org/pipermail/python-dev/2006-January/059437.html>`__
 - `[Python-checkins] r41972 -
python/branches/ssize_t/Objects/funcobject.c
<http://mail.python.org/pipermail/python-dev/2006-January/059516.html>`__
 - `Py_ssize_t output parameters (Was: [Python-checkins] r41971 ...)
<http://mail.python.org/pipermail/python-dev/2006-January/059597.html>`__

[TAM]

-------------------------------------
Adding ctypes to the standard library
-------------------------------------

Thomas Heller suggested that ctypes be included in core Python
(starting with 2.5).  The common response was that while ctypes is a
useful, popular, mature module, it does make it very easy to get a
core dump, which violates the guideline that if you get a core dump
it's a bug in Python or in a third party extension or you're doing
something harebrained.  On the other hand, it was pointed out that the
dl module suffers from the same problem, and is included without any
warnings (the documentation was later fixed to include warnings).

Martin v. L?wis suggested making ctypes a dynamically loaded module
(ctypes.pyd), so administrators could remove it, and
I could also make it a separate option in the Windows installer, so
administrators could opt out of installing it.  Everyone seemed happy
with prominent warnings in the documentation, and so this is how it
was checked in.

Contributing thread:

 - `Include ctypes into core Python?
<http://mail.python.org/pipermail/python-dev/2006-January/059604.html>`__

[TAM]

-----------------------------------------
Adding tests when no fix is yet available
-----------------------------------------

After Georg Brandl checked in a test for the compiler module without a
corresponding fix, Tim Peters reverted the change, indicating that no
check-in should ever leave the repository in such a state that
``regrtest.py -uall`` fails. The ensuing thread discussed a number of
ways of checking in a test for a bug when code to fix the bug was not
yet available. Neal Norwitz proposed adding two files to Lib/test,
outstanding_bugs.py and outstanding_crashes.py, which would include
the appropriate tests, but which would not be run under ``regrtest.py
-uall``.  Some people were concerned about the maintainability of
keeping failing tests for a module separate from the other tests of
that module, so Neal Norwitz and Scott David Daniels presented a
patch_ and a recipe_ for decorators that mark individual test
functions as failing.

No official pronouncement on how failing tests should be added to the
Python test suite had been made at the time of this summary.

.. _patch: http://python.org/sf/1399935
.. _recipe: http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/466288

Contributing thread:

 - `Checking in a broken test was: Re: [Python-checkins] r41940 -
python/trunk/Lib/test/test_compiler.py
<http://mail.python.org/pipermail/python-dev/2006-January/059481.html>`__
 - `r42003 - python/trunk/Lib/test/outstanding_bugs.pypython/trunk/Lib/test/outstanding_crashes.py
<http://mail.python.org/pipermail/python-dev/2006-January/059599.html>`__

[SJB]

-------------------------------------------
Running the test suites in multiple locales
-------------------------------------------

Georg Brandl noted that unicode format modifiers were (in SVN)
incorrectly using the locale (one should always use locale.format to
get locale-aware formatting).  Fredrik Lundh suggested that all test
should be run in a different locale than C to detect these bugs,
perhaps adding a "use setlocale" flag to the test runner, to simplify
this.  Neal Norwitz made a patch to do this, finding a failure in
test_email and two failures in test_re.

Contributing thread:

 - `locale and LC_NUMERIC
<http://mail.python.org/pipermail/python-dev/2006-January/059530.html>`__

[TAM]

-----------------------------------------------------------
Replacing the docutils directory with an svn:externals link
-----------------------------------------------------------

David Goodger replaced the static copy of docutils in the "peps"
repository with a svn:externals link to the docutils repository. There
was some concern about svn querying servers other than svn.python.org
(the docutils repository is hosted on svn.berlios.de) and about
linking to a potentially broken repository.  David assured python-dev
that the svn.berlios.de server was hardly ever unreachable, and
changed the svn:externals link to select the 0.4 revision of docutils
instead of the trunk.  That guaranteed a non-broken copy of docutils,
and made updating to a new revision just a simple propedit.

Contributing thread:

 - `r42015 - peps/trunk
<http://mail.python.org/pipermail/python-dev/2006-January/059676.html>`__

[SJB]

--------------
Tkinter status
--------------

A brief message from Joseph Rodrigues asking about the status of
Tkinter in Python started a short thread about how Tkinter is
maintained in Python. Martin v. L?wis is the primary maintainer, and
he mainly focuses on integrating user contributions and making sure
that Python is compatible with the latest release of Tcl/Tk.  New
widgets don't get wrapped automatically - if you want one supported,
you should submit a patch.

Contributing thread:

 - `Tkinter <http://mail.python.org/pipermail/python-dev/2006-January/059588.html>`__

[SJB]

------------------------------------
Roadmap for ConfigParser development
------------------------------------

Once again, the output capability of ConfigParser was brought up; this
time by Facundo Bastista, who `submitted a patch`_ that orders the
output so that doctest output can be easily reliable.  He pointed out
that there is `another open patch`_ that allows the user to specify
the order through an "ordered dictionary".  Guido explained that he
didn't feel that it mattered, as long as the patch also allows
comments to be preserved; his preference was for everything (ordering,
blank lines, and comments) to be preserved.  Tony Meyer point out some
examples of previous_ `python-dev`_ discussion_ about how ConfigParser
should be modified, and that SpamBayes_ includes a ConfigParser
subclass that allows 'surgical' editing of files.

 .. _submitted a patch: http://python.org/sf/1399309
 .. _another open patch: http://python.org/sf/1371075
 .. _previous: http://mail.python.org/pipermail/python-dev/2004-October/049454.html
 .. _python-dev:
http://mail.python.org/pipermail/python-dev/2004-October/049527.html
 .. _discussion:
http://aspn.activestate.com/ASPN/Mail/Message/python-dev/1483518
 .. _SpamBayes: http://spambayes.org

Contributing thread:

 - `ConfigParser to save with order
<http://mail.python.org/pipermail/python-dev/2006-January/059480.html>`__

[TAM]

-------------------------------
Patches for the logging package
-------------------------------

Shane Hathaway asked about the process for submitting patches for the
logging package.  Vinay Sajip is the primary maintainer, and accepts
patches (compatible with Python 1.5.2) through sourceforge as normal,
though personal emails are appropriate if you are worried about
spending time on a patch that is not accepted.  There was a brief
followup discussion about how to get the logging package to interact
nicely with external log rotation tools, including the possible
creation of a ``reopen()`` function for files, but the thread trailed
off without any solid conclusion on this issue.

Contributing thread:

 - `Logging enhancements
<http://mail.python.org/pipermail/python-dev/2006-January/059566.html>`__

[SJB]

----------------------------------
os.path.getmtime() bugs on Windows
----------------------------------

Christian Tismer noted that if, in a single interactive session on
Windows, create a file, get its os.path.getmtime(), change your time
zone, and get its os.path.getmtime() again, the time stamps are
different.  He asked whether there was a way to circumvent the
problem, or if a patch could be created.  Tim Peters gave him a
`reference to a CodeProject page` that discusses the problem.  Martin
v. L?wis has meant to work on a patch for several years; his plan is
to drop usage of msvcrt's stat(3), and instead implement os.stat in
terms of Windows API directly - this would also have the advantage
that subsecond time-stamps can be exposed.  This patch would need to
be implemented twice; once for Win9X (ANSI file names) and once for
NT+ (unicode file names).

 .. _reference to a CodeProject page:
http://www.codeproject.com/datetime/dstbugs.asp

Contributing thread:

 - `os.path.getmtime on Windows
<http://mail.python.org/pipermail/python-dev/2006-January/059741.html>`__

[TAM]

----------------------------------------
Generating Python-ast.c and Python-ast.h
----------------------------------------

Python-ast.c and Python-ast.h are generated automatically from
Parser/asdl_c.py and then checked into Subversion. However, a ``make
distclean`` removes these files, and if you do not have a 2.2+ version
of Python available, they cannot be regenerated.  The best solution
seemed to be to alter ``make distclean`` to not remove Python-ast.c
and Python-ast.h as they only need to be regenerated when the AST
definition changes, which is rather infrequent.

Contributing thread:

- `[Python-checkins] commit of r41880 -
python/trunk/Python/Python-ast.c
<http://mail.python.org/pipermail/python-dev/2006-January/059356.html>`__

[SJB]

-----------------------------------------
Allowing dict subclasses for exec globals
-----------------------------------------

Currently, the ``exec`` statement allows the locals expression (the
second expression after ``in``) to be any mapping type.  The globals
expression (the first expression after ``in``) should be a dict
object, but Python does not currently raise an exception if you use a
dict subclass, though it will not call any overridden methods
correctly.  `Crutcher Dunnavant presented a patch`_ that allows the
globals expression to be a dictionary subclass such that overridden
methods are called properly.  Because it gives Python new
capabilities, if the patch is accepted, it will not become part of
Python until 2.5.

.. _Crutcher Dunnavant presented a patch: http://www.python.org/sf/1402289

Contributing thread:

 - `[PATCH] Fix dictionary subclass semantics when used as global
dictionaries <http://mail.python.org/pipermail/python-dev/2006-January/059625.html>`__

[SJB]

==================
Previous Summaries
==================

 - `When do sets shrink?
<http://mail.python.org/pipermail/python-dev/2006-January/059346.html>`__
 - `Including zlib...
<http://mail.python.org/pipermail/python-dev/2006-January/059372.html>`__
 - `a quit that actually quits
<http://mail.python.org/pipermail/python-dev/2006-January/059384.html>`__
 - `Real time behaviour of Pythons memory management; WAS: RE:
Documentation about Python's GC,python-dev list messages referenced in
Modules/gcmodule.c notreachable anymore
<http://mail.python.org/pipermail/python-dev/2006-January/059408.html>`__
 - `slight inconsistency in svn checkin email subject lines
<http://mail.python.org/pipermail/python-dev/2006-January/059355.html>`__
 - `buildno (Was: [Python-checkins] commit of r41907-
python/trunk/Makefile.pre.in) 
<http://mail.python.org/pipermail/python-dev/2006-January/059419.html>`__

===============
Skipped Threads
===============

 - `Weekly Python Patch/Bug Summary
<http://mail.python.org/pipermail/python-dev/2006-January/059352.html>`__
 - `current test problems
<http://mail.python.org/pipermail/python-dev/2006-January/059360.html>`__
 - `mac memory leaks
<http://mail.python.org/pipermail/python-dev/2006-January/059367.html>`__
 - `API changes
<http://mail.python.org/pipermail/python-dev/2006-January/059374.html>`__
 - `file.next() vs. file.readline()
<http://mail.python.org/pipermail/python-dev/2006-January/059410.html>`__
 - `Sharing between multiple interpreters and restricted mode
<http://mail.python.org/pipermail/python-dev/2006-January/059412.html>`__
 - `TAR problems under Solaris
<http://mail.python.org/pipermail/python-dev/2006-January/059415.html>`__
 - `bsddb broken
<http://mail.python.org/pipermail/python-dev/2006-January/059425.html>`__
 - `Jython and CPython
<http://mail.python.org/pipermail/python-dev/2006-January/059434.html>`__
 - `Compiler warnings for 64-bit portability problems
<http://mail.python.org/pipermail/python-dev/2006-January/059464.html>`__
 - `some interesting readings
<http://mail.python.org/pipermail/python-dev/2006-January/059482.html>`__
 - `Birkenfeld's gone
<http://mail.python.org/pipermail/python-dev/2006-January/059492.html>`__
 - `Python-Dev Digest, Vol 29, Issue 111
<http://mail.python.org/pipermail/python-dev/2006-January/059494.html>`__
 - `PyCon TX 2006: Early-bird registration ends Jan. 15!
<http://mail.python.org/pipermail/python-dev/2006-January/059511.html>`__
 - `test_curses
<http://mail.python.org/pipermail/python-dev/2006-January/059528.html>`__
 - `sudo security hole w/ potential Python connection
<http://mail.python.org/pipermail/python-dev/2006-January/059586.html>`__
 - `Python-Dev Digest, Vol 30, Issue 32
<http://mail.python.org/pipermail/python-dev/2006-January/059587.html>`__
 - `PyThreadState_Delete vs PyThreadState_DeleteCurrent
<http://mail.python.org/pipermail/python-dev/2006-January/059595.html>`__
 - `Hello <http://mail.python.org/pipermail/python-dev/2006-January/059628.html>`__
 - `Limiting the recursion limit
<http://mail.python.org/pipermail/python-dev/2006-January/059652.html>`__
 - `PEP and stdlib
<http://mail.python.org/pipermail/python-dev/2006-January/059694.html>`__
 - `DRAFT: python-dev Summary for 2005-12-01 through 2005-12-15
<http://mail.python.org/pipermail/python-dev/2006-January/059713.html>`__
 - `Unicode print/stdoutput exceptions are not nice
<http://mail.python.org/pipermail/python-dev/2006-January/059715.html>`__
 - `pystate.c changes for Python 2.4.2
<http://mail.python.org/pipermail/python-dev/2006-January/059716.html>`__
 - `DRAFT: python-dev Summary for 2005-12-16 through 2005-12-31
<http://mail.python.org/pipermail/python-dev/2006-January/059717.html>`__
 - `DRAFT: python-dev Summary for 2005-12-16 through2005-12-31
<http://mail.python.org/pipermail/python-dev/2006-January/059726.html>`__
 - `Python icon
<http://mail.python.org/pipermail/python-dev/2006-January/059738.html>`__

From theller at python.net  Thu Jan 26 09:54:51 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 26 Jan 2006 09:54:51 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
Message-ID: <k6cne5d0.fsf@python.net>

Tony Meyer <tony.meyer at gmail.com> writes:

> -------------------------------------
> Adding ctypes to the standard library
> -------------------------------------
>
> Thomas Heller suggested that ctypes be included in core Python
> (starting with 2.5).  The common response was that while ctypes is a
> useful, popular, mature module, it does make it very easy to get a
> core dump, which violates the guideline that if you get a core dump
> it's a bug in Python or in a third party extension or you're doing
> something harebrained.  On the other hand, it was pointed out that the
> dl module suffers from the same problem, and is included without any
> warnings (the documentation was later fixed to include warnings).
>
> Martin v. L?wis suggested making ctypes a dynamically loaded module
> (ctypes.pyd), so administrators could remove it, and
> I could also make it a separate option in the Windows installer, so
> administrators could opt out of installing it.  Everyone seemed happy
> with prominent warnings in the documentation, and so this is how it
> was checked in.

Well, it is *not* yet checked in.  The current state is that ctypes uses
GPL'd tools to build libffi, and those can't be committed into Python SVN.

http://mail.python.org/pipermail/python-dev/2006-January/059937.html

Currently I tend to agree with Martin and drop the idea for now, but
this probably doesn't belong into your summary ;-).

Thomas


From thomas at xs4all.net  Thu Jan 26 10:08:22 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 26 Jan 2006 10:08:22 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <k6cne5d0.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net>
Message-ID: <20060126090822.GF18916@xs4all.nl>

On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote:

> The current state is that ctypes uses GPL'd tools to build libffi, and
> those can't be committed into Python SVN.

> http://mail.python.org/pipermail/python-dev/2006-January/059937.html

But http://mail.python.org/pipermail/python-dev/2006-January/059938.html
was never responded to. And licenses are fluid, it may be a piece of cake to
get one of those 'tools' un-GPL'ed, even if they are.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From hoffman at ebi.ac.uk  Thu Jan 26 10:26:27 2006
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Thu, 26 Jan 2006 09:26:27 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <20060126004113.GD18916@xs4all.nl>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<20060126004113.GD18916@xs4all.nl>
Message-ID: <Pine.LNX.4.64.0601260915100.29964@qnzvnan.rov.np.hx>

[Thomas Wouters]

>> This issue has been brought up before when people were discussing the
>> path module. I think the consensus is that, while the inheritance
>> isn't pure, practicality beats purity. It would require to big changes
>> to Python and would break to much existing code to not extend string.
>
> This is my only problem with the PEP. It's all very nice that subclassing
> from string makes it easier not to break things, but subclassing implies a
> certain relationship.

This is the soul of arguing for purity's sake when practicality would
dictate something else.

If you remove the basestring superclass, then you remove the ability
to use path objects as a drop-in replacement for any path string right
now.  You will either have to use str(pathobj) or carefully check that
the function/framework you are passing the path to does not use
isinstance() or any of the string methods that are now gone.

http://groups.google.com/group/comp.lang.python/browse_thread/thread/1f5bcb67c4c73f15
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From gjc at inescporto.pt  Thu Jan 26 13:03:28 2006
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Thu, 26 Jan 2006 12:03:28 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D8518B.70900@colorstudy.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<1138235459.6668.14.camel@localhost.localdomain>
	<43D8518B.70900@colorstudy.com>
Message-ID: <1138277008.8256.26.camel@localhost>

On Wed, 2006-01-25 at 22:35 -0600, Ian Bicking wrote:
> Gustavo J. A. M. Carneiro wrote:
> >   On a slightly different subject, regarding path / path, I think it
> > feels much more natural path + path.  Path.join is really just a string
> > concatenation, except that it adds a path separator in the middle if
> > necessary, if I'm not mistaken.
> 
> No, it isn't, which maybe is why / is bad.  os.path.join(a, b) basically 
> returns the path as though b is interpreted to be relative to a.  I.e., 

> os.path.join('/foo', '/bar') == '/bar'.  Not much like concatenation at 
> all.

  Really?  This is not like the unix command line.  At least in
Linux, /foo/bar is the same as /foo//bar and /foo///bar, etc.  But I
admit it can be useful in some cases.

>   Plus string concatenation is quite useful with paths, e.g., to add 
> an extension.

  I see your point.  Although perhaps adding an extension to a file
should be the exception and not the rule, since adding extensions is
rarely used compared to joining paths?  Maybe Path.add_extension() ?

  BTW, regarding Path subclassing basestr, there exists "prior art" for
this Path thing in SCons.  In SCons, we (users, I'm not a scons dev)
have to constantly deal with Node instances.  Most scons functions that
accept Nodes also accept strings, but a Node is not a string.  When
calling an os function with Nodes, one has to convert it to string
first, using str().  IMHO, having to decorate Node instances with str()
sometimes is no big deal, really.  And, given enough time, perhaps most
of the python standard library could be enhanced to accept Path
instances in addition to C strings.

> If a URI class implemented the same methods, it would be something of a 
> question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz' 
> (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does). 
> I assume it would be be the latter, and urljoin would be a different 
> method, maybe something novel like "urljoin".

  I honestly don't understand the usefulness of join('/foo/bar', 'baz')
ever returning '/foo/baz' instead of '/foo/bar/baz'.  How would the
former be of any use?  If it has no use, then please don't complicate
things even more :)

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic.


From p.f.moore at gmail.com  Thu Jan 26 13:51:00 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 26 Jan 2006 12:51:00 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <20060126004113.GD18916@xs4all.nl>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<20060126004113.GD18916@xs4all.nl>
Message-ID: <79990c6b0601260451q3015bbf9j5d3d539234c49f07@mail.gmail.com>

On 1/26/06, Thomas Wouters <thomas at xs4all.net> wrote:
> On Wed, Jan 25, 2006 at 09:37:04PM +0100, BJ?rn Lindqvist wrote:
>
> > Inheritance from string (Jason)
>
> > This issue has been brought up before when people were discussing the
> > path module. I think the consensus is that, while the inheritance
> > isn't pure, practicality beats purity. It would require to big changes
> > to Python and would break to much existing code to not extend string.

I don't think there is consensus at all. I've seen plenty of
arguments, either directly against inheritance from string, or against
features which exist *because* of the inheritance (e.g., we can't use
join() because it's a string method).

> This is my only problem with the PEP. It's all very nice that subclassing
> from string makes it easier not to break things, but subclassing implies a
> certain relationship. That relationship doesn't exist, in this case. Having
> the string subclass behave differently than strings (consider the __iter__
> and join methods) is a bad idea. I can dish up a half dozen contrived
> problem cases, but the main reason I don't like it is that it feels wrong.

Agreed. Path objects don't feel like strings to me, either. It's
certainly *arguable* that "paths are strings" in some ideal sense, but
in a very practical sense they are not. Operations like split, join,
justification, trimming, all of which are part of the Python string
type (and hence constitute part of what it means to "be a string" in
Python) do not have any sensible meaning on paths.

The only justification for making Path a string subtype seems to me to
avoid a few conversions - open(path) rather than open(str(path)), for
example. I'd rather have to explicitly convert, to be honest. (But I'd
happily accept changes to open etc to take path objects directly).

> If the reason to subclass string is that it's too hard to make an object
> 'string-like' at a low enough level for the C API, I suggest fixing that,
> instead. If that means Path has to wait until Python 2.6, then that's too
> bad. The inability to feed C functions/types open() non-string objects has
> troubled me before, and though I haven't invested a lot of time in it, I
> don't quite understand why it isn't possible. Fixing it in a
> backward-compatible manner may require a new __hook__, although I think
> using __str__ or __unicode__ shouldn't be too problematic.
>
> Even if fixing the "%es"/"%et" etc formats to the arg-parsing methods is
> infeasible, I would prefer a new format code for 'string-alike as C string'
> over the unpythonic inappropriate subclassing. Although, even if the
> subclassing was completely appropriate, I would much rather improve builtin
> support for ducktyping than showcase subclassing ;)

Adaptation (PEP 246) would let paths be *adaptable* to strings,
without *being* strings... :-)

Paul.

From theller at python.net  Thu Jan 26 13:29:55 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 26 Jan 2006 13:29:55 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
Message-ID: <ek2vdvek.fsf@python.net>

Thomas Wouters <thomas at xs4all.net> writes:

> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote:
>
>> The current state is that ctypes uses GPL'd tools to build libffi, and
>> those can't be committed into Python SVN.
>
>> http://mail.python.org/pipermail/python-dev/2006-January/059937.html
>
> But http://mail.python.org/pipermail/python-dev/2006-January/059938.html
> was never responded to.

Lack of time - sorry.

> And licenses are fluid, it may be a piece of cake to
> get one of those 'tools' un-GPL'ed, even if they are.

I wouldn't even know whom to ask.

Thomas


From p.f.moore at gmail.com  Thu Jan 26 14:15:08 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Thu, 26 Jan 2006 13:15:08 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
Message-ID: <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>

On 1/25/06, BJ?rn Lindqvist <bjourne at gmail.com> wrote:
> My comments on the issues. It was easier this way than trying to reply
> on every message individually.
>
> Inheritance from string (Jason)
>
> This issue has been brought up before when people were discussing the
> path module. I think the consensus is that, while the inheritance
> isn't pure, practicality beats purity. It would require to big changes
> to Python and would break to much existing code to not extend string.
>
> I'll add this to Resolved Issues if nobody minds.

I mind (see my previous post)...

> Remove __div__ (Ian, Jason, Michael, Oleg)
>
> This is one of those where everyone (me too) says "I don't care either
> way." If that is so, then I see no reason to change it unless someone
> can show a scenario in which it hurts readability. Plus, a few people
> have said that they like the shortcut.

Hardly. I've seen some pretty strong arguments (both for and against)
- not what I'd describe as everyone saying they don't care.

FWIW, I find the / operator ugly. Also, multiple concatenation (path /
"a" / "b" / "c") results in building lots of intermediates, where
path.join("a", "b", "c") need not. Arguing that you can't reuse string
methods is bogus, IMHO, as the requirement to subclass from string is
far from clear.

Actually, reading that, I'd suggest:
- an append() method to add extra components to a path
- a multi-arg Path() constructor

So, we have
- path.append("a", "b", "c")
- Path("C:", "Windows", "System32")

Quick question - how do Path objects constructed from relative paths
behave? Are there such things as relative path objects? Consider

    p1 = Path("a")
    p2 = Path("b")

Is p1.append(p2) (or p1/p2) legal? What does it mean? I'd have to
assume it's the same as Path("a", "b"), but I'm not sure I like
that... What about Path("/a").append(Path("/b")) ???

Also note that my example Path("C:", "Windows", "System32") above is
an *absolute* path on Windows. But a relative (albeit stupidly-named
:-)) path on Unix. How would that be handled?

Not that os.path gets it perfect:

Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>> import os
>>> os.path.join("C:", "Windows", "System32")
'C:Windows\\System32'
>>> os.path.join(".", os.path.join("C:", "Windows", "System32"))
'.\\C:Windows\\System32'
>>>

But os.path manipulates strings representing pathnames (and I can
forgive oddities like this by noting that some rules about pathnames
are pretty subtle...). I'd have higher standards for a dedicated Path
object.

Arguably, Path objects should always maintain an absolute path - there
should be no such thing as a relative Path. So you would have

    str(Path("whatever")) === os.path.abspath("whatever")

It also allows Path("C:", "Windows") to do different things on Windows
and Unix (absolute on Windows, relative to os.curdir on Unix).

This would imply that Path("a", a_path) or a_path.append(another_path)
is an error. And of course, for this to work, Path objects *can't* be
a subclass of string... :-)

Paul.

From thomas at xs4all.net  Thu Jan 26 14:26:41 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 26 Jan 2006 14:26:41 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <Pine.LNX.4.64.0601260915100.29964@qnzvnan.rov.np.hx>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<20060126004113.GD18916@xs4all.nl>
	<Pine.LNX.4.64.0601260915100.29964@qnzvnan.rov.np.hx>
Message-ID: <20060126132641.GG18916@xs4all.nl>

On Thu, Jan 26, 2006 at 09:26:27AM +0000, Michael Hoffman wrote:
> [Thomas Wouters]
> > [Subclassing string] is my only problem with the PEP. It's all very nice
> > that subclassing from string makes it easier not to break things, but
> > subclassing implies a certain relationship.

> This is the soul of arguing for purity's sake when practicality would
> dictate something else.

If we're going to argue that point, I don't believe this is the practicality
that the 'zen of python' talks about. Practicality is the argument for
'print', and for requiring the ':' before suites, and for some of the
special cases in the Python syntax and module behaviour. It isn't about the
implementation. The argument to subclass string is, as far as I can tell,
only the ease of implementation and the ease of transition. Nothing in the
old thread convinced me otherwise, either. I've never seen Guido go for an
implementation-practical solution just because he couldn't be arsed to do
the work to get a conceptually-practical solution. And subclassing string
isn't conceptually-practical at all.

> If you remove the basestring superclass, then you remove the ability
> to use path objects as a drop-in replacement for any path string right
> now.  You will either have to use str(pathobj) or carefully check that
> the function/framework you are passing the path to does not use
> isinstance() or any of the string methods that are now gone.

More to the point, you will have to carefully check whether the
function/framework will use the Path object in a way the Path object can
handle. There's already discussion about whether certain methods should be
'disabled', in Path objects, or whether they should be doing something
conceptually different. And subclassing string is not going to solve all
these issues anyway. Even in the standard library there's a scary amount of
'type(x) == type("")' checks, most noteably in exactly the type of function
that takes both a filename and a file-like object.

I don't believe going out of your way to cater to these kind of typechecks
is the right way to solve the problem. I believe the problem is more that
there isn't a unified, obvious, 'one-way' to do the typechecks -- or rather,
to avoid the typechecks. Parts of Python do duck-typing and nothing else;
this usually works fine, and is quite flexible. Other parts, especially (but
not exclusively) the parts written in C and Python code that directly deals
with those C parts, need to care more about actual types. Python offers no
standard or accepted or even vaguely preferred way of doing that. The
standard library doesn't even do this uniformly, so how can we expect anyone
to ever get this 'right'? Especially since the 'right' way depends on what
you want to do with the result. This problem pops up most visibly in
treating-as-int (indexing) and in treating-as-string (open() and friends),
but I'm sure there are more.

You could see this as an argument for formal interfaces. Perhaps it is. It
could also be an argument for just a few more __hooks__. Guido already
suggested __index__, which would mean 'treat me as this int for indexing and
other such operations' -- which is not the same thing as __int__. Likewise,
treating-as-string would require a different hook than __str__ or __repr__,
and should explicitly not be defined for most objects, only those that
actually *want* to be treated as a string. And there should probably be
more.

Or, maybe, we could do it all in one __hook__. Say, __equivalent__.
obj.__equivalent__(cls) would return the instance of 'cls' that represents
'obj', or raise an appropriate error. I say 'instance of 'cls'', which means
it could be an instance of a subclass, too. It isn't duck-typing and it
doesn't replace interfaces, not by far, since it actually returns a new type
(and thus can't be a proxy-type; I don't think changing the equivalent
should change the original, anyway.) Rather, it'd be the standard way to
glue duck-typing (and formal interfaces, if you use those) with strict
typing mechanisms (like C's.)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From hoffman at ebi.ac.uk  Thu Jan 26 15:20:04 2006
From: hoffman at ebi.ac.uk (Michael Hoffman)
Date: Thu, 26 Jan 2006 14:20:04 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <20060126132641.GG18916@xs4all.nl>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<20060126004113.GD18916@xs4all.nl>
	<Pine.LNX.4.64.0601260915100.29964@qnzvnan.rov.np.hx>
	<20060126132641.GG18916@xs4all.nl>
Message-ID: <Pine.LNX.4.64.0601261403440.30122@qnzvnan.rov.np.hx>

[Thomas Wouters]
>>> [Subclassing string] is my only problem with the PEP. It's all very nice
>>> that subclassing from string makes it easier not to break things, but
>>> subclassing implies a certain relationship.

[Michael Hoffman]
>> This is the soul of arguing for purity's sake when practicality would
>> dictate something else.

[Thomas Wouters]
> If we're going to argue that point, I don't believe this is the practicality
> that the 'zen of python' talks about. Practicality is the argument for
> 'print', and for requiring the ':' before suites, and for some of the
> special cases in the Python syntax and module behaviour. It isn't about the
> implementation. The argument to subclass string is, as far as I can tell,
> only the ease of implementation and the ease of transition. Nothing in the
> old thread convinced me otherwise, either. I've never seen Guido go for an
> implementation-practical solution just because he couldn't be arsed to do
> the work to get a conceptually-practical solution. And subclassing string
> isn't conceptually-practical at all.

I don't understand what "conceptually-practical" is or how it differs
from "conceptually pure" which is what it seems that you're looking
for. It's not hard to give Path a has-a relationship to basestring
instead of an is-a relationship, so it really doesn't save much in
terms of implementation.

>> If you remove the basestring superclass, then you remove the ability
>> to use path objects as a drop-in replacement for any path string right
>> now.  You will either have to use str(pathobj) or carefully check that
>> the function/framework you are passing the path to does not use
>> isinstance() or any of the string methods that are now gone.
>
> More to the point, you will have to carefully check whether the
> function/framework will use the Path object in a way the Path object can
> handle. There's already discussion about whether certain methods should be
> 'disabled', in Path objects, or whether they should be doing something
> conceptually different.

Yes, and I think all of this discussion is focused on conceptual
purity and misses one of the big wins of the Path module for current
users--it can be trivially used anywhere where a str is expected
today. If you're going to start deciding that certain str methods
should be disabled for some reason, then it shouldn't be a str
subclass, because it will no longer behave like string-plus.

In previous discussions, string methods were identified that one
programmer thought would not be useful on a path, but others
disagreed. Disabling methods serves no useful purpose, except to
shorten dir().

I've been using path.py for some time, and I can tell you that it
would be a lot less useful if it no longer behaved like string-plus.
-- 
Michael Hoffman <hoffman at ebi.ac.uk>
European Bioinformatics Institute


From ronaldoussoren at mac.com  Thu Jan 26 15:22:17 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 26 Jan 2006 15:22:17 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <ek2vdvek.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
Message-ID: <864123BC-0DCD-4AD3-8BDA-EC9625090FF4@mac.com>


On 26-jan-2006, at 13:29, Thomas Heller wrote:

> Thomas Wouters <thomas at xs4all.net> writes:
>
>> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote:
>>
>>> The current state is that ctypes uses GPL'd tools to build  
>>> libffi, and
>>> those can't be committed into Python SVN.
>>
>>> http://mail.python.org/pipermail/python-dev/2006-January/059937.html
>>
>> But http://mail.python.org/pipermail/python-dev/2006-January/ 
>> 059938.html
>> was never responded to.
>
> Lack of time - sorry.
>
>> And licenses are fluid, it may be a piece of cake to
>> get one of those 'tools' un-GPL'ed, even if they are.
>
> I wouldn't even know whom to ask.

It shouldn't be too hard to use Python's main configure script to  
calculate
the information necessary to build libffi. A lot of it is already  
calculated
anyway (sizeof various type, endianness), some can be hardcoded  
(FFI_NO_RAW_API).

In PyObjC I just compile the files I need from my setup.py. But I  
have an easy task,
I just need to support two CPU architectures on one OS.

Ronald

>
> Thomas
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060126/8f8b38de/attachment-0001.bin 

From stefan.rank at ofai.at  Thu Jan 26 15:47:05 2006
From: stefan.rank at ofai.at (Stefan Rank)
Date: Thu, 26 Jan 2006 15:47:05 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>
Message-ID: <43D8E0E9.9080202@ofai.at>

on 26.01.2006 14:15 Paul Moore said the following:
[snip]
> 
> Also note that my example Path("C:", "Windows", "System32") above is
> an *absolute* path on Windows. But a relative (albeit stupidly-named
> :-)) path on Unix. How would that be handled?

wrong, Path("C:", "Windows", "System32") is a relative path on windows.
see below.

> Not that os.path gets it perfect:
> 
> Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
> Type "help", "copyright", "credits" or "license" for more information.
>>>> import os
>>>> os.path.join("C:", "Windows", "System32")
> 'C:Windows\\System32'
>>>> os.path.join(".", os.path.join("C:", "Windows", "System32"))
> '.\\C:Windows\\System32'
> 

this is misleading. observe::

  In [1]: import os

  In [2]: os.path.join(".", os.path.join("C:", "Windows", "System32"))
  Out[2]: '.\\C:Windows\\System32'

but::

  In [3]: os.path.join(".", os.path.join("C:\\", "Windows", "System32"))
  Out[3]: 'C:\\Windows\\System32'


The second example uses an absolute path as second argument, and as 
os.path.join should do, the first argument is discarded.

The first case is arguably a bug, since, on windows, C:Windows\System32 
is a path relative to the *current directory on disk C:*
If the cwd on C: would be C:\temp then C:Windows\System32 would point to 
C:\temp\Windows\System32

The problem is that Windows has a cwd per partition...
(I cannot even guess why ;-)

For the sake of illustration, the following is a WinXP cmd session::

  Microsoft Windows XP [Version 5.1.2600]
  (C) Copyright 1985-2001 Microsoft Corp.

  C:\temp>d:

  D:\>cd HOME

  D:\HOME>c:

  C:\temp>d:

  D:\HOME>c:

  C:\temp>cd d:bin

  C:\temp>d:

  D:\HOME\bin>


[snip]
> 
> Arguably, Path objects should always maintain an absolute path - there
> should be no such thing as a relative Path. So you would have

you realise that one might need and/or want to represent a relative path?


From fredrik at pythonware.com  Thu Jan 26 16:17:32 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 26 Jan 2006 16:17:32 +0100
Subject: [Python-Dev] The path module PEP
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com><43D6BA85.8070007@colorstudy.com><bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com><740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com><1138235459.6668.14.camel@localhost.localdomain><43D8518B.70900@colorstudy.com>
	<1138277008.8256.26.camel@localhost>
Message-ID: <drap6e$6d0$1@sea.gmane.org>

Gustavo J. A. M. Carneiro wrote:

> > If a URI class implemented the same methods, it would be something of a
> > question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz'
> > (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does).
> > I assume it would be be the latter, and urljoin would be a different
> > method, maybe something novel like "urljoin".
>
>   I honestly don't understand the usefulness of join('/foo/bar', 'baz')
> ever returning '/foo/baz' instead of '/foo/bar/baz'.  How would the
> former be of any use?

it's how URL:s are joined, as noted in the paragraph you replied to

(a "baz" link on the page "/foo/bar" refers to "/foo/baz", not "/foo/bar/baz")

</F>




From theller at python.net  Thu Jan 26 16:33:08 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 26 Jan 2006 16:33:08 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net>
	<20060126090822.GF18916@xs4all.nl> <ek2vdvek.fsf@python.net>
	<864123BC-0DCD-4AD3-8BDA-EC9625090FF4@mac.com>
Message-ID: <d5ifatsb.fsf@python.net>

Ronald Oussoren <ronaldoussoren at mac.com> writes:

> On 26-jan-2006, at 13:29, Thomas Heller wrote:
>
>> Thomas Wouters <thomas at xs4all.net> writes:
>>
>>> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote:
>>>
>>>> The current state is that ctypes uses GPL'd tools to build libffi,
>>>> and those can't be committed into Python SVN.
>>>
>>>> http://mail.python.org/pipermail/python-dev/2006-January/059937.html
>
> It shouldn't be too hard to use Python's main configure script to
> calculate the information necessary to build libffi. A lot of it is
> already calculated anyway (sizeof various type, endianness), some can
> be hardcoded (FFI_NO_RAW_API).
>
> In PyObjC I just compile the files I need from my setup.py. But I have
> an easy task, I just need to support two CPU architectures on one OS.

Thanks for the encouragement - Martin suggested a similar approach.

>From my understanding (which goes not very far) the configuration does
two things: determine the set of source files that needs to go in
depending on the cpu architecture, and to determine some information and
make them available in #defines.  I have to check if this is possible
without patching the libffi sources themselves. I guess I could look
into the PyObjC setuop script.

Personally I only have access to machines running windows, linux (x86
and x86_64), and OS X (plus possibly a Mac running ubuntu), so I could
only do it for those.  Maybe support for other architectures can be
added by volunteers?

Besides:  James Y Knight seems to be correct that all the scripts needed
to build libffi seems to have this special exception from the GPL.

Thomas


From theller at python.net  Thu Jan 26 16:43:58 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 26 Jan 2006 16:43:58 +0100
Subject: [Python-Dev] Include ctypes into core Python?
References: <u0cbyfa1.fsf@python.net>
	<ca471dc20601101314k173b2250q940fb3ed064bb6a2@mail.gmail.com>
	<43C58200.20803@v.loewis.de> <psms7b34.fsf@python.net>
	<ca471dc20601160748o4a965b6q14048f643cbc457f@mail.gmail.com>
	<43CC17FB.5020904@v.loewis.de> <ek33ud1j.fsf@python.net>
	<03C7403A-6B16-4727-B2C6-5FE133E69734@fuhm.net>
Message-ID: <7j8nata9.fsf@python.net>

James Y Knight <foom at fuhm.net> writes:

> On Jan 19, 2006, at 4:24 PM, Thomas Heller wrote:
>
>>
>> Several of these files are licensed under GPL:
>>
>> aclocal.m4 config-ml.in config.guess config.sub depcomp ltcf-c.sh
>> ltconfig missing
>>
>
> Are you sure? The copies of aclocal.m4 and config-ml.in both disagree
> with you. aclocal seems to have a completely liberal license, and
> config-ml has a "whatever the license of the program it's building"
> license.
>

It seems you are right:

config-ml.in: GPL with special exception.
config.guess: GPL with special exception.
config.sub:   GPL with special exception.
configure: no limitation
depcomp:      GPL with special exception.
install-sh:   X-11 license
ltcf-c.sh:    GPL with special exception.
ltconfig:     GPL with special exception.
ltmain.sh:    GPL with special exception.

aclocal.m4:   see below

Is aclocal.m4 an exception? It has several copyright notices.  The first
one gives unlimited permissions to copy and/or distribute, but sections
after that have no exception clause.  I'm unsure what this means.

The files that ctypes uses are in CVS here:

http://cvs.sourceforge.net/viewcvs.py/ctypes/ctypes/source/gcc/libffi/?only_with_tag=branch_1_0

Thomas


From bingham at cenix-bioscience.com  Thu Jan 26 16:34:32 2006
From: bingham at cenix-bioscience.com (Aaron Bingham)
Date: Thu, 26 Jan 2006 16:34:32 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D8E0E9.9080202@ofai.at>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	<79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>
	<43D8E0E9.9080202@ofai.at>
Message-ID: <43D8EC08.5040702@cenix-bioscience.com>

Stefan Rank wrote:

>on 26.01.2006 14:15 Paul Moore said the following:
>[snip]
>  
>
>>Arguably, Path objects should always maintain an absolute path - there
>>should be no such thing as a relative Path. So you would have
>>    
>>
>
>you realise that one might need and/or want to represent a relative path?
>  
>
Of course, but it seems to me a relative path is a different type from 
an absolute path, in the same way that a timedelta is different from a 
datetime.

For example:

 * You can't open a relative path without reference to some absolute 
path (possibly the cwd).
 * You can't join two absolute paths, but you can join a relative path 
to another relative path, or to an absolute path.

Cheers,

Aaron

--------------------------------------------------------------------
Aaron Bingham
Senior Software Engineer
Cenix BioScience GmbH
--------------------------------------------------------------------



From bjourne at gmail.com  Thu Jan 26 17:16:43 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Thu, 26 Jan 2006 17:16:43 +0100
Subject: [Python-Dev] Path inherits from string
Message-ID: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>

This seems to be the only really major issue with the PEP. Everything
else is negotiable, IMHO. But the string inheritance seem to be such a
critical issue it deserves its own thread. I have tried to address all
criticism of it here:

Really, it is the same arguments that have been rehashed over and over
again. I also think that they have been suitably countered already -
Practicality beats Purity. The fact that you can plug Path into
already existing code is a major benefit. It makes it possible to use
and integrate the Path class *now* and not wait until the mythical
Python 3000 when every single flaw in Python have been fixed. Remember
that the Path module have existed for three years in the wild and is
extra-ordinarily well-liked. If the fact that Path inherits from
string is such a big deal, then *why* is the path module so hugely
popular?? :)

The inheritance is a design trade-off. An implementation detail, an
insignificant wart in an otherwise very practical design. Over the
years I have used the path module this wart has never ever hurt
me. Not even once. But if someone can construct a Path class that does
*not* inherit from string and that works equally well as Path then
ofcourse, that is preferable. However, because of this (and similar
issues) I don't think it is possible:

    class Foo:
        def __str__(self):
            return "foo"
    open(Foo())
    TypeError: coercing to Unicode: need string or buffer, instance found

However, I might be wrong because according to [1] it should work. And
having to wrap the Path object in str() (open(str(somepath))) each and
every time the called function expects a string is not a practical
solution.

Also note that because compatibility with existing code is important,
Path can not override or change any methods at all from str - Liskov
Substitution Principle. As Path is, it only violates LSP once:

    >>> type(Path()) == type('')
    False

And there is absolutely nothing that can be done about that. As far as
I can tell, the string inheritance is either livable with or is a
showstopper. If it is the latter, then:

    1. Someone has to make the required modifications to the Python
       core.
    2. Create a Path class (or adapt the existing one so) that does
       not inherit from string.
    3. Release it and wait a few years hoping for it to gain
       widespread acceptance in the Python community.
    4. Make a PEP (or adapt this PEP) that gets accepted.

This scenario makes me sad because it basically means that there will
never be a Path module in Python, atleast not during my lifetime. :(
Why? Because Jason Orendorff's path module already exists and
works. But I may be overestimating the problems and doing what Jason
suggests in [2] may be enough. Maybe someone who knows what he is
talking about can expand on it further?

1 http://mail.python.org/pipermail/python-dev/2001-August/016674.html
2 http://mail.python.org/pipermail/python-dev/2006-January/060030.html

--
mvh Bj?rn

From fredrik at pythonware.com  Thu Jan 26 17:35:05 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 26 Jan 2006 17:35:05 +0100
Subject: [Python-Dev] Path inherits from string
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
Message-ID: <dratnq$lbp$1@sea.gmane.org>

BJörn Lindqvist wrote:

> However, I might be wrong because according to [1] it should work. And
> having to wrap the Path object in str() (open(str(somepath))) each and
> every time the called function expects a string is not a practical
> solution.

in Python, the usual way to access an attribute of an object is to
access the attribute; e.g.

    f = open(p.name)

</F>




From mal at egenix.com  Thu Jan 26 17:51:48 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 26 Jan 2006 17:51:48 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
Message-ID: <43D8FE24.1030500@egenix.com>

BJ?rn Lindqvist wrote:
> This seems to be the only really major issue with the PEP. Everything
> else is negotiable, IMHO. But the string inheritance seem to be such a
> critical issue it deserves its own thread. I have tried to address all
> criticism of it here:

I don't see why this is critical for the success of the Path
object. I agree with Thomas that interfaces should be made
compatible to Path object.

Please note that inheritance from string will cause the C type
checks of the form PyString_Check(obj) to return true.
C code will then assume that it has an object which is
compatible to string C API which instances aren't.

If the C code then uses the C API string macros, you
get segfaults - and lot's of old code does, since there
was no way to inherit from a string type at the time.

In fact, you're lucky that open() doesn't give you segfaults,
since the code used to fetch the string argument does exactly
that...

>...
>
> And there is absolutely nothing that can be done about that. As far as
> I can tell, the string inheritance is either livable with or is a
> showstopper. If it is the latter, then:
> 
>     1. Someone has to make the required modifications to the Python
>        core.

Right.

Plus convert a few PyString_Check()s to PyString_CheckExact()...

>     2. Create a Path class (or adapt the existing one so) that does
>        not inherit from string.
>     3. Release it and wait a few years hoping for it to gain
>        widespread acceptance in the Python community.
>     4. Make a PEP (or adapt this PEP) that gets accepted.
> 
> This scenario makes me sad because it basically means that there will
> never be a Path module in Python, atleast not during my lifetime. :(

Why not ? We've added Unicode support to at least some
file I/O APIs - adding support for instances which
support the string interface shouldn't be all that
difficult :-)

BTW, if you're fine with this API:

class File:
    def __unicode__(self):
        return u"test.txt"

then the required change is minimal: we'd just need to
use PyObject_Unicode() in getargs.c:837 and you should
be set.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 26 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From stefan.rank at ofai.at  Thu Jan 26 17:55:24 2006
From: stefan.rank at ofai.at (Stefan Rank)
Date: Thu, 26 Jan 2006 17:55:24 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D8EC08.5040702@cenix-bioscience.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	<79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>	<43D8E0E9.9080202@ofai.at>
	<43D8EC08.5040702@cenix-bioscience.com>
Message-ID: <43D8FEFC.40708@ofai.at>

on 26.01.2006 16:34 Aaron Bingham said the following:
> Stefan Rank wrote:
>> on 26.01.2006 14:15 Paul Moore said the following:
>> [snip]
>>
>>> Arguably, Path objects should always maintain an absolute path - there
>>> should be no such thing as a relative Path.
>>>
>> you realise that one might need and/or want to represent a relative path?
>>  
> Of course, but it seems to me a relative path is a different type from 
> an absolute path, in the same way that a timedelta is different from a 
> datetime.
> 
> For example:
> 
>  * You can't open a relative path without reference to some absolute 
> path (possibly the cwd).
>  * You can't join two absolute paths, but you can join a relative path 
> to another relative path, or to an absolute path.

I think the datetime/timedelta analogy is not bad:
A datetime is actually also a time delta - relative to some given 
start-time, internally this is normally the "epoch". For human-readable 
representations it is the birth of your chosen deity, or the creation of 
the universe, ...

The start time for datetime is implicit.
Python has chosen some absolute reference.

For paths that absolute reference (the root) is very much context 
dependent (platform dependent).
You *can* open a relative path - because processes always have an 
implicit cwd as part of their context.

But you might also want to represent paths that are relative on another 
host than the one your program is running on.

I don't think it makes sense to design a Path class that captures the 
abstract concept of a path - because of the semantic differences between 
unix paths, windows paths, URL paths, ...

I see the Path object as a special kind of string, that has helpful 
methods for relating it to the workings of filesystems in general and 
the local filesystem in particular. But it is still just an ordinary 
string that happens to be used as a special kind of address.

I try to separate the concept of the 'object in the filesystem' (which 
is the domain of Python's file objects) from the 'hierarchical address 
to an object' (which is what the Path objects make easier).

(Java has these two conflated in one.)

So, to get to the point, a `file` is a thing that should always have an 
absolute path. (and it does. it should probably grow a .path attribute 
in addition to .name ? This could return None for files without paths.)
A Path should be able to contain absolute, relative, valid, as well as 
invalid (on a given OS) paths.

In case that future systems manage to reduce the importance of the 
legacy crutch that is the hierarchical filesystem ;-)
we might get query-like addresses:
'/tmp/[author=me;type=text/html]'
and Path might grow to deal with it.

Sorry I digress.

+1 on Path as an enhanced string that bundles file-system-address 
related methods.

stefan


From ianb at colorstudy.com  Thu Jan 26 18:00:04 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Thu, 26 Jan 2006 11:00:04 -0600
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <dratnq$lbp$1@sea.gmane.org>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<dratnq$lbp$1@sea.gmane.org>
Message-ID: <43D90014.10807@colorstudy.com>

Fredrik Lundh wrote:
>>However, I might be wrong because according to [1] it should work. And
>>having to wrap the Path object in str() (open(str(somepath))) each and
>>every time the called function expects a string is not a practical
>>solution.
> 
> 
> in Python, the usual way to access an attribute of an object is to
> access the attribute; e.g.
> 
>     f = open(p.name)

You mean "f = open(Path(p).name)", because it is likely that people will 
  also have to accept strings for the nearterm (and probably longeterm) 
future.  And the error message without will be inscrutable (and will 
still be inscrutable in many cases when you try to access other methods, 
sadly).  And currently .name is taken for something else in the API. 
And the string path is not really an attribute because the string path 
*is* the object, it is not *part* of the object.

OTOH, str(path) will break unicode filenames.  And unicode() breaks 
anything that simply desires to pass data through without effecting its 
encoding.

An open method on paths simplifies many of these issues, but doesn't do 
anything for passing a path to legacy code.  Changing open() and all the 
functions that Path replaces (e.g., os.path.join) to accept Path objects 
may resolve issues with a substantial portion of code.  But any code 
that does a typecheck on arguments will be broken -- which in the case 
of paths is quite common since many functions take both filename and 
file object arguments.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From foom at fuhm.net  Thu Jan 26 18:04:06 2006
From: foom at fuhm.net (James Y Knight)
Date: Thu, 26 Jan 2006 12:04:06 -0500
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <ek2vdvek.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
Message-ID: <BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>


On Jan 26, 2006, at 7:29 AM, Thomas Heller wrote:
>> And licenses are fluid, it may be a piece of cake to
>> get one of those 'tools' un-GPL'ed, even if they are.
>
> I wouldn't even know whom to ask.


On Jan 26, 2006, at 9:22 AM, Ronald Oussoren wrote:
> It shouldn't be too hard to use Python's main configure script to  
> calculate
> the information necessary to build libffi. A lot of it is already  
> calculated
> anyway (sizeof various type, endianness), some can be hardcoded  
> (FFI_NO_RAW_API).
>
> In PyObjC I just compile the files I need from my setup.py. But I  
> have an easy task,
> I just need to support two CPU architectures on one OS.

Let's please be sure the license isn't fine _as is_ before thinking  
about asking people to change them or rewriting the build system! I  
did not look at *all* the files listed as being GPL, but the first  
two in the list were not.

James


From gjc at inescporto.pt  Thu Jan 26 18:22:58 2006
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Thu, 26 Jan 2006 17:22:58 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <drap6e$6d0$1@sea.gmane.org>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<1138235459.6668.14.camel@localhost.localdomain>
	<43D8518B.70900@colorstudy.com> <1138277008.8256.26.camel@localhost>
	<drap6e$6d0$1@sea.gmane.org>
Message-ID: <1138296178.8256.37.camel@localhost>

On Thu, 2006-01-26 at 16:17 +0100, Fredrik Lundh wrote:
> Gustavo J. A. M. Carneiro wrote:
> 
> > > If a URI class implemented the same methods, it would be something of a
> > > question whether uri.joinpath('/foo/bar', 'baz') would return '/foo/baz'
> > > (and urlparse.urljoin would) or '/foo/bar/baz' (as os.path.join does).
> > > I assume it would be be the latter, and urljoin would be a different
> > > method, maybe something novel like "urljoin".
> >
> >   I honestly don't understand the usefulness of join('/foo/bar', 'baz')
> > ever returning '/foo/baz' instead of '/foo/bar/baz'.  How would the
> > former be of any use?
> 
> it's how URL:s are joined, as noted in the paragraph you replied to
> 
> (a "baz" link on the page "/foo/bar" refers to "/foo/baz", not "/foo/bar/baz")

  That's not how I see it.  A web browser, in order to resolve the link
'baz' in the page '/foo/bar', should do:

	join(basename('/foo/bar'), 'baz')
	== join('/foo', 'baz')
	== '/foo/baz'.

  Regards.

-- 
Gustavo J. A. M. Carneiro
<gjc at inescporto.pt> <gustavo at users.sourceforge.net>
The universe is always one step beyond logic.


From martin at v.loewis.de  Thu Jan 26 19:41:36 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jan 2006 19:41:36 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
Message-ID: <43D917E0.4050606@v.loewis.de>

BJ?rn Lindqvist wrote:
> This seems to be the only really major issue with the PEP. 

I'd like to call for order here. What PEP? I can't find it
on

http://www.python.org/peps/

Also, if this is a major issue, then the PEP owner should not
start a thread discussing it, but instead edit the PEP should
summarize the discussion, pointing out the most common arguments,
and either declare them as open, or explain why they aren't
issues. People can then discuss these explanations.

Regards,
Martin

From bjourne at gmail.com  Thu Jan 26 19:45:09 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Thu, 26 Jan 2006 19:45:09 +0100
Subject: [Python-Dev] / as path join operator (was: Re: The path module
	PEP)
In-Reply-To: <1138247468.30252.11.camel@geddy.wooz.org>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<034C8957-94EF-4F36-906D-D4EC57221FAF@ihug.co.nz>
	<1138247468.30252.11.camel@geddy.wooz.org>
Message-ID: <740c3aec0601261045l211c1e0fp54584f72259ae76c@mail.gmail.com>

I think that everything that can be said aboud __div__() has already
been said. But this argument was really convincing:

[Tony Meyer]
> The vast majority of people (at least at the time) were either +0 or
> -0, not +1.  +0's are not justification for including something.

There is no clear consensus either way. Ultimately, Guido will decide
if he thinks it is clever or not. Meanwhile I'll remove it from the
PEP and keep it as an "optional extension." Also, like Jason said, the
removal of __div__() leads to the ultimate demise of joinpath(), woho!

[Jason Orendorff]
> in which case I propose using the Path constructor as the
> replacement for os.path.join().  In that case, Path.joinpath can be
> dropped.

    Path.cwd() / "foobar"
    ==>
    Path(Path.cwd(), "foobar")

    Path("foo") / "bar" / "baz"
    ==>
    Path("foo", "bar", "baz")

Still, in the simpler cases, __div__() looks really handy:

    os.chdir(pkgdir / "include")
    ==>
    os.chdir(Path(pkgdir, "include"))

Oh well. You can't have everything, can you? The updated PEP and an
implementation is availible from
http://wiki.python.org/moin/PathClass.

--
mvh Bj?rn

From martin at v.loewis.de  Thu Jan 26 19:54:03 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Thu, 26 Jan 2006 19:54:03 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <43D8FE24.1030500@egenix.com>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<43D8FE24.1030500@egenix.com>
Message-ID: <43D91ACB.3090401@v.loewis.de>

M.-A. Lemburg wrote:
> Please note that inheritance from string will cause the C type
> checks of the form PyString_Check(obj) to return true.
> C code will then assume that it has an object which is
> compatible to string C API which instances aren't.

Oh, sure they are. Types inheriting from str have the same
layout as str, and C code assuming that layout will work fine
with them. Inheritance works (saying "inheritance *just* works
would deny the many fine details that have been engineered to
actually make it work").

Regards,
Martin

From news-nospam at northportal.net  Thu Jan 26 20:23:30 2006
From: news-nospam at northportal.net (VanL)
Date: Thu, 26 Jan 2006 12:23:30 -0700
Subject: [Python-Dev] The path module PEP
In-Reply-To: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
Message-ID: <drb7j4$2cn$1@sea.gmane.org>

<Delurking>

The path module has a great idea, but it does too much -- it conflates 
too many ideas into a single module.

It has methods for dealing with files (open, bytes, read, etc) as well 
as methods for dealing with a filesystem tree as a whole (relpath, 
abspath, etc).  Both of these ideas are tangentially related to paths, 
but really aren't in the same conceptual box.

Not too long ago, I had to build something loosely based upon the path 
module.  Instead of using it as-is, I broke it up into three modules:

Tree (filesystem interfaces)
Path (*just* path interfaces)
File (a generic filelike object)

Doing it this way had two benefits:

First, it put different concepts into different modules.  I note that 
some other virtual filesystem modules also maintedned a similar 
separation - probably for similar reasons.

Second, I was able to define an interface which could be used across 
remote systems -- e.g. I was able to have an FTPTree (built on the 
standard library ftplib) and SSHTree (built upon paramiko) as well as 
FileTree (a standard filesystem).  This isn't full-fledged interfaces - 
I just implemented common functionality in a class and then delegated to 
a ._ops class which passed through the necessary operations.  However, I 
was able to use the various trees and file-like objects interchangeably.



From steve at holdenweb.com  Thu Jan 26 20:55:08 2006
From: steve at holdenweb.com (Steve Holden)
Date: Thu, 26 Jan 2006 19:55:08 +0000
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <43D90014.10807@colorstudy.com>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>	<dratnq$lbp$1@sea.gmane.org>
	<43D90014.10807@colorstudy.com>
Message-ID: <43D9291C.9040009@holdenweb.com>

Ian Bicking wrote:
> Fredrik Lundh wrote:
> 
>>>However, I might be wrong because according to [1] it should work. And
>>>having to wrap the Path object in str() (open(str(somepath))) each and
>>>every time the called function expects a string is not a practical
>>>solution.
>>
>>
>>in Python, the usual way to access an attribute of an object is to
>>access the attribute; e.g.
>>
>>    f = open(p.name)
> 
> 
> You mean "f = open(Path(p).name)", because it is likely that people will 
>   also have to accept strings for the nearterm (and probably longeterm) 
> future.  And the error message without will be inscrutable (and will 
> still be inscrutable in many cases when you try to access other methods, 
> sadly).  And currently .name is taken for something else in the API. 
> And the string path is not really an attribute because the string path 
> *is* the object, it is not *part* of the object.
> 
> OTOH, str(path) will break unicode filenames.  And unicode() breaks 
> anything that simply desires to pass data through without effecting its 
> encoding.
> 
> An open method on paths simplifies many of these issues, but doesn't do 
> anything for passing a path to legacy code.  Changing open() and all the 
> functions that Path replaces (e.g., os.path.join) to accept Path objects 
> may resolve issues with a substantial portion of code.  But any code 
> that does a typecheck on arguments will be broken -- which in the case 
> of paths is quite common since many functions take both filename and 
> file object arguments.
> 
Would it help to redefine file/open so they called an __open__() method 
on the argument were one defined, otherwise reverting to current behaviour?

That way we could just just define an __open__(self) method for path 
objects. I doubt performance is a huge issue here.

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From ronaldoussoren at mac.com  Thu Jan 26 20:57:30 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 26 Jan 2006 20:57:30 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
Message-ID: <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>


On 26-jan-2006, at 18:04, James Y Knight wrote:

>
> On Jan 26, 2006, at 7:29 AM, Thomas Heller wrote:
>>> And licenses are fluid, it may be a piece of cake to
>>> get one of those 'tools' un-GPL'ed, even if they are.
>>
>> I wouldn't even know whom to ask.
>
>
> On Jan 26, 2006, at 9:22 AM, Ronald Oussoren wrote:
>> It shouldn't be too hard to use Python's main configure script to  
>> calculate
>> the information necessary to build libffi. A lot of it is already  
>> calculated
>> anyway (sizeof various type, endianness), some can be hardcoded  
>> (FFI_NO_RAW_API).
>>
>> In PyObjC I just compile the files I need from my setup.py. But I  
>> have an easy task,
>> I just need to support two CPU architectures on one OS.
>
> Let's please be sure the license isn't fine _as is_ before thinking  
> about asking people to change them or rewriting the build system! I  
> did not look at *all* the files listed as being GPL, but the first  
> two in the list were not.

Merging the two configure files might be a good idea anyway, that  
would take away the need to run configure from setup.py. IANAL, but I  
don't quite get how a GPL'd support script, if there is such a thing,  
in the build machinery of an extension library would require that  
Python itself is GPL'd.

Anyhow, in my copy of libffi (a snapshot where the last entry in the  
Changelog is from 2004-04-26) the only the following files at the  
toplevel op
libffi are GPL licensed: config.guess, config.sub,  config-ml.in,  
ltcf-c.sh, ltconfig, ltmain.sh, missing. All of these contain an  
exception clause like this one in config.guess:

# As a special exception to the GNU General Public License, if you
# distribute this file as part of a program that contains a
# configuration script generated by Autoconf, you may include it under
# the same distribution terms that you use for the rest of that program.

I'd say that it should be save to include these in the Python  
distribution.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060126/2301384a/attachment.bin 

From ronaldoussoren at mac.com  Thu Jan 26 21:02:49 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Thu, 26 Jan 2006 21:02:49 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <d5ifatsb.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<864123BC-0DCD-4AD3-8BDA-EC9625090FF4@mac.com>
	<d5ifatsb.fsf@python.net>
Message-ID: <5B31C78A-E34F-4751-847A-CB35B3E623A8@mac.com>


On 26-jan-2006, at 16:33, Thomas Heller wrote:

> Ronald Oussoren <ronaldoussoren at mac.com> writes:
>
>> On 26-jan-2006, at 13:29, Thomas Heller wrote:
>>
>>> Thomas Wouters <thomas at xs4all.net> writes:
>>>
>>>> On Thu, Jan 26, 2006 at 09:54:51AM +0100, Thomas Heller wrote:
>>>>
>>>>> The current state is that ctypes uses GPL'd tools to build libffi,
>>>>> and those can't be committed into Python SVN.
>>>>
>>>>> http://mail.python.org/pipermail/python-dev/2006-January/ 
>>>>> 059937.html
>>
>> It shouldn't be too hard to use Python's main configure script to
>> calculate the information necessary to build libffi. A lot of it is
>> already calculated anyway (sizeof various type, endianness), some can
>> be hardcoded (FFI_NO_RAW_API).
>>
>> In PyObjC I just compile the files I need from my setup.py. But I  
>> have
>> an easy task, I just need to support two CPU architectures on one OS.
>
> Thanks for the encouragement - Martin suggested a similar approach.
>
>> From my understanding (which goes not very far) the configuration  
>> does
> two things: determine the set of source files that needs to go in
> depending on the cpu architecture, and to determine some  
> information and
> make them available in #defines.  I have to check if this is possible
> without patching the libffi sources themselves. I guess I could look
> into the PyObjC setuop script.

PyObjC's solution is kind of a hack: I always compile all files needed
for i386 and PPC support and use #ifdef statements to make sure only the
files for the current platform are actually used. This is a hack to make
it easier to build a universal (aka fat) binary of PyObjC.

>
> Personally I only have access to machines running windows, linux (x86
> and x86_64), and OS X (plus possibly a Mac running ubuntu), so I could
> only do it for those.  Maybe support for other architectures can be
> added by volunteers?
>
> Besides:  James Y Knight seems to be correct that all the scripts  
> needed
> to build libffi seems to have this special exception from the GPL.

I should catch up on python-dev before looking into this. I just  
noted the
same thing :-)

Ronald

>
> Thomas
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/ 
> ronaldoussoren%40mac.com

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060126/c38ddc0a/attachment.bin 

From news-nospam at northportal.net  Thu Jan 26 21:07:57 2006
From: news-nospam at northportal.net (VanL)
Date: Thu, 26 Jan 2006 13:07:57 -0700
Subject: [Python-Dev] The path module PEP
In-Reply-To: <drb7j4$2cn$1@sea.gmane.org>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<drb7j4$2cn$1@sea.gmane.org>
Message-ID: <drba6h$dp0$1@sea.gmane.org>

One other benefit that I neglected to put into the previous post - I was 
able to maintain separate cwd's for each tree.

An example of use:

Each tree has its own context, independent of the context of python:

 >>> local, local2 = fs.LocalTree(), fs.LocalTree()
 >>> local.pwd
'/home/targoz'
 >>> local2.pwd
'/home/targoz'
 >>> os.getcwd()
'/home/targoz'
 >>> local.chdir('..')
 >>> local2.chdir('www')
 >>> local.pwd
'/home'
 >>> local2.pwd
'/home/targoz/www'
 >>> os.getcwd()
'/home/targoz'

Remote trees have the same interface:

 >>> remote_login_data = {'username': 'targoz', 'password': 'my_pass', 
host': 'hostname.com'}
 >>> remote = fs.SSHTree(access=remote_login_data)
 >>> remote.pwd()
'/home/nportal'

Trees can interact, regardless of whether they are local or remote:

 >>> local2.listdir('files')
['myfile', 'otherfile.txt']
 >>> remote.listdir()
[]
 >>> localfile = local2.open(('targoz/myfile') # Opens a file-like object
 >>> remote.savefile(localfile, 'remote_name')
 >>> remote.listdir()
['myfile']


From thomas at xs4all.net  Thu Jan 26 21:09:56 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 26 Jan 2006 21:09:56 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <43D9291C.9040009@holdenweb.com>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<dratnq$lbp$1@sea.gmane.org> <43D90014.10807@colorstudy.com>
	<43D9291C.9040009@holdenweb.com>
Message-ID: <20060126200956.GI18916@xs4all.nl>

On Thu, Jan 26, 2006 at 07:55:08PM +0000, Steve Holden wrote:

> Would it help to redefine file/open so they called an __open__() method 
> on the argument were one defined, otherwise reverting to current behaviour?

Not really, open() is (by far!) not the only function that breaks. Most
posixmodule functions, by the looks of it, as well as a few in _tkinter and
socket. And that's just a quick grep for PyArg_Parse* with '"et' as an
argument. Perhaps if __open__ was generalized into __equivalent_string__...
but we've been there already. ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From thomas at xs4all.net  Thu Jan 26 21:24:56 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Thu, 26 Jan 2006 21:24:56 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <20060125005918.GC18916@xs4all.nl>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
	<20060125005918.GC18916@xs4all.nl>
Message-ID: <20060126202456.GJ18916@xs4all.nl>

On Wed, Jan 25, 2006 at 01:59:18AM +0100, Thomas Wouters wrote:

> [ iffy isatty behaviour on Solaris ]

Considering that:
 - the approach for opening pty's, while not the only one, is the preferred
   way of doing it on Solaris,
 - the actual pty's seem to be completely functional,
 - the usual way to use pty's is not like the test does (inside a single
   process), and I'd say using pty's like the test does is extremely
   unlikely to happen in real life,
 - testing inside the tty-creating process is quite possibly the reason for
   the fickleness of the test, since its behaviour isn't guaranteed
   anywhere,
 - the test inside a subprocess, the normal way of using pty's, is not
   failing (as far as I can tell),

I'd like to check in the attached patch. It doesn't fix anything, it just
removes this one test failure on Solaris. I don't want to skip the entire
test, because it still tests whether everything else works as expected, and
I don't want spurious failures as they can mask a real failure later in the
test.

I'd need developer access back to check it in, though. Unless anyone
objects, of course :)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!
-------------- next part --------------
Index: Lib/test/test_pty.py
===================================================================
--- Lib/test/test_pty.py	(revision 42187)
+++ Lib/test/test_pty.py	(working copy)
@@ -4,6 +4,13 @@
 TEST_STRING_1 = "I wish to buy a fish license.\n"
 TEST_STRING_2 = "For my pet fish, Eric.\n"
 
+# Solaris (at least 2.9 and 2.10) seem to have a ficke isatty(). The first
+# test below, testing the result of os.openpty() for tty-ness, sometimes
+# (but not always) fails. The second isatty test, in the sub-process, always
+# works. Allow that fickle first test to fail on these platforms, since it
+# doesn't actually affect functionality.
+fickle_isatty = ["sunos5"]
+
 if verbose:
     def debug(msg):
         print msg
@@ -26,7 +33,7 @@
         # " An optional feature could not be imported " ... ?
         raise TestSkipped, "Pseudo-terminals (seemingly) not functional."
 
-    if not os.isatty(slave_fd):
+    if not os.isatty(slave_fd) and sys.platform not in fickle_isatty:
         raise TestFailed, "slave_fd is not a tty"
 
     # IRIX apparently turns \n into \r\n. Allow that, but avoid allowing other

From fredrik at pythonware.com  Thu Jan 26 21:33:04 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu, 26 Jan 2006 21:33:04 +0100
Subject: [Python-Dev] building a module catalogue with buildbot
References: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com><dq3u93$s4g$1@sea.gmane.org><20060111223039.GC6642@activestate.com><ee2a432c0601230036l61c79c4dp617d1521f514afd6@mail.gmail.com><dr4qrp$dhh$1@sea.gmane.org>
	<dr5k4c$agj$1@sea.gmane.org>
Message-ID: <drbbm3$k2b$1@sea.gmane.org>

any progress ?  does the script work in the buildbot setting, or
does it need tweaking ?

</F>

> > Neal Norwitz wrote:
> >
> > > > > > Does that make sense?  We would just need /f's script in SVN.
> > > > >
> > > > > in python/Tools/something or sandbox/something ?
> > > >
> > > > python/Doc/tools/something?
> > >
> > > Fredrik were you still working on that?  I can make the changes to the
> > > bb master.  I thought Trent's suggested placement was good.
> >
> > iirc, the script needed some minor tweaking (using os.path.splitext to
> > test for the module.so extension isn't a good idea), and I don't recall
> > if I've fixed that or not...
> >
> > (probably not, since I never checked it in).
> >
> > I'll take a look asap.
>
> alright, I just checked in a
>
>     Doc/tools/listmodules.py
>
> which attempts to produce a sorted list of all modules available in
> a given python build.  by default, it prints the list to stdout, which
> should be good enough for a "catalog" buildbot step.




From theller at python.net  Thu Jan 26 21:33:13 2006
From: theller at python.net (Thomas Heller)
Date: Thu, 26 Jan 2006 21:33:13 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net>
	<20060126090822.GF18916@xs4all.nl> <ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
Message-ID: <fyna4tme.fsf@python.net>

Ronald Oussoren <ronaldoussoren at mac.com> writes:

>> On Jan 26, 2006, at 9:22 AM, Ronald Oussoren wrote:
>>> It shouldn't be too hard to use Python's main configure script to
>>> calculate the information necessary to build libffi. A lot of it is
>>> already calculated anyway (sizeof various type, endianness), some
>>> can be hardcoded (FFI_NO_RAW_API).
>>>
>>> In PyObjC I just compile the files I need from my setup.py. But I
>>> have an easy task, I just need to support two CPU architectures on
>>> one OS.
>>
> Merging the two configure files might be a good idea anyway, that
> would take away the need to run configure from setup.py. IANAL, but I
> don't quite get how a GPL'd support script, if there is such a thing,
> in the build machinery of an extension library would require that
> Python itself is GPL'd.
>
> Anyhow, in my copy of libffi (a snapshot where the last entry in the
> Changelog is from 2004-04-26) the only the following files at the
> toplevel op libffi are GPL licensed: config.guess, config.sub,
> config-ml.in, ltcf-c.sh, ltconfig, ltmain.sh, missing. All of these
> contain an exception clause like this one in config.guess:

ctypes libffi is somewhat newer, around 2005-05-09.

> # As a special exception to the GNU General Public License, if you
> # distribute this file as part of a program that contains a
> # configuration script generated by Autoconf, you may include it under
> # the same distribution terms that you use for the rest of that program.
>
> I'd say that it should be save to include these in the Python
> distribution.

As I said in the other thread  (where the discussion should probably be
continued anyway):

http://mail.python.org/pipermail/python-dev/2006-January/060113.html

only aclocal.m4 isn't clear to me about the license.  Anyway, it could
be that this file isn't needed after all - I don't know enough about the
GNU toolchain to be sure.  Can anyone comment on this?

Neither do I know enough to merge the configure scripts.  Contributions
would really, really gratefully be accepted.

Thomas


From mal at egenix.com  Thu Jan 26 21:48:16 2006
From: mal at egenix.com (M.-A. Lemburg)
Date: Thu, 26 Jan 2006 21:48:16 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <43D91ACB.3090401@v.loewis.de>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>	<43D8FE24.1030500@egenix.com>
	<43D91ACB.3090401@v.loewis.de>
Message-ID: <43D93590.7040009@egenix.com>

Martin v. L?wis wrote:
> M.-A. Lemburg wrote:
>> Please note that inheritance from string will cause the C type
>> checks of the form PyString_Check(obj) to return true.
>> C code will then assume that it has an object which is
>> compatible to string C API which instances aren't.
> 
> Oh, sure they are. Types inheriting from str have the same
> layout as str, and C code assuming that layout will work fine
> with them. Inheritance works (saying "inheritance *just* works
> would deny the many fine details that have been engineered to
> actually make it work").

You're right, I forgot about how the .__new__() works on
new-style classes and that extra space is allocated
appended to the base type object for the extra
instance features.

>From PEP 253:
"""
    class C(B): pass

...

    In any case, the work for creating C is done by M's tp_new() slot.
    It allocates space for an "extended" type structure, containing:
    the type object; the auxiliary structures (as_sequence etc.); the
    string object containing the type name (to ensure that this object
    isn't deallocated while the type object is still referencing it);and
    some auxiliary storage (to be described later).  It initializes this
    storage to zeros except for a few crucial slots (for example,tp_name
    is set to point to the type name) and then sets the tp_base slot to
    point to B.

"""

Sorry for the FUD,
-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Jan 26 2006)
>>> Python/Zope Consulting and Support ...        http://www.egenix.com/
>>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
>>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From jjl at pobox.com  Thu Jan 26 22:21:28 2006
From: jjl at pobox.com (John J Lee)
Date: Thu, 26 Jan 2006 21:21:28 +0000 (UTC)
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <fyna4tme.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<fyna4tme.fsf@python.net>
Message-ID: <Pine.LNX.4.58.0601262119050.6219@alice>

On Thu, 26 Jan 2006, Thomas Heller wrote:
[...]
> As I said in the other thread  (where the discussion should probably be
> continued anyway):
> 
> http://mail.python.org/pipermail/python-dev/2006-January/060113.html
> 
> only aclocal.m4 isn't clear to me about the license.  Anyway, it could
> be that this file isn't needed after all - I don't know enough about the
> GNU toolchain to be sure.  Can anyone comment on this?

>From 'info autoconf':

|   The Autoconf macros are defined in several files.  Some of the files
| are distributed with Autoconf; `autoconf' reads them first.  Then it
| looks for the optional file `acsite.m4' in the directory that contains
| the distributed Autoconf macro files, and for the optional file
| `aclocal.m4' in the current directory.  Those files can contain your
| site's or the package's own Autoconf macro definitions (*note Writing
[...]

So, I assume aclocal.m4 is under the same license as the rest of the
libffi you're using.


John

From nnorwitz at gmail.com  Thu Jan 26 22:31:06 2006
From: nnorwitz at gmail.com (Neal Norwitz)
Date: Thu, 26 Jan 2006 13:31:06 -0800
Subject: [Python-Dev] building a module catalogue with buildbot
In-Reply-To: <drbbm3$k2b$1@sea.gmane.org>
References: <ee2a432c0601102349i5814f01cn7083e66fec92e751@mail.gmail.com>
	<dq3u93$s4g$1@sea.gmane.org> <20060111223039.GC6642@activestate.com>
	<ee2a432c0601230036l61c79c4dp617d1521f514afd6@mail.gmail.com>
	<dr4qrp$dhh$1@sea.gmane.org> <dr5k4c$agj$1@sea.gmane.org>
	<drbbm3$k2b$1@sea.gmane.org>
Message-ID: <ee2a432c0601261331h5500188du25898e03ea3e3e0@mail.gmail.com>

On 1/26/06, Fredrik Lundh <fredrik at pythonware.com> wrote:
> any progress ?  does the script work in the buildbot setting, or
> does it need tweaking ?

I haven't gotten to it and won't be able to in the next week+.  If no
one beats me to it, I will get to it in a few weeks.  I've got most of
the buildbot machines green and working towards stable.

n

From ncoghlan at gmail.com  Thu Jan 26 22:31:51 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Fri, 27 Jan 2006 07:31:51 +1000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <Pine.LNX.4.64.0601261403440.30122@qnzvnan.rov.np.hx>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	<20060126004113.GD18916@xs4all.nl>	<Pine.LNX.4.64.0601260915100.29964@qnzvnan.rov.np.hx>	<20060126132641.GG18916@xs4all.nl>
	<Pine.LNX.4.64.0601261403440.30122@qnzvnan.rov.np.hx>
Message-ID: <43D93FC7.3020401@gmail.com>

Michael Hoffman wrote:
> I've been using path.py for some time, and I can tell you that it
> would be a lot less useful if it no longer behaved like string-plus.

As Jason pointed out elsewhere, the strict typechecks that exist *in* the 
Python core, and the fact that path.py is *outside* that core makes the 
workaround of subclassing string necessary.

Since the PEP process has the power to alter the *core*, then we have other 
options than "this is a string, only not".

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From fuzzyman at voidspace.org.uk  Thu Jan 26 21:28:16 2006
From: fuzzyman at voidspace.org.uk (Fuzzyman)
Date: Thu, 26 Jan 2006 20:28:16 +0000
Subject: [Python-Dev] Extension to ConfigParser
Message-ID: <43D930E0.6070805@voidspace.org.uk>

Hello all,

In the past there has been some discussion about a new module to replace
ConfigParser. Most notably at
http://wiki.python.org/moin/ConfigParserShootout

Specific issues that could be addressed include :

* Simpler API
* Nested subsections
* List values
* Storing/converting datatypes
* Config file schema
* Keeps track of order of values

Plus other issues.

I'm the (co-)author of ConfigObj -
http://www.voidspace.org.uk/python/configobj.html

This is a reasonably mature project (now in it's fourth incarnation),
and is being used in projects like `Bazaar <http://www.bazaar-ng.org/>`_
and `PlanetPlus <http://planetplus.python-hosting.com/>`_.

It occurs to me that the ConfigObj API and syntax is *almost* fully
compatible with ConfigParser.

It would be possible to extend to the ConfigObj API to be backwards
compatible with ConfigParser. This would bring the added benefits of
ConfigObj, without needing to add an extra module to the standard library.

Well nearly. ConfigObj supports config file schema with (optional) type
conversion, through a companion module called validate. This could be
included or left as an added option.

Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and
if accepted, do the work - which is not inconsiderable).

Summary of ConfigObj
================

ConfigObj is a Python 2.2 compatible config file parser. It's major
feature is simplicity of use.

It reads (and writes) INI file like config files and presents the
members using a dictionary interface.

The order of keys/sections is preserved, and it allows nested
subsections to any level :

e.g. ::

    key = value
        [section]
        key = value
          [[sub-section]]
          key = value

It is fully documented with a barrage of doctests.
All comments in the config file are also preserved as attributes of the
object, and will be written back out. This can be useful for including
comments in programatically generated config files.

It is integrated with a powerful validation system.

Difficulties & Differences
=================

A ConfigObj instance is a sub-class of the dictionary datatpe. This
means that the ``get`` method of ConfigParser clashes.

ConfigObj allows values in the root section (why not ?).

ConfigObj doesn't support line continuations (it does all multi-line
values through the use of triple quoted strings).

ConfigObj currently only allows '=' as a valid divider.

Creating ConfigParser (and related classes) compatibility is a big job.

Solution
=====

All of these problems (if deemed necessary) can be resolved. Either
through options, or just by extending the ConfigObj API. I'm happy to
put the work in.

Comments ?

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/index.shtml



From jimjjewett at gmail.com  Thu Jan 26 22:48:41 2006
From: jimjjewett at gmail.com (Jim Jewett)
Date: Thu, 26 Jan 2006 16:48:41 -0500
Subject: [Python-Dev] The path module (class) PEP
Message-ID: <fb6fbf560601261348q3f2c9acdlda3cbd2cf48c3ad0@mail.gmail.com>

(1)  What is the advantage of module Path vs just turning os.path into a class?
(except that people already import from os.path, so I suppose it would
need to be os.Path)

(2)  It sounds like quite a few stdlib calls will be both deprecated
and wrapped.  Having a new stdlib module rely on deprecated features
leaves a bad taste.  Perhaps move the functionality to the Path class
and then forward from the current (about to be deprecated) modules?

-jJ

From tim.peters at gmail.com  Fri Jan 27 03:41:53 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Thu, 26 Jan 2006 21:41:53 -0500
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <20060126202456.GJ18916@xs4all.nl>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
	<20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl>
Message-ID: <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com>

[Thomas Wouters]
> ...
> I'd need developer access back to check it in, though.

AFAICT, twouters has developer access to the Python project --
although maybe someone else re-enabled that w/o mentioning it here.

> Unless anyone objects, of course :)

Of course I object!  I bow to the group will, though.

From robey at lag.net  Fri Jan 27 05:42:16 2006
From: robey at lag.net (Robey Pointer)
Date: Thu, 26 Jan 2006 20:42:16 -0800
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <dp2mmh$45b$1@sea.gmane.org>
References: <dope79$flv$2@sea.gmane.org><dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com><dp1jd8$p4a$1@sea.gmane.org>
	<43B4A238.2030101@python.org>
	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
	<dp2mmh$45b$1@sea.gmane.org>
Message-ID: <05663E18-CDFB-4B18-A515-BAA21C48C1FD@lag.net>


On 29 Dec 2005, at 23:13, Fredrik Lundh wrote:

> Robey Pointer wrote:
>>> [Fredrik Lundh]
>>>> Really?
>>>
>>> Yes, really.
>>
>> Just out of curiosity (really -- not trying to jump into the flames)
>> why not just use epydoc?  If it's good enough for 3rd-party python
>> libraries, isn't that just a small step from being good enough for
>> the builtin libraries?
>
> but epydoc is a docstring-based format, right?
>
> I'm trying to put together a light-weight alternative to the markup
> used for, primarily, the current library reference.
>
> moving all of (or parts of) the reference documentation in to the
> library source code would be an alternative, of course [1], but that
> would basically mean starting over from scratch.

I think that would be a good thing to do no matter what markup is  
used.  It always irritates me when I do 'help(sys.something)' and get  
one line of ASCII art that's probably useful to the module author but  
nobody else.

robey


From robey at lag.net  Fri Jan 27 05:45:12 2006
From: robey at lag.net (Robey Pointer)
Date: Thu, 26 Jan 2006 20:45:12 -0800
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com>
References: <dope79$flv$2@sea.gmane.org> <dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org> <43B4A238.2030101@python.org>
	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
	<60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com>
Message-ID: <F5A461E8-F558-4D6E-B485-7A0C94E730AF@lag.net>


On 30 Dec 2005, at 18:29, Christopher Armstrong wrote:

> On 12/30/05, Robey Pointer <robey at lag.net> wrote:
>>
>> Just out of curiosity (really -- not trying to jump into the flames)
>> why not just use epydoc?  If it's good enough for 3rd-party python
>> libraries, isn't that just a small step from being good enough for
>> the builtin libraries?
>
> It's not really even "good enough" for a lot of my usage without some
> seriously evil hacks. The fundamental design decision of epydoc to
> import code, plus some other design decisions on the way it figures
> types and identity seriously hinder its utility. Ever notice how
> trying to document your third-party-library-using application will
> *also* document that third party library, for example? Or how it'll
> blow up when you're trying to document your gtk-using application on a
> remote server without an X server running? Or how it just plain blows
> right up with most Interface systems? etc.

Err... no, I haven't.  But I may be using a more recent version than  
you were (I'm using 2.1).  It's not perfect, and occasionally  
annoying, but much better than anything else I've found.  Sounds like  
there's some political reason it's hated, but thought I would bring  
it up anyway.

robey


From p.f.moore at gmail.com  Fri Jan 27 11:16:09 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Fri, 27 Jan 2006 10:16:09 +0000
Subject: [Python-Dev] The path module PEP
In-Reply-To: <43D8E0E9.9080202@ofai.at>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>
	<43D8E0E9.9080202@ofai.at>
Message-ID: <79990c6b0601270216x2c43449cj534427f7a8f5234c@mail.gmail.com>

On 1/26/06, Stefan Rank <stefan.rank at ofai.at> wrote:
> on 26.01.2006 14:15 Paul Moore said the following:
> [snip]
> >
> > Also note that my example Path("C:", "Windows", "System32") above is
> > an *absolute* path on Windows. But a relative (albeit stupidly-named
> > :-)) path on Unix. How would that be handled?
>
> wrong, Path("C:", "Windows", "System32") is a relative path on windows.
> see below.

Hmm, relative to the CWD on C: is a valid concept, and that is a
potential meaning. I hadn't thought of that.

> > Not that os.path gets it perfect:
> >
> > Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on win32
> > Type "help", "copyright", "credits" or "license" for more information.
> >>>> import os
> >>>> os.path.join("C:", "Windows", "System32")
> > 'C:Windows\\System32'
> >>>> os.path.join(".", os.path.join("C:", "Windows", "System32"))
> > '.\\C:Windows\\System32'
> >
>
> this is misleading. observe::
>
>   In [1]: import os
>
>   In [2]: os.path.join(".", os.path.join("C:", "Windows", "System32"))
>   Out[2]: '.\\C:Windows\\System32'
>
> but::
>
>   In [3]: os.path.join(".", os.path.join("C:\\", "Windows", "System32"))
>   Out[3]: 'C:\\Windows\\System32'
>
>
> The second example uses an absolute path as second argument, and as
> os.path.join should do, the first argument is discarded.
>
> The first case is arguably a bug, since, on windows, C:Windows\System32
> is a path relative to the *current directory on disk C:*
> If the cwd on C: would be C:\temp then C:Windows\System32 would point to
> C:\temp\Windows\System32
>
> The problem is that Windows has a cwd per partition...
> (I cannot even guess why ;-)

Thanks for the clarification, you are right in your analysis. However,
it doesn't really affect my main point, which was that there should be
no such thing as a relative Path (please note - I say "Path" here, to
refer to the new Path object, as opposed to the general concept of an
OS file path).

[...]
> > Arguably, Path objects should always maintain an absolute path - there
> > should be no such thing as a relative Path. So you would have
>
> you realise that one might need and/or want to represent a relative path?

Absolutely. But not a Path (see distinction above).

Aaron Bingham's analogy with time/timedelta applies well here.
Relative paths, like relative times, have their own special semantics,
which deserve to be addressed in a separate class.

You argue that time is "merely" a timedelta with a fixed start point.
I'd disagree - the key point with timedeltas is that they need careful
handling (DST issues, for example) _depending upon precisely what they
are added to_ - these issues are avoided by the time type exactly
because it has a constant base. In exactly the same way, absolute
paths have simpler semantics precisely because they are absolute.

Paul.

From steve at holdenweb.com  Fri Jan 27 11:42:22 2006
From: steve at holdenweb.com (Steve Holden)
Date: Fri, 27 Jan 2006 10:42:22 +0000
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
Message-ID: <43D9F90E.1040703@holdenweb.com>

You may be aware that Tim Parkin's work on our "next-generation" web 
presence has borne fruit in the shape of beta.python.org. While there's 
still a lot to be done Tim has given us a great start by creating a 
framework that makes it rather easier to manage content.

I'm just starting to work on the beta site content, and this is causing 
some examination of the existing web content. Is there any way to affect 
the target of the very prominent "Download Python" link on 
http://sourceforge.net/projects/python/ ?

I ask because the link currently takes you to a page whose title is 
"Exiting with Error" and whose content is largely "No File Packages". 
While it's not where you arrive from a Google search for "python 
download" it is nevertheless a legitimate way (just now) to approach the 
project, and should therefore really be live.

Is there any chance some SF admin could point it to

     http://www.python.org/download/

There's the further issue that the entire HTML body of 
http://svn.python.org/ currently reads

<!-- not sure what to put here -->

It would seem like the logical place to have pointers to Subversion 
software (downloads, documentation, operating instructions) together 
with an *annotated* summary of http://svn.python.org/projects/ and links 
back to the main developer site at the very least. I'm not even sure 
where that web content is under SVN control at the moment.

The web revision is going to lead to several of these kinds of 
discussions. I'll try to only spill on to the python-dev list what 
impinges on developers. Your opinions on these specific issues are 
probably the most significant.

[pydotorgers: let's try not to spam python-dev with any discussions]

regards
  Steve
-- 
Steve Holden       +44 150 684 7255  +1 800 494 3119
Holden Web LLC                     www.holdenweb.com
PyCon TX 2006                  www.python.org/pycon/


From stefan.rank at ofai.at  Fri Jan 27 11:47:21 2006
From: stefan.rank at ofai.at (Stefan Rank)
Date: Fri, 27 Jan 2006 11:47:21 +0100
Subject: [Python-Dev] The path module PEP
In-Reply-To: <79990c6b0601270216x2c43449cj534427f7a8f5234c@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>	<43D6BA85.8070007@colorstudy.com>	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>	<79990c6b0601260515w22ae9e2dy22d265acc4bce7c3@mail.gmail.com>	<43D8E0E9.9080202@ofai.at>
	<79990c6b0601270216x2c43449cj534427f7a8f5234c@mail.gmail.com>
Message-ID: <43D9FA39.6060407@ofai.at>

on 27.01.2006 11:16 Paul Moore said the following:
> [...]
>>> Arguably, Path objects should always maintain an absolute path - there
>>> should be no such thing as a relative Path. So you would have
>> you realise that one might need and/or want to represent a relative path?
> 
> Absolutely. But not a Path (see distinction above).
> 
> Aaron Bingham's analogy with time/timedelta applies well here.
> Relative paths, like relative times, have their own special semantics,
> which deserve to be addressed in a separate class.
> 
> You argue that time is "merely" a timedelta with a fixed start point.
> I'd disagree - the key point with timedeltas is that they need careful
> handling (DST issues, for example) _depending upon precisely what they
> are added to_ - these issues are avoided by the time type exactly
> because it has a constant base. In exactly the same way, absolute
> paths have simpler semantics precisely because they are absolute.
> 
> Paul.

I see your point.

I guess there are two options:

- `Path` as an enhanced string type that bundles methods related to file 
system addressing

- `Path`s that accurately reflect the possible (abstract) paths. There 
would be a Path and a PathDelta (with appropriate combining semantics), 
and probably a UnixPath, a WindowsPath, an URLPath maybe. And there need 
to be appropriate methods for combining them with/creating them from 
strings.

I actually think the latter would be very neat and somewhere else in 
this thread someone talks about his `Tree` - `Path` - `File` classes 
with specialised subclasses.

The first option, however, has the benefit of simplicity and there is a 
working implementation.

Well - I'm not the one to decide. And I think anything that bundles path 
related stuff (using a little object-orientation) and cleans up the 
standard library is a step forward.

cheers,
s


From thomas at xs4all.net  Fri Jan 27 12:00:53 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 27 Jan 2006 12:00:53 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
	<20060125005918.GC18916@xs4all.nl>
	<20060126202456.GJ18916@xs4all.nl>
	<1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com>
Message-ID: <20060127110053.GO5045@xs4all.nl>

On Thu, Jan 26, 2006 at 09:41:53PM -0500, Tim Peters wrote:
> [Thomas Wouters]
> > ...
> > I'd need developer access back to check it in, though.

> AFAICT, twouters has developer access to the Python project --
> although maybe someone else re-enabled that w/o mentioning it here.

I meant svn-checkin-access (it got disabled for disuse a while back.) If I
weren't schizofrenic in sysadmin jobs I could add it myself, but I can't do
that. Can I? I guess I can't. Who am I again?

Someone-else'ly y'rs,
-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From edloper at gradient.cis.upenn.edu  Fri Jan 27 13:55:01 2006
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Fri, 27 Jan 2006 07:55:01 -0500
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <F5A461E8-F558-4D6E-B485-7A0C94E730AF@lag.net>
References: <dope79$flv$2@sea.gmane.org>
	<dp0qrh$r78$1@sea.gmane.org>	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>	<dp1jd8$p4a$1@sea.gmane.org>
	<43B4A238.2030101@python.org>	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>	<60ed19d40512301829q7a0df01bv3ed782fe28cbcf7a@mail.gmail.com>
	<F5A461E8-F558-4D6E-B485-7A0C94E730AF@lag.net>
Message-ID: <43DA1825.8060608@gradient.cis.upenn.edu>

Robey Pointer wrote:
On 30 Dec 2005, at 18:29, Christopher Armstrong wrote:
>> [epydoc] is not really even "good enough" for a lot of my usage without some
>> seriously evil hacks. The fundamental design decision of epydoc to
>> import code, plus some other design decisions on the way it figures
>> types and identity seriously hinder its utility. [...]

As a side note, I've done a significant amount of work on a newer 
version of epydoc that supports both inspection & source-code parsing 
(and can merge the info from both sources, if they're available). 
Hopefully this new version will address some of these concerns (and some 
other issues I've heard raised).  But I haven't had time to work on it 
in a while -- I need to concentrate on getting my phd thesis done.  If 
anyone is interested in working on this new version, just let me know. :)

-Edward


From tim.peters at gmail.com  Fri Jan 27 16:49:59 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 27 Jan 2006 10:49:59 -0500
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <43D9F90E.1040703@holdenweb.com>
References: <43D9F90E.1040703@holdenweb.com>
Message-ID: <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>

[Steve Holden]
> Is there any way to affect the target of the very prominent "Download
> Python" link on
> http://sourceforge.net/projects/python/ ?
>
> I ask because the link currently takes you to a page whose title is
> "Exiting with Error" and whose content is largely "No File Packages".
> While it's not where you arrive from a Google search for "python
> download" it is nevertheless a legitimate way (just now) to approach the
> project, and should therefore really be live.
>
> Is there any chance some SF admin could point it to
>
>      http://www.python.org/download/

Sorry, looks like SF auto-creates the download "button", hardwired to
the project's SF "Files" section.  The Python project doesn't have any
files on SF anymore.

If you go to

    http://sourceforge.net/projects/zodb/

you'll see the equally prominent "Download ZODB and ZEO" button,
pointing to that project's equally ZODB-free Files area.  But, in that
case, you'll see that there _is_ a package, named README, with a
"release" named "How to get the source".  If you click on that and
fight your way through the download process, you eventually end up
with file "zodb.txt", which contains a real download URL for ZODB
<wink/sigh>.

Overall I'm not sure that's an improvement, but it was the best I
could come up with 2 years ago when ZODB stopped using SF for
downloads.

From thomas at xs4all.net  Fri Jan 27 16:59:22 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Fri, 27 Jan 2006 16:59:22 +0100
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>
References: <43D9F90E.1040703@holdenweb.com>
	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>
Message-ID: <20060127155922.GL18916@xs4all.nl>

On Fri, Jan 27, 2006 at 10:49:59AM -0500, Tim Peters wrote:

> If you go to
> 
>     http://sourceforge.net/projects/zodb/
> 
> you'll see the equally prominent "Download ZODB and ZEO" button,
> pointing to that project's equally ZODB-free Files area.  But, in that
> case, you'll see that there _is_ a package, named README, with a
> "release" named "How to get the source".  If you click on that and
> fight your way through the download process, you eventually end up
> with file "zodb.txt", which contains a real download URL for ZODB
> <wink/sigh>.
> 
> Overall I'm not sure that's an improvement, but it was the best I
> could come up with 2 years ago when ZODB stopped using SF for
> downloads.

What if the package wasn't named README, but rather
'www.python.org' with a release named "visit www.python.org/download
instead"? I'm not sure if that package/release name is allowed ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From tim.peters at gmail.com  Fri Jan 27 16:59:45 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 27 Jan 2006 10:59:45 -0500
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <20060127110053.GO5045@xs4all.nl>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
	<20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl>
	<1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com>
	<20060127110053.GO5045@xs4all.nl>
Message-ID: <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com>

[Thomas]
>>> I'd need developer access back to check it in, though.

[Tim]
>> AFAICT, twouters has developer access to the Python project --
>> although maybe someone else re-enabled that w/o mentioning it here.

[Thomas]
> I meant svn-checkin-access (it got disabled for disuse a while back.)

I know.  AFAICT, you (twouters) already have it.  There's a "Yes" in
the twouters row under the "CVS Access" column on the Python project's
Members admin page.  Have you tried checking in?  What happens when
you do?  If it "doesn't work", one possibility is that you did a
read-only SVN _checkout_, and it's actually SVN griping at you.

> If I weren't schizofrenic in sysadmin jobs I could add it myself, but I can't do
> that. Can I? I guess I can't.

You shouldn't be able to add yourself -- you're not listed as a godly
Python project admin, just as a mortal Python developer.  Many people
who do checkins routinely (like Neal Norwitz) are in the same boat wrt
that.

> Who am I again?

Thomas Wouters ;-)  Your SourceForge name in the Python project is twouters.

From tim.peters at gmail.com  Fri Jan 27 17:04:18 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 27 Jan 2006 11:04:18 -0500
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
	<20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl>
	<1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com>
	<20060127110053.GO5045@xs4all.nl>
	<1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com>
Message-ID: <1f7befae0601270804m3615014eyfbdfedafabf9bc75@mail.gmail.com>

I suppose another possibility for why twouters couldn't check in is
because someone added him to the project's cvs_acls script. If so, I
don't know anything about how to get that changed.

From tim.peters at gmail.com  Fri Jan 27 17:10:33 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Fri, 27 Jan 2006 11:10:33 -0500
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>
	<20060124111528.GB18916@xs4all.nl> <43D6AFC4.3030200@v.loewis.de>
	<20060125005918.GC18916@xs4all.nl> <20060126202456.GJ18916@xs4all.nl>
	<1f7befae0601261841g30a2994ak34fa7742d2971091@mail.gmail.com>
	<20060127110053.GO5045@xs4all.nl>
	<1f7befae0601270759o7bdaac5j1b6e630de4844d75@mail.gmail.com>
Message-ID: <1f7befae0601270810v1e0d5fb7v41b12c2c03769c89@mail.gmail.com>

[Tim]
...
> AFAICT, you (twouters) already have it.  There's a "Yes" in
> the twouters row under the "CVS Access" column on the Python project's
> Members admin page.  Have you tried checking in?  What happens when
> you do? ...

LOL -- what a bubblehead I am!  Whether you can check in has nothing
to do with SourceForge CVS access anymore -- never mind, and sorry.  I
don't have the power to admin svn.python.org either.  Martin?

From theller at python.net  Fri Jan 27 17:14:31 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 27 Jan 2006 17:14:31 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net>
	<20060126090822.GF18916@xs4all.nl> <ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<fyna4tme.fsf@python.net> <Pine.LNX.4.58.0601262119050.6219@alice>
Message-ID: <vew5lkbc.fsf@python.net>

John J Lee <jjl at pobox.com> writes:

> On Thu, 26 Jan 2006, Thomas Heller wrote:
> [...]
>> As I said in the other thread  (where the discussion should probably be
>> continued anyway):
>> 
>> http://mail.python.org/pipermail/python-dev/2006-January/060113.html
>> 
>> only aclocal.m4 isn't clear to me about the license.  Anyway, it could
>> be that this file isn't needed after all - I don't know enough about the
>> GNU toolchain to be sure.  Can anyone comment on this?
>
>>From 'info autoconf':
>
> |   The Autoconf macros are defined in several files.  Some of the files
> | are distributed with Autoconf; `autoconf' reads them first.  Then it
> | looks for the optional file `acsite.m4' in the directory that contains
> | the distributed Autoconf macro files, and for the optional file
> | `aclocal.m4' in the current directory.  Those files can contain your
> | site's or the package's own Autoconf macro definitions (*note Writing
> [...]
>
> So, I assume aclocal.m4 is under the same license as the rest of the
> libffi you're using.

I cannot uinderstand your reasoning.  How can 'info autoconf' incluence
the license of the aclocal.m4 file?  Or do I misunderstand something?

Given that all kind of *nix experts are here on this list - can someone
tell if aclocal.m4 is needed for building libffi at all or not?

Thomas


From theller at python.net  Fri Jan 27 18:03:58 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 27 Jan 2006 18:03:58 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <1138378937.3090.255.camel@localhost.localdomain> (Anthony
	Green's message of "Fri, 27 Jan 2006 08:22:17 -0800")
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
Message-ID: <lkx1k3gh.fsf_-_@python.net>

[I've added python-dev to cc:]

Anthony Green <green at redhat.com> writes:

> On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote:
>> Anyway, another question is: Is aclocal.m4 needed at all for building
>> (or maybe for regenerating the configure scripts), or is it optional?
>
> aclocal.m4 is required, but is only used as a build-time tool.  The fact
> that aclocal.m4 is distributed under the GPL should have no impact on
> the licensing terms used for software built using aclocal.m4.

If I understand correctly this means that the Python source distribution
would have to be GPL licensed, while the built programs would be able to
use another license.

I'm pretty sure this kills the whole idea (to include libffi in python).

Thanks, Thomas



From ronaldoussoren at mac.com  Fri Jan 27 18:04:03 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Fri, 27 Jan 2006 18:04:03 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <vew5lkbc.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<fyna4tme.fsf@python.net> <Pine.LNX.4.58.0601262119050.6219@alice>
	<vew5lkbc.fsf@python.net>
Message-ID: <8A7F07B4-8847-4D37-B96F-E9D86AC429C6@mac.com>


On 27-jan-2006, at 17:14, Thomas Heller wrote:

> John J Lee <jjl at pobox.com> writes:
>
>> On Thu, 26 Jan 2006, Thomas Heller wrote:
>> [...]
>>> As I said in the other thread  (where the discussion should  
>>> probably be
>>> continued anyway):
>>>
>>> http://mail.python.org/pipermail/python-dev/2006-January/060113.html
>>>
>>> only aclocal.m4 isn't clear to me about the license.  Anyway, it  
>>> could
>>> be that this file isn't needed after all - I don't know enough  
>>> about the
>>> GNU toolchain to be sure.  Can anyone comment on this?
>>
>>> From 'info autoconf':
>>
>> |   The Autoconf macros are defined in several files.  Some of the  
>> files
>> | are distributed with Autoconf; `autoconf' reads them first.   
>> Then it
>> | looks for the optional file `acsite.m4' in the directory that  
>> contains
>> | the distributed Autoconf macro files, and for the optional file
>> | `aclocal.m4' in the current directory.  Those files can contain  
>> your
>> | site's or the package's own Autoconf macro definitions (*note  
>> Writing
>> [...]
>>
>> So, I assume aclocal.m4 is under the same license as the rest of the
>> libffi you're using.
>
> I cannot uinderstand your reasoning.  How can 'info autoconf'  
> incluence
> the license of the aclocal.m4 file?  Or do I misunderstand something?
>
> Given that all kind of *nix experts are here on this list - can  
> someone
> tell if aclocal.m4 is needed for building libffi at all or not?

aclocal.m4 is needed to build configure, it's a library of configure  
fragments.

I try to stay away from configure as far as possible so cannot say if  
those
fragments are  really needed.

Ronald

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060127/73605721/attachment-0001.bin 

From theller at python.net  Fri Jan 27 19:32:54 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 27 Jan 2006 19:32:54 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <1138381908.3090.259.camel@localhost.localdomain> (Anthony
	Green's message of "Fri, 27 Jan 2006 09:11:48 -0800")
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
Message-ID: <zmlha5d5.fsf@python.net>

Anthony Green <green at redhat.com> writes:

> On Fri, 2006-01-27 at 18:03 +0100, Thomas Heller wrote:
>> [I've added python-dev to cc:]
>> 
>> Anthony Green <green at redhat.com> writes:
>> 
>> > On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote:
>> >> Anyway, another question is: Is aclocal.m4 needed at all for building
>> >> (or maybe for regenerating the configure scripts), or is it optional?
>> >
>> > aclocal.m4 is required, but is only used as a build-time tool.  The fact
>> > that aclocal.m4 is distributed under the GPL should have no impact on
>> > the licensing terms used for software built using aclocal.m4.
>> 
>> If I understand correctly this means that the Python source distribution
>> would have to be GPL licensed, while the built programs would be able to
>> use another license.
>> 
>> I'm pretty sure this kills the whole idea (to include libffi in python).
>
> I guess I wasn't clear.  aclocal.m4 is just a tool used to build libffi.
> Like your C compiler.  Bundling it with the Python source distribution
> should have no impact on the licensing of Python itself, since it isn't
> really part of the resulting Python binary - just like your C compiler
> isn't.

I guess I understood this already.  The difference to the C compiler is
that the compiler is not 'bundled' with Python, it is installed
separately.

Can anyone of the python-dev core team comment: can we live with the GPL
licensed aclocal.m4 file, in the source distribution and in SVN?

Thomas



From rasky at develer.com  Fri Jan 27 20:00:00 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Fri, 27 Jan 2006 20:00:00 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net>
Message-ID: <074501c62373$df526cd0$bf03030a@trilan>

Thomas Heller <theller at python.net> wrote:

>>>> On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote:
>>>>> Anyway, another question is: Is aclocal.m4 needed at all for building
>>>>> (or maybe for regenerating the configure scripts), or is it optional?
>>>>
>>>> aclocal.m4 is required, but is only used as a build-time tool.  The
fact
>>>> that aclocal.m4 is distributed under the GPL should have no impact on
>>>> the licensing terms used for software built using aclocal.m4.
>>>
>>> If I understand correctly this means that the Python source distribution
>>> would have to be GPL licensed, while the built programs would be able to
>>> use another license.
>>>
>>> I'm pretty sure this kills the whole idea (to include libffi in python).
>>
>> I guess I wasn't clear.  aclocal.m4 is just a tool used to build libffi.
>> Like your C compiler.  Bundling it with the Python source distribution
>> should have no impact on the licensing of Python itself, since it isn't
>> really part of the resulting Python binary - just like your C compiler
>> isn't.
>
> I guess I understood this already.  The difference to the C compiler is
> that the compiler is not 'bundled' with Python, it is installed
> separately.

That's no different. If you burn a CD containing a copy of the GCC and a
copy of a commercial software you are not violating any license. If you
distribute an .ISO file containing a copy of the GCC and a copy of a
commercial software, you are not violating any license. If you distribute a
.zip file containing a copy of GCC and a copy of a commercial software, you
are not violating any license.

There is an important difference between aggreggation of different programs,
and static/dynamic linking of parts. Autoconf is a build tool, and it does
not impose any license on the software you use it with. Plus some files have
the special exception from GPL so that even the files *generated* by
autoconf (and partly made of pieces of autoconf) do not require to be GPL.
This is exactly like GCC's runtime library (libgcc, and also libstdc++)
which are GPL with the special exception, and you can use them also for
commercial products.

Also, do not understimate previous history. There are zillions of programs
out there using Autconf and *not* being GPL.
-- 
Giovanni Bajo


From jjl at pobox.com  Fri Jan 27 21:43:20 2006
From: jjl at pobox.com (John J Lee)
Date: Fri, 27 Jan 2006 20:43:20 +0000 (UTC)
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <vew5lkbc.fsf@python.net>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<fyna4tme.fsf@python.net> <Pine.LNX.4.58.0601262119050.6219@alice>
	<vew5lkbc.fsf@python.net>
Message-ID: <Pine.LNX.4.58.0601272019150.6338@alice>

On Fri, 27 Jan 2006, Thomas Heller wrote:
> John J Lee <jjl at pobox.com> writes:
> 
> > On Thu, 26 Jan 2006, Thomas Heller wrote:
> >> only aclocal.m4 isn't clear to me about the license.  Anyway, it could
> >> be that this file isn't needed after all - I don't know enough about the
> >> GNU toolchain to be sure.  Can anyone comment on this?
> >
> >>From 'info autoconf':
> >
> > |   The Autoconf macros are defined in several files.  Some of the files
> > | are distributed with Autoconf; `autoconf' reads them first.  Then it
> > | looks for the optional file `acsite.m4' in the directory that contains
> > | the distributed Autoconf macro files, and for the optional file
> > | `aclocal.m4' in the current directory.  Those files can contain your
> > | site's or the package's own Autoconf macro definitions (*note Writing
> > [...]
> >
> > So, I assume aclocal.m4 is under the same license as the rest of the
> > libffi you're using.
> 
> I cannot uinderstand your reasoning.  How can 'info autoconf' incluence
> the license of the aclocal.m4 file?  Or do I misunderstand something?

OK.  I now notice I was confused as to why the license issue arose in the
first place, but FWIW: My point was just that the autoconf info pages
explain (if I read them right) that one keeps one's project-specific
autoconf m4 macros in a file named 'aclocal.m4'.  It's not a file
distributed with autoconf, it's just a file naming convention, so I made
the assumption that since libffi is apparently OK to include in Python, so
must be its aclocal.m4 (even if some of the macros in the libffi
aclocal.m4 originally derived from some other project).

But I'm afraid this would fail with an AssertionError if it weren't
pseudocode:

import legally_compatible as compat
from autoconf import acfiles
from ctypes import libffi

if compat(acfiles, libffi) and compat(libffi, python):
    assert compat(acfiles, python), "John is not legally competent"

:-(


John

From theller at python.net  Fri Jan 27 21:48:02 2006
From: theller at python.net (Thomas Heller)
Date: Fri, 27 Jan 2006 21:48:02 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu> (Andrew
	Pinski's message of "Fri, 27 Jan 2006 13:37:48 -0500")
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net>
	<B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
Message-ID: <slr9xurh.fsf@python.net>

Andrew Pinski <pinskia at physics.uc.edu> writes:

> Does phython already use autoconf? I think it does, if so then there
> should be no issues.

[Anthony Green]
>>> I guess I wasn't clear.  aclocal.m4 is just a tool used to build
>>> libffi.  Like your C compiler.  Bundling it with the Python source
>>> distribution should have no impact on the licensing of Python
>>> itself, since it isn't really part of the resulting Python binary -
>>> just like your C compiler isn't.

[Thomas Heller]
>> I guess I understood this already.  The difference to the C compiler is
>> that the compiler is not 'bundled' with Python, it is installed
>> separately.
>

"Giovanni Bajo" <rasky at develer.com> writes:
> That's no different. If you burn a CD containing a copy of the GCC and a
> copy of a commercial software you are not violating any license. If you
> distribute an .ISO file containing a copy of the GCC and a copy of a
> commercial software, you are not violating any license. If you distribute a
> .zip file containing a copy of GCC and a copy of a commercial software, you
> are not violating any license.
>
> There is an important difference between aggreggation of different programs,
> and static/dynamic linking of parts. Autoconf is a build tool, and it does
> not impose any license on the software you use it with. Plus some files have
> the special exception from GPL so that even the files *generated* by
> autoconf (and partly made of pieces of autoconf) do not require to be GPL.
> This is exactly like GCC's runtime library (libgcc, and also libstdc++)
> which are GPL with the special exception, and you can use them also for
> commercial products.
>
> Also, do not understimate previous history. There are zillions of programs
> out there using Autconf and *not* being GPL.

Ok, understood - there is no problem.  Hopefully the rest of the
pythondev team is also convinced.

The only question I have (maybe this is too off-topic, but since we're
here already): Nearly all the tools that autoconf and automake use are
careful to include an exception clause to the GPL license.  Why is
aclocal.m4 different?

Thomas



From stephen at xemacs.org  Fri Jan 27 21:50:04 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 28 Jan 2006 05:50:04 +0900
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <vew5lkbc.fsf@python.net> (Thomas Heller's message of "Fri, 27
	Jan 2006 17:14:31 +0100")
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<fyna4tme.fsf@python.net> <Pine.LNX.4.58.0601262119050.6219@alice>
	<vew5lkbc.fsf@python.net>
Message-ID: <871wyt75vn.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Thomas" == Thomas Heller <theller at python.net> writes:

    Thomas> I cannot uinderstand your reasoning.  How can 'info
    Thomas> autoconf' incluence the license of the aclocal.m4 file?

It doesn't.  The point is the documentation explains that all of the
other files are _part of autoconf_, and come under the license of
autoconf.  They come with special additional rights for users so that
the GPL does *not* apply to a project simply because the project uses
a configure script generated by autoconf.

The aclocal file is not part of autoconf and so doesn't come under
that license, but rather the license its author gives it.  Presumably
that is the same as the rest of the application (here libffi, since an
aclocal for libffi was almost certainly written by a libffi developer).

    Thomas> Given that all kind of *nix experts are here on this list
    Thomas> - can someone tell if aclocal.m4 is needed for building
    Thomas> libffi at all or not?

Not if libffi is distributed with a prebuilt configure script.

Otherwise, not using the distributed aclocal.m4 may be possible, but
it's a bad idea.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From jason.orendorff at gmail.com  Sat Jan 28 00:19:52 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Fri, 27 Jan 2006 18:19:52 -0500
Subject: [Python-Dev] / as path join operator
In-Reply-To: <87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>

It's controversial that Path subclasses str.  Some people think it's
flat-out wrong.  Even Bjorn argues that it's a practicality-vs-purity
tradeoff.  But a strong argument can be made that Path *should* be a
string subclass, practicality be damned.  Proof follows.

I. Here's an example of the sort of thing you might say if you did
*not* think of paths as strings:

On 1/25/06, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> I think it's logical to expect that
>     Path('home') / 'and/or'
> points to a file named "and/or" in directory "home", not to a file
> named "or" in directory "home/and".

This makes no sense whatsoever.  Ergo, by reductio ad absurdum, paths
are strings.

II. And here is the sort of thing you'd say if you thought of paths
*solely* as strings:

> (2) Note that '/' is also the path separator used by URIs, which RFC
> 2396 gives different semantics from Unix.  Most of my Python usage to
> date has been heavily web-oriented, and I'd have little use for /
> unless it follows RFC 2396.

The quandary is resolved by pointing out that URIs are not paths (in
the sense of os.path and generally this whole horrible thread).  Thus
not all strings are paths.

Hence the paths are a proper subset of the strings.  By the existence
of os.path, they have their own commonly-used operations.  By
definition, then, Path is a subclass of string, QED.


Do I really buy all this?  I dunno.  To say "paths aren't strings" is
all very well, and in a very abstract sense I almost agree--but you
have to admit it sort of flies in the face of, you know, reality. 
Filesystem paths are in fact strings on all operating systems I'm
aware of.  And it's no accident or performance optimization.  It's
good design.

-j

From martin at v.loewis.de  Sat Jan 28 00:25:12 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 00:25:12 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <zmlha5d5.fsf@python.net>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net>
Message-ID: <43DAABD8.5040404@v.loewis.de>

Thomas Heller wrote:
> Can anyone of the python-dev core team comment: can we live with the GPL
> licensed aclocal.m4 file, in the source distribution and in SVN?

My understanding that doing so would be in violation of section 2b) of
the GPL.

However, I still think it is possible to include libffi - we just have
to discard the build process, and do our own.

Regards,
Martin

From thomas at xs4all.net  Sat Jan 28 00:34:03 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Sat, 28 Jan 2006 00:34:03 +0100
Subject: [Python-Dev] / as path join operator
In-Reply-To: <bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
Message-ID: <20060127233403.GM18916@xs4all.nl>

On Fri, Jan 27, 2006 at 06:19:52PM -0500, Jason Orendorff wrote:

> To say "paths aren't strings" is all very well, and in a very abstract
> sense I almost agree--but you have to admit it sort of flies in the face
> of, you know, reality.  Filesystem paths are in fact strings on all
> operating systems I'm aware of.  And it's no accident or performance
> optimization.  It's good design.

The question isn't whether Path objects should *act* like strings. I haven't
seen anyone argue that they shouldn't, except for a few specific aspects,
like iteration, and those are argued on both sides of the subclassing camp.
The question is whether they should be actual subclasses of the Python
string type. As for what platforms do, if we want to stick to the platform
handling of paths, we change nothing. That's apparently not what people
want ;)

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From martin at v.loewis.de  Sat Jan 28 00:41:59 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 00:41:59 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <074501c62373$df526cd0$bf03030a@trilan>
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>
Message-ID: <43DAAFC7.2060603@v.loewis.de>

Giovanni Bajo wrote:
> That's no different. If you burn a CD containing a copy of the GCC and a
> copy of a commercial software you are not violating any license. If you
> distribute an .ISO file containing a copy of the GCC and a copy of a
> commercial software, you are not violating any license. If you distribute a
> .zip file containing a copy of GCC and a copy of a commercial software, you
> are not violating any license.

You misunderstand the GPL. Section 2b) is pretty clear that any
application that contains GPL-licensed code must be, itself, distributed
under the terms ofthe GPL

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

It further elaborates (in 2, below c) that you could distribute
"identifiable sections of [your] work [which] are not derived from the
Program" separately, as long as you omit the Program. But if you
distribute them as a whole, the whole must be licensed under the GPL.

> There is an important difference between aggreggation of different programs,
> and static/dynamic linking of parts.

There is an exception for "mere aggregation":

# In addition, mere aggregation of another work not based on the Program
# with the Program (or with a work based on the Program) on a volume of
# a storage or distribution medium does not bring the other work under
# the scope of this License.

However, inclusion of aclocal.m4 isn't "mere aggregation [...] on
a volume of storage". aclocal.m4 is an essential part of the software
as a whole, so 2b) applies.

> Also, do not understimate previous history. There are zillions of programs
> out there using Autconf and *not* being GPL.

Yes, and SCO is (rightly, IMO) complaining that zillions of open source
people do not honor IP properly. That doesn't mean Python should follow.
Instead, it means we need a build process for libffi which is
independent of autoconf (or convince the authors of aclocal.m4 to
relicense it, but that is likely futile).

As a matter of fact, Python itself uses autoconf, but without
aclocal.m4. autoconf generates configure for us, but configure is
not GPL-licensed, even though it is generated through autoconf:

# Copyright (C) 2003 Free Software Foundation, Inc.
# This configure script is free software; the Free Software Foundation
# gives unlimited permission to copy, distribute and modify it.

Regards,
Martin

From martin at v.loewis.de  Sat Jan 28 00:50:16 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 00:50:16 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <871wyt75vn.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>	<k6cne5d0.fsf@python.net>
	<20060126090822.GF18916@xs4all.nl>	<ek2vdvek.fsf@python.net>	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>	<fyna4tme.fsf@python.net>
	<Pine.LNX.4.58.0601262119050.6219@alice>	<vew5lkbc.fsf@python.net>
	<871wyt75vn.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <43DAB1B8.9060200@v.loewis.de>

Stephen J. Turnbull wrote:
> Otherwise, not using the distributed aclocal.m4 may be possible, but
> it's a bad idea.

That may not be so bad, actually. It looks like libffi's aclocal.m4
is not hand-written, but generated through aclocal(1). Not sure why
this is done, but this seems to be the cause of the explicit
mentionings of the GPL (the snippets that aclocal.m4 is generated
from are each GPL-licensed, make the whole aclocal.m4 GPL-licensed).

That said, I'm uncertain as to what the purpose of aclocal(1) is,
so I might be wrong.

Regards,
Martin

From martin at v.loewis.de  Sat Jan 28 00:53:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 00:53:39 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>	<k6cne5d0.fsf@python.net>
	<20060126090822.GF18916@xs4all.nl>	<ek2vdvek.fsf@python.net>	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
Message-ID: <43DAB283.70207@v.loewis.de>

Ronald Oussoren wrote:
> Merging the two configure files might be a good idea anyway, that  would
> take away the need to run configure from setup.py. IANAL, but I  don't
> quite get how a GPL'd support script, if there is such a thing,  in the
> build machinery of an extension library would require that  Python
> itself is GPL'd.

Section 2b) of the GPL. If a part of the program is GPL, the entire
program must be. Also, you must include the entire source of the
program, including all build scripts (section 3). So just including the
generated configure, and omitting some of its input, would also be a
license violation.

Regards,
Martin

From rasky at develer.com  Sat Jan 28 01:01:57 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 28 Jan 2006 01:01:57 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain><zmlha5d5.fsf@python.net><B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
	<slr9xurh.fsf@python.net>
Message-ID: <0b7a01c6239e$0e464910$12b52997@bagio>

Thomas Heller <theller at python.net> wrote:
>
> Andrew Pinski <pinskia at physics.uc.edu> writes:
>
>> Does phython already use autoconf? I think it does, if so then there
>> should be no issues.
>
> [Anthony Green]
>>>> I guess I wasn't clear.  aclocal.m4 is just a tool used to build
>>>> libffi.  Like your C compiler.  Bundling it with the Python source
>>>> distribution should have no impact on the licensing of Python
>>>> itself, since it isn't really part of the resulting Python binary -
>>>> just like your C compiler isn't.
>
> [Thomas Heller]
>>> I guess I understood this already.  The difference to the C
>>> compiler is
>>> that the compiler is not 'bundled' with Python, it is installed
>>> separately.
>>
>
> "Giovanni Bajo" <rasky at develer.com> writes:
>> That's no different. If you burn a CD containing a copy of the GCC
>> and a
>> copy of a commercial software you are not violating any license. If
>> you
>> distribute an .ISO file containing a copy of the GCC and a copy of a
>> commercial software, you are not violating any license. If you
>> distribute a .zip file containing a copy of GCC and a copy of a
>> commercial software, you
>> are not violating any license.
>>
>> There is an important difference between aggreggation of different
>> programs,
>> and static/dynamic linking of parts. Autoconf is a build tool, and
>> it does
>> not impose any license on the software you use it with. Plus some
>> files have
>> the special exception from GPL so that even the files *generated* by
>> autoconf (and partly made of pieces of autoconf) do not require to
>> be GPL.
>> This is exactly like GCC's runtime library (libgcc, and also
>> libstdc++)
>> which are GPL with the special exception, and you can use them also
>> for
>> commercial products.
>>
>> Also, do not understimate previous history. There are zillions of
>> programs
>> out there using Autconf and *not* being GPL.
>
> Ok, understood - there is no problem.  Hopefully the rest of the
> pythondev team is also convinced.
>
> The only question I have (maybe this is too off-topic, but since we're
> here already): Nearly all the tools that autoconf and automake use are
> careful to include an exception clause to the GPL license.  Why is
> aclocal.m4 different?

Is aclocal.m4 even GPL in the first place? I don't see such a notice in my
libffi copy.

Giovanni Bajo


From rasky at develer.com  Sat Jan 28 00:56:03 2006
From: rasky at develer.com (Giovanni Bajo)
Date: Sat, 28 Jan 2006 00:56:03 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain><zmlha5d5.fsf@python.net>
	<43DAABD8.5040404@v.loewis.de>
Message-ID: <0b6d01c6239d$3b92c570$12b52997@bagio>

Martin v. L?wis <martin at v.loewis.de> wrote:

>> Can anyone of the python-dev core team comment: can we live with the
>> GPL
>> licensed aclocal.m4 file, in the source distribution and in SVN?
>
> My understanding that doing so would be in violation of section 2b) of
> the GPL.

This would be a new interpretation of the license. The whole autotools chain is
GPL and it is used on way too many programs which are not GPL. They're so many
I won't even mention one. Anyway, IANAL, so if you're really concerned you can
mail the FSF and ask clarifications.

BTW, by your reading, libffi itself would be a GPL violation.

Giovanni Bajo


From martin at v.loewis.de  Sat Jan 28 01:06:11 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 01:06:11 +0100
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>
References: <43D9F90E.1040703@holdenweb.com>
	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>
Message-ID: <43DAB573.3040209@v.loewis.de>

Tim Peters wrote:
> Overall I'm not sure that's an improvement, but it was the best I
> could come up with 2 years ago when ZODB stopped using SF for
> downloads.

Please take a look at my attempt to solve that: putting a text just
above the button.

Regards,
Martin

From martin at v.loewis.de  Sat Jan 28 01:18:03 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 01:18:03 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <0b6d01c6239d$3b92c570$12b52997@bagio>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain><zmlha5d5.fsf@python.net>
	<43DAABD8.5040404@v.loewis.de>
	<0b6d01c6239d$3b92c570$12b52997@bagio>
Message-ID: <43DAB83B.2040808@v.loewis.de>

Giovanni Bajo wrote:
> This would be a new interpretation of the license. The whole autotools chain is
> GPL and it is used on way too many programs which are not GPL. They're so many
> I won't even mention one. Anyway, IANAL, so if you're really concerned you can
> mail the FSF and ask clarifications.

No. As Tom Tromey explains, significant parts of autoconf have a special
exception, allowing parts of autoconf to be used without any
restrictions when they get copied into, say, configure. If you only
use those parts of autoconf, you don't violate the GPL.

However, it appears that (at least parts of) aclocal.m4 originate from
code which doesn't have such a liberal license.

BTW, this interpretation isn't new at all. I first heard it in 1992.
People tend to reinterpret it because they can't believe the original
words, and then those interpretations become urban myth.

> BTW, by your reading, libffi itself would be a GPL violation.

Well, no. I interpret it to be dual-licensed: Identifiable parts of
it are independent work, and can be licensed independently of "the
Program (i.e. autoconf)". They meet all requirements of the GPL
(providing all code they need to provide) - it's just difficult
to find out what the license is.

Regards,
Martin

From bjourne at gmail.com  Sat Jan 28 01:29:39 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Sat, 28 Jan 2006 01:29:39 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <43D8FE24.1030500@egenix.com>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<43D8FE24.1030500@egenix.com>
Message-ID: <740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com>

[M.-A. Lemburg]
> I don't see why this is critical for the success of the Path
> object. I agree with Thomas that interfaces should be made
> compatible to Path object.

See the steps I mentioned. Unless step #1 is completed there is no way
to make the following code work:

    open(Path("foobar"))

Well, there is one alternative which is:

    open(Path("foobar").tostring())

And that is a Java-esque workaraound that I think noone would be happy
with.

> Why not ? We've added Unicode support to at least some
> file I/O APIs - adding support for instances which
> support the string interface shouldn't be all that
> difficult :-)

> BTW, if you're fine with this API:

> class File:
>    def __unicode__(self):
>        return u"test.txt"

> then the required change is minimal: we'd just need to
> use PyObject_Unicode() in getargs.c:837 and you should
> be set.

Alright! If that is true then maybe step #1 isn't as hard as I
thought. First problem is the "string interface." What exactly is the
string interface? It needs to be specified. I would have guessed that
it was equal to the stuff in the builtin str class.. :) Then you need
to modify Python's C code so that it accepts all objects implementing
the string interface. That may not be a trivial task [1]. Also don't
forget that Path probably also should work with:

        isinstance(Path('.'), basestring)

Because one of the big selling points of Path is that it acts as a
drop-in replacement for strings [2]. And you don't want to mess with
the current canonical way of testing if an object implements the
"string interface." But you aren't supposed to subclass basestring
[3].

All I'm saying is that I don't have the technical knowledge required
to modify Python to make Path work without subclassing string. If you,
or someone else, do then pretty please with sugar on top make a patch
so that Path doesn't have to subclass string.

Then you could have a much more pure Path without the "dead"
methods. But even then, one could argue that Jason Orendorffs original
Path module is still more practical (atleast for compatibility
reasons).

    def save_data(somedir, fpath, data):
        open(somedir + "/" + fpath.lower(), "w").write(data)

The above code isn't going to work unless fpath is a string (or an
object that inherits from string).  It isn't good code, but it isn't
extremely uncommon either.

1 http://mail.python.org/pipermail/python-dev/2006-January/059780.html
2 http://mail.python.org/pipermail/python-list/2005-July/291213.html
3 http://mail.python.org/pipermail/python-dev/2006-January/059782.html


--
mvh Bj?rn

From t-meyer at ihug.co.nz  Sat Jan 28 01:40:39 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Sat, 28 Jan 2006 13:40:39 +1300
Subject: [Python-Dev] / as path join operator
In-Reply-To: <bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
Message-ID: <C8119D80-69DF-4F1B-A2B3-C9B6B5895803@ihug.co.nz>

[Jason Orendorff]
> Filesystem paths are in fact strings on all operating systems I'm
> aware of.  And it's no accident or performance optimization.  It's
> good design.

Isn't that simply because filesystems aren't object orientated?  I  
can't call methods of a path through the filesystem.  There's a  
difference between a path, which is, yes, always (?) a string, and a  
Path object that provides convenient methods/properties.

(Maybe one of the experimental object-orientated file systems has non- 
string paths.  I have no idea).

=Tony.Meyer

From ncoghlan at gmail.com  Sat Jan 28 03:39:02 2006
From: ncoghlan at gmail.com (Nick Coghlan)
Date: Sat, 28 Jan 2006 12:39:02 +1000
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <43DAB573.3040209@v.loewis.de>
References: <43D9F90E.1040703@holdenweb.com>	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>
	<43DAB573.3040209@v.loewis.de>
Message-ID: <43DAD946.3060107@gmail.com>

Martin v. L?wis wrote:
> Tim Peters wrote:
>> Overall I'm not sure that's an improvement, but it was the best I
>> could come up with 2 years ago when ZODB stopped using SF for
>> downloads.
> 
> Please take a look at my attempt to solve that: putting a text just
> above the button.

Is it possible to make that URL a hyperlink?

It's also worth considering ways to make the alternate URL stand out more, as 
the green button is dominant enough that I'd expect a lot of people to miss 
the preceding text.

Something like:

" The Python programming language, an object-oriented scripting and rapid 
application development language.
Despite what the green button below says, you can <b>NOT</b> download it 
directly from Sourceforge, as SF is used only for bug tracking. You can get 
releases from the main Python website:

<a href="http://www.python.org/download">Download from python.org</a>

<useless green button is here>
"

Cheers,
Nick.

-- 
Nick Coghlan   |   ncoghlan at gmail.com   |   Brisbane, Australia
---------------------------------------------------------------
             http://www.boredomandlaziness.org

From nas at arctrix.com  Sat Jan 28 03:52:26 2006
From: nas at arctrix.com (Neil Schemenauer)
Date: Sat, 28 Jan 2006 02:52:26 +0000 (UTC)
Subject: [Python-Dev] Path inherits from string
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<dratnq$lbp$1@sea.gmane.org> <43D90014.10807@colorstudy.com>
Message-ID: <drem9a$q3v$1@sea.gmane.org>

Ian Bicking <ianb at colorstudy.com> wrote:
> OTOH, str(path) will break unicode filenames.  And unicode()
> breaks anything that simply desires to pass data through without
> effecting its encoding.

That general problem was the motivation for PEP 349.  Originally I
suggested adding a new built-in.  However, Guido's suggestion of
allowing str() to return unicode objects works okay too and that's
the way the current PEP is written.  With Python 2.5 you can do it
the ugly way using a str format (e.g. '%s' % obj).

  Neil


From rhamph at gmail.com  Sat Jan 28 06:49:50 2006
From: rhamph at gmail.com (Adam Olsen)
Date: Fri, 27 Jan 2006 22:49:50 -0700
Subject: [Python-Dev] / as path join operator
In-Reply-To: <bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
Message-ID: <aac2c7cb0601272149n4d81b508lbc6d562f25b189ce@mail.gmail.com>

On 1/27/06, Jason Orendorff <jason.orendorff at gmail.com> wrote:
> On 1/25/06, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> > I think it's logical to expect that
> >     Path('home') / 'and/or'
> > points to a file named "and/or" in directory "home", not to a file
> > named "or" in directory "home/and".
>
> This makes no sense whatsoever.  Ergo, by reductio ad absurdum, paths
> are strings.

It makes perfect sense to me.  However, since posix doesn't permit '/'
in file names I would expect it to emit an error:

Errors should never pass silently.
Unless explicitly silenced.
In the face of ambiguity, refuse the temptation to guess.

However, I'm not sure if the error be emitted when the Path is
created, or when it's passed to open().  The former implies a set of
OS-specific Path classes, and would allow subclassing from str.  The
latter allows (but does not require) a single universal Path class,
and that would prohibit subclassing from str (because you need a
higher-level representation to store path segments before converting
them to a platform-specific format.)

I'm -0 on subclassing str in the shortterm and -1 on it in the
longterm.  It's cruft and not something we want to be stuck with.

--
Adam Olsen, aka Rhamphoryncus

From stephen at xemacs.org  Sat Jan 28 07:56:27 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Sat, 28 Jan 2006 15:56:27 +0900
Subject: [Python-Dev] / as path join operator
In-Reply-To: <bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
	(Jason Orendorff's message of "Fri, 27 Jan 2006 18:19:52 -0500")
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
Message-ID: <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Jason" == Jason Orendorff <jason.orendorff at gmail.com> writes:

    Jason> I. Here's an example of the sort of thing you might say if
    Jason> you did *not* think of paths as strings:

[...]

    Jason> II. And here is the sort of thing you'd say if you thought
    Jason> of paths *solely* as strings:

Please note that my point was entirely different from trying to decide
whether to subclass strings.  My point was precisely that because of
this schizophrenia in the use of / as a path join operator in various
string representations of paths, it's a bad choice.  People are
naturally going to write buggy code because they don't have the
implemented spec in mind.

    Jason> Filesystem paths are in fact strings on all operating
    Jason> systems I'm aware of.

I have no idea what you could mean by that.  The data structure used
to represent a filesystem on all OS filesystems I've used is a graph
of directories and files.  A filesystem object is located by
traversing a path in that graph.

Of course there's a string representation, especially for human use,
but manipulating that representation as a string in programs is a
regular source of bugs.  In most cases, the graph is sufficiently
constrained that string manipulations are mostly accurate
representations of graph traversal, but not always, and you get
defects.

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From martin at v.loewis.de  Sat Jan 28 10:04:01 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 10:04:01 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>
Message-ID: <43DB3381.7040002@v.loewis.de>

Bill Northcott wrote:
> Quite so, but using the autotools does NOT include any GPL code in  the
> resulting program.

Hmm. Please take a look at

http://cvs.sourceforge.net/viewcvs.py/*checkout*/ctypes/ctypes/source/gcc/libffi/aclocal.m4?rev=1.1.4.1

This file contains a large number of licensing text blocks, many of
which read

# This program is free software; you can redistribute it and/or modify
# it under the terms of the GNU General Public License as published by
# the Free Software Foundation; either version 2, or (at your option)
# any later version.

So it seems to me that this specific generated aclocal.m4 *does*
include GPL code.

> So this does not apply.  All that is needed  is to
> include in the source distribution a copy of GPL, note that GPL  applies
> to some files in the sources and ensure that copyright  notices at the
> heads of GPL files are intact.

If nothing in the generated files is licensed under the terms of the
GPL, why would it be necessary to include a copy of the GPL?

> The compiler needs specific exemptions because parts of the GPLed 
> runtime libraries are included in all compiled code.  No part of the 
> autotools ends up in the finished code.  If it did, you would need m4 
> to run Python and you don't.

It doesn't matter whether it ends up in the finished code: if the
aclocal.m4 is indeed GPL-licensed, then the entire Python source
distribution must be GPL-licensed, because it "contains or
is derived from the Program or any part thereof".

Regards,
Martin

From martin at v.loewis.de  Sat Jan 28 11:05:12 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 11:05:12 +0100
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <43DAD946.3060107@gmail.com>
References: <43D9F90E.1040703@holdenweb.com>	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>	<43DAB573.3040209@v.loewis.de>
	<43DAD946.3060107@gmail.com>
Message-ID: <43DB41D8.4090300@v.loewis.de>

Nick Coghlan wrote:
> Is it possible to make that URL a hyperlink?

No, all HTML gets stripped/quoted as text.

> " The Python programming language, an object-oriented scripting and rapid 
> application development language.
> Despite what the green button below says, you can <b>NOT</b> download it 
> directly from Sourceforge, as SF is used only for bug tracking. You can get 
> releases from the main Python website:

Sorry, no HTML allowed there.

I created a support request to change/remove the button at

http://sourceforge.net/tracker/index.php?func=detail&aid=1417298&group_id=1&atid=200001

Regards,
Martin

From fredrik at pythonware.com  Sat Jan 28 11:07:30 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 28 Jan 2006 11:07:30 +0100
Subject: [Python-Dev] / as path join operator
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com><43D6BA85.8070007@colorstudy.com><bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com><740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com><63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz><Pine.LNX.4.58.0601252331300.6232@alice><d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com><87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp><bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
	<87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <drffp5$nkt$1@sea.gmane.org>

Stephen J. Turnbull wrote:

>     Jason> Filesystem paths are in fact strings on all operating
>     Jason> systems I'm aware of.
>
> I have no idea what you could mean by that.  The data structure used
> to represent a filesystem on all OS filesystems I've used is a graph
> of directories and files.  A filesystem object is located by
> traversing a path in that graph.
>
> Of course there's a string representation, especially for human use

if you define everything that can be identified by one or more well-defined
strings as a string, everything is a string.

</F>




From ronaldoussoren at mac.com  Sat Jan 28 11:06:01 2006
From: ronaldoussoren at mac.com (Ronald Oussoren)
Date: Sat, 28 Jan 2006 11:06:01 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <43DAB283.70207@v.loewis.de>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
Message-ID: <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>


On 28-jan-2006, at 0:53, Martin v. L?wis wrote:

> Ronald Oussoren wrote:
>> Merging the two configure files might be a good idea anyway, that   
>> would
>> take away the need to run configure from setup.py. IANAL, but I   
>> don't
>> quite get how a GPL'd support script, if there is such a thing,   
>> in the
>> build machinery of an extension library would require that  Python
>> itself is GPL'd.
>
> Section 2b) of the GPL. If a part of the program is GPL, the entire
> program must be. Also, you must include the entire source of the
> program, including all build scripts (section 3). So just including  
> the
> generated configure, and omitting some of its input, would also be a
> license violation.

You have a point there.  I'm not entirely convinced though, the argument
that Python would be a derived work of libffi's aclocal.m4 when libffi
were included in the Python repository seems very weak to me.

It is a good argument to just drop libffi's configure script and  
integrate
the functionality of it in the main python configure script. That would
remove all possible doubt and shouldn't be too much work.

BTW. The argument that the readline module should be GPL licensed seems
rather stronger, it's designed to work with a GPL-ed library and doesn't
work with a BSD licensed work-alike of that library.

Ronald


>
> Regards,
> Martin

-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2157 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20060128/c1a198bd/attachment-0001.bin 

From martin at v.loewis.de  Sat Jan 28 11:10:54 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 11:10:54 +0100
Subject: [Python-Dev] stabilizing builds
In-Reply-To: <20060126202456.GJ18916@xs4all.nl>
References: <ee2a432c0601222301s6ded02ccwe53f2adb79350da5@mail.gmail.com>	<20060124111528.GB18916@xs4all.nl>
	<43D6AFC4.3030200@v.loewis.de>	<20060125005918.GC18916@xs4all.nl>
	<20060126202456.GJ18916@xs4all.nl>
Message-ID: <43DB432E.20201@v.loewis.de>

Thomas Wouters wrote:
> I'd need developer access back to check it in, though. Unless anyone
> objects, of course :)

I copied ~/thomas/authorized_keys to ~pythondev/keys/thomas.wouters,
changed ownership/permissions, and ran make_authorized_keys in the
pythondev account. So you should have access now.

Regards,
Martin

From fredrik at pythonware.com  Sat Jan 28 11:36:22 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sat, 28 Jan 2006 11:36:22 +0100
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
References: <43D9F90E.1040703@holdenweb.com>	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>	<43DAB573.3040209@v.loewis.de><43DAD946.3060107@gmail.com>
	<43DB41D8.4090300@v.loewis.de>
Message-ID: <drfhf8$ri1$1@sea.gmane.org>

Martin v. Löwis wrote:

> Sorry, no HTML allowed there.
>
> I created a support request to change/remove the button at
>
> http://sourceforge.net/tracker/index.php?func=detail&aid=1417298&group_id=1&atid=200001

instead of spending more time and creativity on a sourceforge account
that's only used for tracking, how about moving the trackers to python.org ?

there are at least two "powered-by-python" alternatives that are vastly
better than source-forge's tracker: roundup, which just went 1.0:

    http://cheeseshop.python.org/pypi/roundup/1.0

and trac,

    http://www.edgewall.com/trac/

which almost everyone is using these days...

</F>




From martin at v.loewis.de  Sat Jan 28 11:42:02 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 11:42:02 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
Message-ID: <43DB4A7A.8070606@v.loewis.de>

Ronald Oussoren wrote:
> You have a point there.  I'm not entirely convinced though, the argument
> that Python would be a derived work of libffi's aclocal.m4 when libffi
> were included in the Python repository seems very weak to me.

The GPL says "contains or is derived from". Clearly, "identifiable
parts" of Python are not derived from aclocal.m4. However, the work
as a whole (i.e. the entire Python distribution) then becomes derived
work of libffi, and aclocal.m4.

> It is a good argument to just drop libffi's configure script and  integrate
> the functionality of it in the main python configure script. That would
> remove all possible doubt and shouldn't be too much work.

Several autoconf people have argued that this aclocal.m4 is a mistake,
and that the intention was that it shouldn't be GPL-licensed. So if we
cannot find a volunteer to rewrite the build process of libffi for
use in Python, that would also be a strategy (but one that probably
takes many months to complete).

I would personally drop the use of automake (and just rely on autoconf),
and then the need for aclocal would go away.

> BTW. The argument that the readline module should be GPL licensed seems
> rather stronger, it's designed to work with a GPL-ed library and doesn't
> work with a BSD licensed work-alike of that library.

This is the question what constitutes derivative work, and different
lawyers have said different things in the past. If we really want to
find out, we should ask a lawyer.

IANAL, and my understanding is that
a) we don't include readline in the Python distribution, so the
   Python source code cannot be derivative work. In U.S. copyright law,
   the term is apparently defined as

    "any . . . work [that] may be recast, transformed, or adapted. A
    work consisting of editorial revisions, annotations, elaborations,
    or other modifications which, as a whole, represents an original
    work of authorship, is a 'derivative work.'"

   http://www.chillingeffects.org/derivative/

   I would argue that Modules/readline.c does *not* represent the
   original work of authorship (i.e. libreadline).

   Of course, the GPL says "derived", not "derivative", and doesn't
   define the term, so you should ask your lawyer; ultimately, a
   court might decide what it means.

b) if we were to distribute binaries of Python, this issue would
   yet again be different: it seems that a binary readline module
   (builtin or not) is derivative work of the readline library,
   whether it is dynamically linked with that library or not.
   So anybody distributing such a binary might have to distribute
   it under the terms of the GPL.

   I say "might" because there is an exception in the GPL for
   major components normally distributed with the operating
   system. On those systems where Python's readline module is
   available, the readline library could be considered a part of
   the operating system. To be sure, ask your lawyer; ultimately,
   a court might decide whether this clause is relevant here.

Regards,
Martin

From mwh at python.net  Sat Jan 28 12:54:55 2006
From: mwh at python.net (Michael Hudson)
Date: Sat, 28 Jan 2006 11:54:55 +0000
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com> (
	=?iso-8859-1?q?BJ=F6rn_Lindqvist's_message_of?= "Sat,
	28 Jan 2006 01:29:39 +0100")
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<43D8FE24.1030500@egenix.com>
	<740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com>
Message-ID: <2mfyn8in3k.fsf@starship.python.net>

BJ?rn Lindqvist <bjourne at gmail.com> writes:

> [M.-A. Lemburg]
>> I don't see why this is critical for the success of the Path
>> object. I agree with Thomas that interfaces should be made
>> compatible to Path object.
>
> See the steps I mentioned. Unless step #1 is completed there is no way
> to make the following code work:
>
>     open(Path("foobar"))
> 
> Well, there is one alternative which is:
>
>     open(Path("foobar").tostring())
>
> And that is a Java-esque workaraound that I think noone would be happy
> with.

Now maybe I'm missing context here but: what on earth are you talking
about?  Of course there's a way to make "open(Path("foobar"))" work --
you change how the builtin open() works.

This post is not intended as arguing for or against anything, except
technical accuracy.

Cheers,
mwh

-- 
  Richard Gabriel was wrong: worse is not better, lying is better.
  Languages and systems succeed in the marketplace to the extent that
  their proponents lie about what they can do.
                                       -- Tim Bradshaw, comp.lang.lisp

From green at redhat.com  Fri Jan 27 18:11:48 2006
From: green at redhat.com (Anthony Green)
Date: Fri, 27 Jan 2006 09:11:48 -0800
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <lkx1k3gh.fsf_-_@python.net>
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
Message-ID: <1138381908.3090.259.camel@localhost.localdomain>

On Fri, 2006-01-27 at 18:03 +0100, Thomas Heller wrote:
> [I've added python-dev to cc:]
> 
> Anthony Green <green at redhat.com> writes:
> 
> > On Fri, 2006-01-27 at 17:08 +0100, Thomas Heller wrote:
> >> Anyway, another question is: Is aclocal.m4 needed at all for building
> >> (or maybe for regenerating the configure scripts), or is it optional?
> >
> > aclocal.m4 is required, but is only used as a build-time tool.  The fact
> > that aclocal.m4 is distributed under the GPL should have no impact on
> > the licensing terms used for software built using aclocal.m4.
> 
> If I understand correctly this means that the Python source distribution
> would have to be GPL licensed, while the built programs would be able to
> use another license.
> 
> I'm pretty sure this kills the whole idea (to include libffi in python).

I guess I wasn't clear.  aclocal.m4 is just a tool used to build libffi.
Like your C compiler.  Bundling it with the Python source distribution
should have no impact on the licensing of Python itself, since it isn't
really part of the resulting Python binary - just like your C compiler
isn't.

AG



From pinskia at physics.uc.edu  Fri Jan 27 19:37:48 2006
From: pinskia at physics.uc.edu (Andrew Pinski)
Date: Fri, 27 Jan 2006 13:37:48 -0500
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <zmlha5d5.fsf@python.net>
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net>
Message-ID: <B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>


On Jan 27, 2006, at 1:32 PM, Thomas Heller wrote:
>
> I guess I understood this already.  The difference to the C  
> compiler is
> that the compiler is not 'bundled' with Python, it is installed
> separately.
>
> Can anyone of the python-dev core team comment: can we live with  
> the GPL
> licensed aclocal.m4 file, in the source distribution and in SVN?

Does phython already use autoconf? I think it does, if so then there  
should
be no issues.

-- Pinski

From tromey at redhat.com  Sat Jan 28 01:04:25 2006
From: tromey at redhat.com (Tom Tromey)
Date: 27 Jan 2006 17:04:25 -0700
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <0b6d01c6239d$3b92c570$12b52997@bagio>
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <43DAABD8.5040404@v.loewis.de>
	<0b6d01c6239d$3b92c570$12b52997@bagio>
Message-ID: <m3u0bpp69i.fsf@localhost.localdomain>

>>>>> "Giovanni" == Giovanni Bajo <rasky at develer.com> writes:

Giovanni> This would be a new interpretation of the license. The whole
Giovanni> autotools chain is GPL and it is used on way too many
Giovanni> programs which are not GPL. They're so many I won't even
Giovanni> mention one. Anyway, IANAL, so if you're really concerned
Giovanni> you can mail the FSF and ask clarifications.

No, Martin is correct about this.  The output of autoconf is
explicitly *not* under the GPL, by design.  This is also true for the
m4 macros from automake -- again, an explicit decision.

The problem is, some other projects have not been so careful.

Tom

From tromey at redhat.com  Sat Jan 28 00:50:53 2006
From: tromey at redhat.com (Tom Tromey)
Date: 27 Jan 2006 16:50:53 -0700
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DAAFC7.2060603@v.loewis.de>
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
Message-ID: <m34q3pqlgi.fsf@localhost.localdomain>

>>>>> "Martin" == Martin v L?wis <martin at v.loewis.de> writes:

Martin> Instead, it means we need a build process for libffi which is
Martin> independent of autoconf (or convince the authors of aclocal.m4 to
Martin> relicense it, but that is likely futile).

Martin> As a matter of fact, Python itself uses autoconf, but without
Martin> aclocal.m4. autoconf generates configure for us, but configure is
Martin> not GPL-licensed, even though it is generated through autoconf:

Martin> # Copyright (C) 2003 Free Software Foundation, Inc.
Martin> # This configure script is free software; the Free Software Foundation
Martin> # gives unlimited permission to copy, distribute and modify it.

libffi's aclocal.m4 is created by the aclocal tool, which stitches it
together from different .m4 files it finds.

For m4 files that are part of the automake distribution, the intent
was explicitly to have the same more liberal permissions that apply to
'configure'.  If you can't find a statement saying this somewhere,
then I believe that to be a bug (I see one at the top of aclocal.m4
FWIW -- this applies to all the automake-shipped m4 files).  I know I
explicitly brought this up with RMS at some point in the distant past
and got the needed permissions to make this so.

However, libffi probably also uses macros from other places -- at
least GCC and libtool.  I don't know the legal status of these.

The GCC macros are probably not really necessary for your situation.
The libffi configury in the GCC tree is set up to build libffi as a
target library.  Most likely, you don't need this.

libffi/acinclude.m4 needs some license clarification.
It isn't clear who owns this code :-(

I think a real fix is probably not out of reach, should you wish to
pursue it.

Tom

From w.northcott at internode.on.net  Sat Jan 28 02:04:50 2006
From: w.northcott at internode.on.net (Bill Northcott)
Date: Sat, 28 Jan 2006 12:04:50 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DAAFC7.2060603@v.loewis.de>
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
Message-ID: <4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>

On 28/01/2006, at 10:41 AM, Martin v. L?wis wrote:
> You misunderstand the GPL. Section 2b) is pretty clear that any
> application that contains GPL-licensed code must be, itself,  
> distributed
> under the terms ofthe GPL

Quite so, but using the autotools does NOT include any GPL code in  
the resulting program.  So this does not apply.  All that is needed  
is to include in the source distribution a copy of GPL, note that GPL  
applies to some files in the sources and ensure that copyright  
notices at the heads of GPL files are intact.

The compiler needs specific exemptions because parts of the GPLed  
runtime libraries are included in all compiled code.  No part of the  
autotools ends up in the finished code.  If it did, you would need m4  
to run Python and you don't.

Cheers
Bill Northcott



From martin at v.loewis.de  Sat Jan 28 19:46:40 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 19:46:40 +0100
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <drfhf8$ri1$1@sea.gmane.org>
References: <43D9F90E.1040703@holdenweb.com>	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>	<43DAB573.3040209@v.loewis.de><43DAD946.3060107@gmail.com>	<43DB41D8.4090300@v.loewis.de>
	<drfhf8$ri1$1@sea.gmane.org>
Message-ID: <43DBBC10.6000601@v.loewis.de>

Fredrik Lundh wrote:
> instead of spending more time and creativity on a sourceforge account
> that's only used for tracking, how about moving the trackers to python.org ?

That's an old plan. It failed so far because no volunteer ever appeared
to make it happen (actually, some did appear, but didn't actually make
it happen).

Notice that any such moving would have to carry over the existing
tracker items; any volunteer should be available for hand-holding after
the move occurs. Groups of people would be better than individual
volunteers.

> there are at least two "powered-by-python" alternatives that are vastly
> better than source-forge's tracker: roundup, which just went 1.0:
> 
>     http://cheeseshop.python.org/pypi/roundup/1.0

Guido's preference is on roundup.

> and trac,
> 
>     http://www.edgewall.com/trac/
> 
> which almost everyone is using these days...

Barry Warsaw favours Jira as a tracker.

Regards,
Martin

From martin at v.loewis.de  Sat Jan 28 19:48:51 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sat, 28 Jan 2006 19:48:51 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
Message-ID: <43DBBC93.7010703@v.loewis.de>

Andrew Pinski wrote:
> Does phython already use autoconf? I think it does, if so then there  
> should be no issues.

Yes, but your conclusion is wrong. Python uses autoconf, but not
aclocal/automake. The generated configure is explicitly not covered by
the GPL; the status of the generated aclocal.m4 is unclear (it says
it is GPL'ed; it also says it isn't).

Regards,
Martin

From nyamatongwe at gmail.com  Sat Jan 28 22:42:41 2006
From: nyamatongwe at gmail.com (Neil Hodgson)
Date: Sun, 29 Jan 2006 08:42:41 +1100
Subject: [Python-Dev] / as path join operator
In-Reply-To: <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
	<87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <50862ebd0601281342r75566d1cke18b4b65554ff713@mail.gmail.com>

Stephen J. Turnbull:

>     Jason> Filesystem paths are in fact strings on all operating
>     Jason> systems I'm aware of.
>
> I have no idea what you could mean by that.  The data structure used
> to represent a filesystem on all OS filesystems I've used is a graph
> of directories and files.  A filesystem object is located by
> traversing a path in that graph.
>
> Of course there's a string representation, especially for human use,

   Not always. IIRC very old MacOS used an integer directory ID and a
string file name. The directory ID was a cookie that you received from
the UI and passed through to the file system and there was little
support for manipulating the directory ID. Textualized paths were
never supposed to be shown to users.

   Neil

From unknown_kev_cat at hotmail.com  Sat Jan 28 22:55:14 2006
From: unknown_kev_cat at hotmail.com (Joe Smith)
Date: Sat, 28 Jan 2006 16:55:14 -0500
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
References: <43D9F90E.1040703@holdenweb.com>
Message-ID: <drgp84$gt6$1@sea.gmane.org>


"Steve Holden" <steve at holdenweb.com> wrote in message 
news:43D9F90E.1040703 at holdenweb.com...
> You may be aware that Tim Parkin's work on our "next-generation" web
> presence has borne fruit in the shape of beta.python.org. While there's
> still a lot to be done Tim has given us a great start by creating a
> framework that makes it rather easier to manage content.
>

For the record, I'm just a user of the language. However, I tend to prefer 
the current site design over the new one.
But I also felt the same way when the Mozilla.org site was updated to be 
more 'friendly' (quite a while ago), and
have learned to live with it, so it is not a major problem.


>
> There's the further issue that the entire HTML body of
> http://svn.python.org/ currently reads
>
> <!-- not sure what to put here -->
>
> It would seem like the logical place to have pointers to Subversion
> software (downloads, documentation, operating instructions) together
> with an *annotated* summary of http://svn.python.org/projects/ and links
> back to the main developer site at the very least. I'm not even sure
> where that web content is under SVN control at the moment.

Also If there is any sort of webcvs software running, it may be ice to link 
to it from that page. 



From bjourne at gmail.com  Sun Jan 29 08:13:38 2006
From: bjourne at gmail.com (=?ISO-8859-1?Q?BJ=F6rn_Lindqvist?=)
Date: Sun, 29 Jan 2006 08:13:38 +0100
Subject: [Python-Dev] Path inherits from string
In-Reply-To: <2mfyn8in3k.fsf@starship.python.net>
References: <740c3aec0601260816u30d7f16fu4322fdbdfffcd758@mail.gmail.com>
	<43D8FE24.1030500@egenix.com>
	<740c3aec0601271629k238e90a4jed3fc189d75bea24@mail.gmail.com>
	<2mfyn8in3k.fsf@starship.python.net>
Message-ID: <740c3aec0601282313m3f671c0al142f350cdb586087@mail.gmail.com>

> > See the steps I mentioned. Unless step #1 is completed there is no way
> > to make the following code work:
> >
> >     open(Path("foobar"))
> >
> > Well, there is one alternative which is:
> >
> >     open(Path("foobar").tostring())
> >
> > And that is a Java-esque workaraound that I think noone would be happy
> > with.
>
> Now maybe I'm missing context here but: what on earth are you talking
> about?  Of course there's a way to make "open(Path("foobar"))" work --
> you change how the builtin open() works.

That's what I said: Someone has to make the required modifications to
the Python core. Changing how open() works would AFAIK be a
modification to the Python core. open() was just an example and
changing only how open() works  would not be sufficient I think.
__import__(), execfile() and third party extensions implemented in C
would also have to be modified so that they treat Path("foobar")
equivalent to "foobar."

--
mvh Bj?rn

From martin at v.loewis.de  Sun Jan 29 09:00:13 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jan 2006 09:00:13 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>
	<43DB3381.7040002@v.loewis.de>
	<B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>
Message-ID: <43DC760D.2020308@v.loewis.de>

Bill Northcott wrote:
> The build tools: m4 scripts, the configure shell script and the 
> Makefiles all contain GPL code and are under GPL.
> 
> However, none of this ends up in the 'finished program' which is the 
> executable versions of Python and its associated libraries.  The  build
> tools are just tools not part of the program.  The program is  not
> 'derived' from the build tools.

Again: What matters is what ends up in the source distribution,
http://www.python.org/ftp/python/2.4/Python-2.4.tgz

The source distribution would contain aclocal.m4; it would not
contain the autoconf/autoheader tools themselves.

The configure script is *NOT* under GPL, it explicitly says

# Copyright (C) 2003 Free Software Foundation, Inc.
# This configure script is free software; the Free Software Foundation
# gives unlimited permission to copy, distribute and modify it.

Likewise, the Makefile.in is not under GPL, either: Makefile.in says

# Copyright (C) 1994, 1995, 1996, 1997, 1998, 1999, 2000, 2001, 2002,
# 2003, 2004  Free Software Foundation, Inc.
# This Makefile.in is free software; the Free Software Foundation
# gives unlimited permission to copy and/or distribute it,
# with or without modifications, as long as this notice is preserved.

> Of course the computer on which I wrote the book has an operating 
> system which is copyright.  The word processing software I used to do 
> the writing is also copyright.  However none of either program ends  up
> in my novel.  So the novel is not derived from them, and their 
> copyright holders have no rights over the novel.

See, and this is precisely the difference. The word processor doesn't
end up in the distributed book. However, aclocal.m4 ends up in the
source distribution.

> I can happily package appropriately licensed copies of the word 
> processing software with the finished book.

Right: "appropriately licensed". Whether you *actually* can do this
depends on the license of the word processor. For example, if the
license says "not for resale", you cannot do this - even if you
are, yourself, licensed to use the copy you got.

So do I have permission to distribute copies of aclocal.m4? Yes,
the GPL allows this. Does it put additional obligations in doing
so? Yes, it does:

    b) You must cause any work that you distribute or publish, that in
    whole or in part contains or is derived from the Program or any
    part thereof, to be licensed as a whole at no charge to all third
    parties under the terms of this License.

Is (a future) Python-2.5.tgz "work that I distribute or publish"?
Yes, it is.

Does it "in whole or in part contain" "the Program or any part thereof"?
Yes, it does.

So I must "cause" Python-2.5.tgz "to be licensed as  whole at
no charge to all third parties under the terms of" the GPL.

> A Python binary is no more derived from the autotools than the book  is
> derived from the word processing software.

No, the Python binary isn't. The Python source is.

Regards,
Martin

From martin at v.loewis.de  Sun Jan 29 09:04:56 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jan 2006 09:04:56 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <ADA29999-DC79-4062-80C3-A96029B09E49@unsw.edu.au>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
	<43DBBC93.7010703@v.loewis.de>
	<ADA29999-DC79-4062-80C3-A96029B09E49@unsw.edu.au>
Message-ID: <43DC7728.3040801@v.loewis.de>

Bill Northcott wrote:
> What makes you think that?  I can see no such concession in the 
> autoconf source distribution.  A configure script is built up from  lots
> of code fragments out of the autoconf and automake M4 files, and  would
> clearly be covered by GPL.

No. As I just said in the other mail: The generated configure contains
the explicit permission:

# Copyright (C) 2003 Free Software Foundation, Inc.
# This configure script is free software; the Free Software Foundation
# gives unlimited permission to copy, distribute and modify it.

If you look at the autoconf sources, you will find various such blocks
(e.g. in lib/autoconf/general.m4):

# As a special exception, the Free Software Foundation gives unlimited
# permission to copy, distribute and modify the configure scripts that
# are the output of Autoconf.  You need not follow the terms of the GNU
# General Public License when using or distributing such scripts, even
# though portions of the text of Autoconf appear in them.  The GNU
# General Public License (GPL) does govern all other use of the material
# that constitutes the Autoconf program.

Regards,
Martin

From mwh at python.net  Sun Jan 29 10:11:38 2006
From: mwh at python.net (Michael Hudson)
Date: Sun, 29 Jan 2006 09:11:38 +0000
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DC760D.2020308@v.loewis.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "Sun,
	29 Jan 2006 09:00:13 +0100")
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>
	<43DB3381.7040002@v.loewis.de>
	<B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>
	<43DC760D.2020308@v.loewis.de>
Message-ID: <2m3bj7iek5.fsf@starship.python.net>

"Martin v. L?wis" <martin at v.loewis.de> writes:

> The source distribution would contain aclocal.m4; it would not
> contain the autoconf/autoheader tools themselves.

To a rather different point, do we need aclocal.m4 at all?  This is
the log for aclocal.m4:

------------------------------------------------------------------------
r34284 | anthonybaxter | 2003-09-27 11:12:27 +0200 (Sat, 27 Sep 2003) | 5 lines

fix for bug #811160 - autoconf vs. hp/ux system header files.
also applied to release23-maint.

Note that aclocal.m4 can go away when autoconf 2.58 is out.

------------------------------------------------------------------------

I think 2.58 actually had a brown-paper-bag release style bug, but
2.59 has been out for ages now.  If we were prepared to
AC_PREREQ(2.59), I think this whole issue could go away.

Cheers,
mwh

-- 
  Finding a needle in a haystack is a lot easier if you burn down
  the haystack and scan the ashes with a metal detector.
      -- the Silicon Valley Tarot (another one nicked from David Rush)

From martin at v.loewis.de  Sun Jan 29 10:54:56 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jan 2006 10:54:56 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <2m3bj7iek5.fsf@starship.python.net>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>	<43DAAFC7.2060603@v.loewis.de>	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>	<43DB3381.7040002@v.loewis.de>	<B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>	<43DC760D.2020308@v.loewis.de>
	<2m3bj7iek5.fsf@starship.python.net>
Message-ID: <43DC90F0.8060105@v.loewis.de>

Michael Hudson wrote:
> I think 2.58 actually had a brown-paper-bag release style bug, but
> 2.59 has been out for ages now.  If we were prepared to
> AC_PREREQ(2.59), I think this whole issue could go away.

It seems you are right, so I removed the file, and require ac 2.59.

Regards,
Martin

From hyeshik at gmail.com  Sun Jan 29 14:50:59 2006
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Sun, 29 Jan 2006 22:50:59 +0900
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DAABD8.5040404@v.loewis.de>
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <43DAABD8.5040404@v.loewis.de>
Message-ID: <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>

On 1/28/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Thomas Heller wrote:
> > Can anyone of the python-dev core team comment: can we live with the GPL
> > licensed aclocal.m4 file, in the source distribution and in SVN?
>
> My understanding that doing so would be in violation of section 2b) of
> the GPL.
>
> However, I still think it is possible to include libffi - we just have
> to discard the build process, and do our own.
>

I did some work to make ctypes+libffi compacter and liberal.
http://openlook.org/svnpublic/ctypes-compactffi/  (svn)

I removed sources/gcc and put sources/libffi copied from gcc 4.0.2.
And removed all automake-related build processes and integrated
them into setup.py. There's still aclocal.m4 in sources/libffi. But
it is just identical to libffi's acinclude.m4 which looks liberal.

Hye-Shik

From stephen at xemacs.org  Sun Jan 29 17:55:36 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 30 Jan 2006 01:55:36 +0900
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: =?iso-8859-1?q?=3C43DB4A7A=2E8070606=40v=2Eloewis=2Ede=3E_?=
	=?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Sat=2C_28_Jan_200?=
	=?iso-8859-1?q?6_11=3A42=3A02_+0100=22=29?=
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
Message-ID: <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v L?wis <martin at v.loewis.de> writes:

    >> BTW. The argument that the readline module should be GPL
    >> licensed seems rather stronger, it's designed to work with a
    >> GPL-ed library and doesn't work with a BSD licensed work-alike
    >> of that library.

    Martin> This is the question what constitutes derivative work, and
    Martin> different lawyers have said different things in the
    Martin> past. If we really want to find out, we should ask a
    Martin> lawyer.

You also need to ask about the cost of defending against a lawsuit by
the FSF, which is both the copyright holder of the library and the
primary advocate of the interpretation that a work which is intended
to be linked with another work is a derivative.  I think the FSF
pretty much would have to fight any claims that contest its
interpretation of the concept of "derived work", because any
interpretation that requires a direct source-to-source copy will make
the GPL irrelevant.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From martin at v.loewis.de  Sun Jan 29 19:09:24 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jan 2006 19:09:24 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>
References: <1137071518.12975.6.camel@devel002>	
	<m3zmlzzwn2.fsf@localhost.localdomain>	
	<1137186878.3098.87.camel@localhost.localdomain>	
	<wtglmz6e.fsf_-_@python.net>	
	<1138378937.3090.255.camel@localhost.localdomain>	
	<lkx1k3gh.fsf_-_@python.net>	
	<1138381908.3090.259.camel@localhost.localdomain>	
	<zmlha5d5.fsf@python.net> <43DAABD8.5040404@v.loewis.de>
	<4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>
Message-ID: <43DD04D4.8010601@v.loewis.de>

Hye-Shik Chang wrote:
> I did some work to make ctypes+libffi compacter and liberal.
> http://openlook.org/svnpublic/ctypes-compactffi/  (svn)
> 
> I removed sources/gcc and put sources/libffi copied from gcc 4.0.2.
> And removed all automake-related build processes and integrated
> them into setup.py. There's still aclocal.m4 in sources/libffi. But
> it is just identical to libffi's acinclude.m4 which looks liberal.

Well done! Would you like to derive a Python patch from that?
Don't worry about MSVC, yet, I will do that once the sources
are in the subversion.

(Of course, for due process, it would be better if this code gets
integrated into the official ctypes first, and then we incorporate
some named/versioned snapshot into /external, and svn cp it into
python/trunk from there).

Regards,
Martin

From martin at v.loewis.de  Sun Jan 29 19:20:31 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jan 2006 19:20:31 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <drj08c$sso$1@sea.gmane.org>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net><074501c62373$df526cd0$bf03030a@trilan>	<43DAAFC7.2060603@v.loewis.de>	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>	<43DB3381.7040002@v.loewis.de>	<B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>	<43DC760D.2020308@v.loewis.de><2m3bj7iek5.fsf@starship.python.net>	<43DC90F0.8060105@v.loewis.de>
	<drj08c$sso$1@sea.gmane.org>
Message-ID: <43DD076F.5030003@v.loewis.de>

Terry Reedy wrote:
>>>I think 2.58 actually had a brown-paper-bag release style bug, but
>>>2.59 has been out for ages now.  If we were prepared to
>>>AC_PREREQ(2.59), I think this whole issue could go away.
>>
>>It seems you are right, so I removed the file, and require ac 2.59.
> 
> 
> Does this mean that ctypes can and will be included with 2.5?

No; as Michael Hudson said: this is an entirely unrelated issue,
with Python's aclocal.m4 (which I deleted). It just occurred to
him that this had TODOs as well (from a technological view, not from
a licensing view).

However, Hye-Shik Chan is to praise for looking into the libffi
build process. So there is still a good chance that ctypes will
be in 2.5 (if he or somebody else follows up).

Regards,
Martin

From martin at v.loewis.de  Sun Jan 29 19:39:30 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Sun, 29 Jan 2006 19:39:30 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>	<k6cne5d0.fsf@python.net>
	<20060126090822.GF18916@xs4all.nl>	<ek2vdvek.fsf@python.net>	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>	<43DAB283.70207@v.loewis.de>	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <43DD0BE2.4050306@v.loewis.de>

Stephen J. Turnbull wrote:
> You also need to ask about the cost of defending against a lawsuit by
> the FSF, which is both the copyright holder of the library and the
> primary advocate of the interpretation that a work which is intended
> to be linked with another work is a derivative.  I think the FSF
> pretty much would have to fight any claims that contest its
> interpretation of the concept of "derived work", because any
> interpretation that requires a direct source-to-source copy will make
> the GPL irrelevant.

So would you just like to see the readline module to be removed from
the Python distribution?

I personally don't, because I don't believe that the status quo
conflicts with FSF's interpretation of the GPL, atleast not wrt.
to anything the PSF does (i.e. source and Windows distribution).

Also, I firmly believe that the FSF would *not* sue the PSF, but
instead first ask that the status is corrected.

Notice that the LGPL 2.1 somewhat elaborates on what the FSF
considers "derived" wrt. linking:

# When a program is linked with a library, whether statically or using a
# shared library, the combination of the two is legally speaking a
# combined work, a derivative of the original library. The ordinary
# General Public License therefore permits such linking only if the
# entire combination fits its criteria of freedom.

So it is the act of linking (and probably to some extent, the act
of compiling) that creates the derivative work.

Regards,
Martin

From tim.peters at gmail.com  Sun Jan 29 20:31:20 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Sun, 29 Jan 2006 14:31:20 -0500
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <43DD0BE2.4050306@v.loewis.de>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<20060126090822.GF18916@xs4all.nl> <ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
Message-ID: <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com>

[Martin v. L?wis]
> ...
> Also, I firmly believe that the FSF would *not* sue the PSF, but
> instead first ask that the status is corrected.

I'd say that's almost certain.  Like any organization with something
fuzzy to protect, the FSF has far more to lose than to gain by daring
a court to rule on their beliefs.  Of course the PSF is in a similar
boat:  both parties would view a lawsuit as a rock-bottom last resort.

I'm one PSF Director who would vote to capitulate in an instant to
avoid fighting the FSF over readline, and not because of legal
arguments, but because I respect their wishes about how _their_ work
can be used.  OTOH, in the absence of a statement from the FSF that
they're unhappy about Python's readline module, I wouldn't yank it
just to avoid a theoretical possibility that the FSF might complain
someday.

From vinay_sajip at yahoo.co.uk  Mon Jan 30 00:53:14 2006
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Sun, 29 Jan 2006 23:53:14 +0000 (UTC)
Subject: [Python-Dev] Extension to ConfigParser
References: <43D930E0.6070805@voidspace.org.uk>
Message-ID: <loom.20060130T004747-592@post.gmane.org>

Fuzzyman <fuzzyman <at> voidspace.org.uk> writes:

> In the past there has been some discussion about a new module to replace
> ConfigParser. Most notably at
> http://wiki.python.org/moin/ConfigParserShootout
[snip]
> It would be possible to extend to the ConfigObj API to be backwards
> compatible with ConfigParser. This would bring the added benefits of
> ConfigObj, without needing to add an extra module to the standard library.
> 
> Well nearly. ConfigObj supports config file schema with (optional) type
> conversion, through a companion module called validate. This could be
> included or left as an added option.
> 
> Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and
> if accepted, do the work - which is not inconsiderable).

Personally, I'd prefer to have the different candidates in the Shootout be
evaluated and a "winner" picked (I'm not sure who would do this, or when it
would be done). I'll readily declare an interest - I've implemented an
alternative hierarchical config module (which is in no way backward compatible
with ConfigParser), see

http://www.red-dove.com/python_config.html

Regards,

Vinay Sajip


From barry at python.org  Mon Jan 30 00:57:07 2006
From: barry at python.org (Barry Warsaw)
Date: Sun, 29 Jan 2006 18:57:07 -0500
Subject: [Python-Dev] SourceForge Download Page, Subversion Home Page
In-Reply-To: <43DBBC10.6000601@v.loewis.de>
References: <43D9F90E.1040703@holdenweb.com>
	<1f7befae0601270749w1dbb25f5lb785e7294c81fcff@mail.gmail.com>
	<43DAB573.3040209@v.loewis.de><43DAD946.3060107@gmail.com>
	<43DB41D8.4090300@v.loewis.de> <drfhf8$ri1$1@sea.gmane.org>
	<43DBBC10.6000601@v.loewis.de>
Message-ID: <1138579027.11097.13.camel@geddy.wooz.org>

On Sat, 2006-01-28 at 19:46 +0100, "Martin v. L?wis" wrote:

> Barry Warsaw favours Jira as a tracker.

Still do!  At at one time the Atlassian folks offered to help us import
the SF tracker data into Jira if we could get a machine readable
(hopefully XML-ish) dump of the current SF tracker data.  I don't know
if that offer still stands, but if Jira were in the running I would
contact them again about it.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060129/895a13fe/attachment-0001.pgp 

From stephen at xemacs.org  Mon Jan 30 07:39:06 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 30 Jan 2006 15:39:06 +0900
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: =?iso-8859-1?q?=3C43DD0BE2=2E4050306=40v=2Eloewis=2Ede=3E_?=
	=?iso-8859-1?q?=28Martin_v=2E_L=F6wis's_message_of_=22Sun=2C_29_Jan_200?=
	=?iso-8859-1?q?6_19=3A39=3A30_+0100=22=29?=
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<k6cne5d0.fsf@python.net> <20060126090822.GF18916@xs4all.nl>
	<ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
Message-ID: <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Martin" == Martin v L?wis <martin at v.loewis.de> writes:

    Martin> So would you just like to see the readline module to be
    Martin> removed from the Python distribution?

No.  I would much prefer that the readline module be made compatible
with libedit (or whatever the pseudo-readline library is called) and
that is what I recommend in the short term.  Alternatively, libedit
should be made more completely compatible with libreadline.

In the long term, I would like to see your interpretation established
in law (either by legislation or by precedent), but I certainly don't
want any precedent set by having the PSF and the FSF at odds with each
other.

    Martin> So it is the act of linking (and probably to some extent,
    Martin> the act of compiling) that creates the derivative work.

The FSF did not accept that position when Aladdin advanced it in the
case of Ghostscript.  Cf. my reply to Tim Peters.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From theller at python.net  Mon Jan 30 08:16:01 2006
From: theller at python.net (Thomas Heller)
Date: Mon, 30 Jan 2006 08:16:01 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>
	(Hye-Shik Chang's message of "Sun, 29 Jan 2006 22:50:59 +0900")
References: <1137071518.12975.6.camel@devel002>
	<m3zmlzzwn2.fsf@localhost.localdomain>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <43DAABD8.5040404@v.loewis.de>
	<4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>
Message-ID: <hd7mchji.fsf@python.net>

Hye-Shik Chang <hyeshik at gmail.com> writes:

> On 1/28/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Thomas Heller wrote:
>> > Can anyone of the python-dev core team comment: can we live with the GPL
>> > licensed aclocal.m4 file, in the source distribution and in SVN?
>>
>> My understanding that doing so would be in violation of section 2b) of
>> the GPL.
>>
>> However, I still think it is possible to include libffi - we just have
>> to discard the build process, and do our own.
>>
>
> I did some work to make ctypes+libffi compacter and liberal.
> http://openlook.org/svnpublic/ctypes-compactffi/  (svn)
>
> I removed sources/gcc and put sources/libffi copied from gcc 4.0.2.
> And removed all automake-related build processes and integrated
> them into setup.py. There's still aclocal.m4 in sources/libffi. But
> it is just identical to libffi's acinclude.m4 which looks liberal.

That's great!  Would you like to integrate these changes into to ctypes
CVS repository yourself?  I indend to do a few releases still from
there, before the first Python 2.5.  If so, please tell me your SF
username and I'll add you as developer.

Also, please note that all work should be done on the 'branch_1_0' CVS
branch - the HEAD is only experimental code (and not currently used).

Thomas



From stephen at xemacs.org  Mon Jan 30 09:51:11 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Mon, 30 Jan 2006 17:51:11 +0900
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com> (Tim
	Peters's message of "Sun, 29 Jan 2006 14:31:20 -0500")
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<20060126090822.GF18916@xs4all.nl> <ek2vdvek.fsf@python.net>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
	<1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com>
Message-ID: <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Tim" == Tim Peters <tim.peters at gmail.com> writes:

    Tim> [Martin v. L?wis]

    >> Also, I firmly believe that the FSF would *not* sue the PSF,
    >> but instead first ask that the status is corrected.

They would ask first.  That's what they did in the case of Aladdin
Ghostscript's use of readline.

    Tim> I'd say that's almost certain.  Like any organization with
    Tim> something fuzzy to protect, the FSF has far more to lose than
    Tim> to gain by daring a court to rule on their beliefs.  Of
    Tim> course the PSF is in a similar boat: both parties would view
    Tim> a lawsuit as a rock-bottom last resort.

Aladdin took a position similar to Martin's, and only yanked the
offending Makefile stanza when the FSF called them and said "we're
ready to go to court; are you?"

    Tim> I wouldn't yank it just to avoid a theoretical possibility
    Tim> that the FSF might complain someday.

It's not theoretical; it's almost identical to the Aladdin case.
Legally the PSF is, if anything, in a weaker position than Aladdin
(which did not distribute the module that interfaced to libreadline in
Ghostscript, but merely a makefile stanza that used it if it were
found).

-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From fuzzyman at voidspace.org.uk  Mon Jan 30 11:11:45 2006
From: fuzzyman at voidspace.org.uk (Fuzzyman)
Date: Mon, 30 Jan 2006 10:11:45 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <loom.20060130T004747-592@post.gmane.org>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
Message-ID: <43DDE661.4020808@voidspace.org.uk>

Vinay Sajip wrote:
> Fuzzyman <fuzzyman <at> voidspace.org.uk> writes:
>
>   
Hello Vinjay,
>> In the past there has been some discussion about a new module to replace
>> ConfigParser. Most notably at
>> http://wiki.python.org/moin/ConfigParserShootout
>>     
> [snip]
>   
>> It would be possible to extend to the ConfigObj API to be backwards
>> compatible with ConfigParser. This would bring the added benefits of
>> ConfigObj, without needing to add an extra module to the standard library.
>>
>> Well nearly. ConfigObj supports config file schema with (optional) type
>> conversion, through a companion module called validate. This could be
>> included or left as an added option.
>>
>> Anyway. If this stands a *chance* of acceptance, I'll write the PEP (and
>> if accepted, do the work - which is not inconsiderable).
>>     
>
> Personally, I'd prefer to have the different candidates in the Shootout be
> evaluated and a "winner" picked (I'm not sure who would do this, or when it
> would be done). 
Quite.

I'm suggesting an alternative that bypasses that tortuous and unlikely 
process. ;-)
> I'll readily declare an interest - I've implemented an
> alternative hierarchical config module (which is in no way backward compatible
> with ConfigParser), see
>
> http://www.red-dove.com/python_config.html
>
>   

I realise that there are several alternative modules available . 
Obviously my personal preference is ConfigObj (I'm not unbiased of 
course). :-)

Lack of complexity is the major feature I would tout here - I guess 
other people would have different priorities.

However, this not the only issue. Adding a new module, with a different 
API and possibly a different syntax for files, is a recipe for (some) 
confusion. Not to mention the difficulty of achieving a consensus on 
python-dev. (Again - ;-)

The resolution I'm suggesting means that people can continue to use 
ConfigParser, with major feature enhancements. *Or* they can migrate to 
a slightly different API that is easier to use - without needing to 
switch between incompatible modules.

I'm currently adding the ``ConfigParser.get*`` methods to ConfigObj 
(user request) and also adding full (straightforward) unicode support 
for reading and writing. These changes will be available in a beta 
release in the next few days.

Anyway, debate on the issue is welcomed.

All the best,


Fuzzyman
http://www.voidspace.org.uk/python/configobj.html
> Regards,
>
> Vinay Sajip
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>   

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20060130/e709ee5b/attachment.html 

From hyeshik at gmail.com  Mon Jan 30 11:38:19 2006
From: hyeshik at gmail.com (Hye-Shik Chang)
Date: Mon, 30 Jan 2006 19:38:19 +0900
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <hd7mchji.fsf@python.net>
References: <1137071518.12975.6.camel@devel002>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <43DAABD8.5040404@v.loewis.de>
	<4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>
	<hd7mchji.fsf@python.net>
Message-ID: <4f0b69dc0601300238l635c2c35v7a1e1bf88f394bda@mail.gmail.com>

On 1/30/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> Well done! Would you like to derive a Python patch from that?

Yup. I'll do.

On 1/30/06, Thomas Heller <theller at python.net> wrote:
> That's great!  Would you like to integrate these changes into to ctypes
> CVS repository yourself?  I indend to do a few releases still from
> there, before the first Python 2.5.  If so, please tell me your SF
> username and I'll add you as developer.
>
> Also, please note that all work should be done on the 'branch_1_0' CVS
> branch - the HEAD is only experimental code (and not currently used).

My SF username is perky. I'll try to make it portable but it'll
lose some platforms through compilers because few of libffi
sources are in preprocessed assembly (.S) which isn't supported
by distutils yet.  So, we'll need tests on various compiler/platforms
before the release.

Hye-Shik

From theller at python.net  Mon Jan 30 13:18:16 2006
From: theller at python.net (Thomas Heller)
Date: Mon, 30 Jan 2006 13:18:16 +0100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <4f0b69dc0601300238l635c2c35v7a1e1bf88f394bda@mail.gmail.com>
	(Hye-Shik Chang's message of "Mon, 30 Jan 2006 19:38:19 +0900")
References: <1137071518.12975.6.camel@devel002>
	<1137186878.3098.87.camel@localhost.localdomain>
	<wtglmz6e.fsf_-_@python.net>
	<1138378937.3090.255.camel@localhost.localdomain>
	<lkx1k3gh.fsf_-_@python.net>
	<1138381908.3090.259.camel@localhost.localdomain>
	<zmlha5d5.fsf@python.net> <43DAABD8.5040404@v.loewis.de>
	<4f0b69dc0601290550s4543d2c7tad7c3374b96ac4f1@mail.gmail.com>
	<hd7mchji.fsf@python.net>
	<4f0b69dc0601300238l635c2c35v7a1e1bf88f394bda@mail.gmail.com>
Message-ID: <wtghsyd3.fsf@python.net>

Hye-Shik Chang <hyeshik at gmail.com> writes:

> On 1/30/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
>> Well done! Would you like to derive a Python patch from that?
>
> Yup. I'll do.
>
> On 1/30/06, Thomas Heller <theller at python.net> wrote:
>> That's great!  Would you like to integrate these changes into to ctypes
>> CVS repository yourself?  I indend to do a few releases still from
>> there, before the first Python 2.5.  If so, please tell me your SF
>> username and I'll add you as developer.
>>
>> Also, please note that all work should be done on the 'branch_1_0' CVS
>> branch - the HEAD is only experimental code (and not currently used).
>
> My SF username is perky.

Added.

> I'll try to make it portable but it'll
> lose some platforms through compilers because few of libffi
> sources are in preprocessed assembly (.S) which isn't supported
> by distutils yet.  So, we'll need tests on various compiler/platforms
> before the release.

Isn't it sufficient to append ".S" to self.compiler.src_extensions?

Thanks,

Thomas



From gmccaughan at synaptics-uk.com  Mon Jan 30 14:56:44 2006
From: gmccaughan at synaptics-uk.com (Gareth McCaughan)
Date: Mon, 30 Jan 2006 13:56:44 +0000
Subject: [Python-Dev] / as path join operator (was: Re: The path module
	PEP)
In-Reply-To: <d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
Message-ID: <200601301356.44616.gmccaughan@synaptics-uk.com>

Steven Bethard wrote:

> My only fear with the / operator is that we'll end up with the same
> problems we have for using % in string formatting -- the order of
> operations might not be what users expect.  Since join is conceptually
> an addition-like operator, I would expect:
> 
>     Path('home') / 'a' * 5
> 
> to give me:
> 
>     home/aaaaa
> 
> If I understand it right, it would actually give me something like:
> 
>     home/ahome/ahome/ahome/ahome/a

Is there any very deep magic that says / is the One True Operator
to use for this, given that there's to be an operator for it? For
instance, & has lower correct precedence (so that Path('home') & 'a'*5
does something less unexpected), and doesn't look quite so much
as if it denotes arithmetic, and avoids semantic interference
from the idea that "/" should divide things or make them smaller.
(Though, for what it's worth, I think sticking another subdirectory
onto a path *is* dividing and making smaller: think of a path as
representing a subtree.) You do lose the pun on the Unix path separator,
which is a shame.

-- 
g


From anthony at interlink.com.au  Mon Jan 30 16:56:29 2006
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue, 31 Jan 2006 02:56:29 +1100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<43DD0BE2.4050306@v.loewis.de>
	<87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <200601310256.31337.anthony@interlink.com.au>

Rather than the back-n-forth about what the FSF might or might not do, 
can we just ask them for an official opinion and settle the matter?

The Aladdin case makes me think we should do this, probably before 2.5 
comes out - because if we do have to yank readline, I'd really not 
like to see this happen in a bugfix release, for fairly obvious 
reasons. 

Who would we need to talk to for a definitive answer? I'm sure there's 
various FSF mailing lists where we could get 157 different potential 
answers, but I'm talking about an actual, official FSF statement <0.5 
wink> 

Anthony

From jeff at taupro.com  Mon Jan 30 18:17:44 2006
From: jeff at taupro.com (Jeff Rush)
Date: Mon, 30 Jan 2006 11:17:44 -0600
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <200601310256.31337.anthony@interlink.com.au>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>	<43DD0BE2.4050306@v.loewis.de>	<87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp>
	<200601310256.31337.anthony@interlink.com.au>
Message-ID: <43DE4A38.1020904@taupro.com>

Anthony Baxter wrote:
> Rather than the back-n-forth about what the FSF might or might not do, 
> can we just ask them for an official opinion and settle the matter?
> 
> Who would we need to talk to for a definitive answer? I'm sure there's 
> various FSF mailing lists where we could get 157 different potential 
> answers, but I'm talking about an actual, official FSF statement <0.5 
> wink> 

 From http://www.fsf.org/licensing/licenses/index_html:

"If you want help choosing a license, evaluating a license, or have any
other questions about licenses, you can email us at <licensing at gnu.org>."

-Jeff

From ianb at colorstudy.com  Mon Jan 30 18:50:47 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 30 Jan 2006 11:50:47 -0600
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DDE661.4020808@voidspace.org.uk>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
Message-ID: <43DE51F7.4010007@colorstudy.com>

Fuzzyman wrote:
> The resolution I'm suggesting means that people can continue to use 
> ConfigParser, with major feature enhancements. *Or* they can migrate to 
> a slightly different API that is easier to use - without needing to 
> switch between incompatible modules.

I don't think enhancing ConfigParser significantly is a good way 
forward.  Because of ConfigParser's problems people have made all sorts 
of workarounds, and so I don't think there's any public interface that 
we can maintain while changing the internals without breaking lots of 
code.  In practice, everything is a public interface.  So I think the 
implementation as it stands should stay in place, and if anything it 
should be deprecated instead of being enhanced in-place.

Another class or module could be added that fulfills the documented 
interface to ConfigParser.  This would provide an easy upgrade path, 
without calling it a backward-compatible interface.  I personally would 
like if any new config system included a parser, and then an interface 
to the configuration that was read (ConfigParser is only the latter). 
Then people who want to do their own thing can work with just the 
parser, without crudely extending and working around the configuration 
interface.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From jason.orendorff at gmail.com  Mon Jan 30 19:06:16 2006
From: jason.orendorff at gmail.com (Jason Orendorff)
Date: Mon, 30 Jan 2006 13:06:16 -0500
Subject: [Python-Dev] / as path join operator
In-Reply-To: <87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
	<87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <bb8868b90601301006s4d9a6ea7y8221e713430499b4@mail.gmail.com>

On 1/28/06, Stephen J. Turnbull <stephen at xemacs.org> wrote:
> Please note that my point was entirely different from trying to decide
> whether to subclass strings.

Noted -- sorry I took you out of context there; that was careless.

>     Jason> Filesystem paths are in fact strings on all operating
>     Jason> systems I'm aware of.
>
> I have no idea what you could mean by that.  The data structure used
> to represent a filesystem on all OS filesystems I've used is a graph
> of directories and files.  A filesystem object is located by
> traversing a path in that graph.

You seem to think that because I said "operating systems", I'm talking
about kernel algorithms and such.  I'm not.  By "on all operating
systems" I really mean systems, not kernels:  system APIs, standard
tools, documentation, the conventions everyone follows--that sort of
thing.  Userspace.

Thought experiment:  How are filesystem paths used?  Well, programs
pass them into system calls like open() and chmod().  Programs use
them to communicate with other programs.  Users pass them to programs.
 Compare this to how you'd answer the question "How are integers
used?":  I think paths are used more for communication, less for
computation.  Their utility for communication is tightly bound to
their string-nature.

Essentially all APIs involving filesystem paths treat them as strings.
 It's not just that they take string parameters.  The way they're
designed, they encourage users to think of paths as strings, not
graph-paths.  Java's stdlib is the only API that even comes close to
distinguishing paths from strings.  The .NET class library doesn't
bother.  Many many people much smarter than I have thought about
creating non-string-oriented filesystem APIs.  Somehow it hasn't
caught on.

Essentially all users expect to see a filesystem path as a string of
characters in the conventional format.  Display it any other way (say,
as a sequence of edge-names) and you risk baffling them.

My position is (a) the convention that paths are strings really does
exist, embedded in the design and culture of the dominant operating
systems--in fact it's overwhelming, and I'm surprised anyone can miss
it; (b) there might be a reason for it, even aside from momentum.

-j

From w.northcott at internode.on.net  Sun Jan 29 01:57:37 2006
From: w.northcott at internode.on.net (Bill Northcott)
Date: Sun, 29 Jan 2006 11:57:37 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DB3381.7040002@v.loewis.de>
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>
	<43DB3381.7040002@v.loewis.de>
Message-ID: <B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>

On 28/01/2006, at 8:04 PM, Martin v. L?wis wrote:
>> The compiler needs specific exemptions because parts of the GPLed
>> runtime libraries are included in all compiled code.  No part of the
>> autotools ends up in the finished code.  If it did, you would need m4
>> to run Python and you don't.
>
> It doesn't matter whether it ends up in the finished code: if the
> aclocal.m4 is indeed GPL-licensed, then the entire Python source
> distribution must be GPL-licensed, because it "contains or
> is derived from the Program or any part thereof".

The build tools: m4 scripts, the configure shell script and the  
Makefiles all contain GPL code and are under GPL.

However, none of this ends up in the 'finished program' which is the  
executable versions of Python and its associated libraries.  The  
build tools are just tools not part of the program.  The program is  
not 'derived' from the build tools.

Maybe it would help to put it in a another domain:
Say I decide to write the great Australian novel.  I sit at my  
computer and type in the words.  Those words are my copyright and I  
am able to restrict their use as provided in copyright legislation.   
These rights are inherent and exist regardless of whether or not I  
add a copyright notice.  The notice is just a courtesy and a  
convenience which tells anyone who would like to use the work they  
need to get my permission.  A license is just a set of standard  
permissions passed on with the work.

Now I print out my novel.  That printed version is 'derived' from the  
representation in the computer.  So it is covered by the same  
copyright.  If I compress the computer files, those are still derived  
work.  If some American decides to write the great American novel by  
boiler-plating large chunks of my work with a little of his, then  
that work is derived and copyright to both of us.  His actions are  
legal unless he tries to publish it, which would require my  
permission.  This is analogous to linking a program with a library  
provided by another party.  That is why one needs the specific  
concessions in the gcc compiler to cover the linked run time libraries.

Of course the computer on which I wrote the book has an operating  
system which is copyright.  The word processing software I used to do  
the writing is also copyright.  However none of either program ends  
up in my novel.  So the novel is not derived from them, and their  
copyright holders have no rights over the novel.

I can happily package appropriately licensed copies of the word  
processing software with the finished book.  So that others can try  
their hand at the same thing.  In no way does such an operation give  
the software copyright holders any rights over the book.

This is exactly analogous to including the GPL tools with your source  
distribution.  You must comply with the GPL in respect of the GPL  
code.  So you must include copyright notices and and make any  
modifications or improvements freely available.  However, if you  
build a binary from the sources which does not include or link GPL  
code then the FSF have no rights over it and you are not obliged to  
acknowledge them or include their copyright notice.

A Python binary is no more derived from the autotools than the book  
is derived from the word processing software.

Bill Northcott

From w.northcott at unsw.edu.au  Sun Jan 29 02:04:58 2006
From: w.northcott at unsw.edu.au (Bill Northcott)
Date: Sun, 29 Jan 2006 12:04:58 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DBBC93.7010703@v.loewis.de>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
	<43DBBC93.7010703@v.loewis.de>
Message-ID: <ADA29999-DC79-4062-80C3-A96029B09E49@unsw.edu.au>


On 29/01/2006, at 5:48 AM, Martin v. L?wis wrote:
> Yes, but your conclusion is wrong. Python uses autoconf, but not
> aclocal/automake. The generated configure is explicitly not covered by
> the GPL;

What makes you think that?  I can see no such concession in the  
autoconf source distribution.  A configure script is built up from  
lots of code fragments out of the autoconf and automake M4 files, and  
would clearly be covered by GPL.

Bill Northcott

From w.northcott at unsw.edu.au  Sun Jan 29 02:14:10 2006
From: w.northcott at unsw.edu.au (Bill Northcott)
Date: Sun, 29 Jan 2006 12:14:10 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DBBC93.7010703@v.loewis.de>
References: <1137071518.12975.6.camel@devel002>	<m3zmlzzwn2.fsf@localhost.localdomain>	<1137186878.3098.87.camel@localhost.localdomain>	<wtglmz6e.fsf_-_@python.net>	<1138378937.3090.255.camel@localhost.localdomain>	<lkx1k3gh.fsf_-_@python.net>	<1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<B0887577-1048-46E8-A6FB-94871BE255A6@physics.uc.edu>
	<43DBBC93.7010703@v.loewis.de>
Message-ID: <BC8CEC09-BCE8-4C7C-9136-43AAAB6FE4DE@unsw.edu.au>

OTOH this (from python-sig-mac) is a worry if it is correct:

>     s> Apparently the readline library in MacOSX isn't really  
> readline.
>     s> It's a renamed libedit.  Not having encountered this deception
>     s> before, Python's build procedure doesn't know to test the  
> capability
>     s> of readline.  I suggest you just comment out the readline  
> checks in
>     s> setup.py.
>
>     Piet> libedit (editline) is a different library with a similar  
> function
>     Piet> as readline, but not compatible with it. It has a BSD  
> license. It
>     Piet> is often used instead of readline for non-GPL software.
>
> True.  I believe it was written by Rich Salz several years ago.   
> Python used
> to work with it, but eventually started using some readline  
> functionality
> which libedit doesn't provide.  I don't recall what that is.

If you distribute Python sources that link the GNU libreadline then  
all your code will be covered by GPL, because libreadline uses the  
full GPL not LGPL.  That is why Apple and other commercial vendors do  
not include it in their OS.

Bill Northcott

From w.northcott at unsw.edu.au  Sun Jan 29 02:19:14 2006
From: w.northcott at unsw.edu.au (Bill Northcott)
Date: Sun, 29 Jan 2006 12:19:14 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
References: <B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>
Message-ID: <99D8275D-F7A2-4863-82C3-9AFB8E3D9055@unsw.edu.au>

On 28/01/2006, at 8:04 PM, Martin v. L?wis wrote:
>> The compiler needs specific exemptions because parts of the GPLed
>> runtime libraries are included in all compiled code.  No part of the
>> autotools ends up in the finished code.  If it did, you would need m4
>> to run Python and you don't.
>
> It doesn't matter whether it ends up in the finished code: if the
> aclocal.m4 is indeed GPL-licensed, then the entire Python source
> distribution must be GPL-licensed, because it "contains or
> is derived from the Program or any part thereof".

The build tools: m4 scripts, the configure shell script and the  
Makefiles all contain GPL code and are under GPL.

However, none of this ends up in the 'finished program' which is the  
executable versions of Python and its associated libraries.  The  
build tools are just tools not part of the program.  The program is  
not 'derived' from the build tools.

Maybe it would help to put it in a another domain:
Say I decide to write the great Australian novel.  I sit at my  
computer and type in the words.  Those words are my copyright and I  
am able to restrict their use as provided in copyright legislation.   
These rights are inherent and exist regardless of whether or not I  
add a copyright notice.  The notice is just a courtesy and a  
convenience which tells anyone who would like to use the work they  
need to get my permission.  A license is just a set of standard  
permissions passed on with the work.

Now I print out my novel.  That printed version is 'derived' from the  
representation in the computer.  So it is covered by the same  
copyright.  If I compress the computer files, those are still derived  
work.  If some American decides to write the great American novel by  
boiler-plating large chunks of my work with a little of his, then  
that work is derived and copyright to both of us.  His actions are  
legal unless he tries to publish it, which would require my  
permission.  This is analogous to linking a program with a library  
provided by another party.  That is why one needs the specific  
concessions in the gcc compiler to cover the linked run time libraries.

Of course the computer on which I wrote the book has an operating  
system which is copyright.  The word processing software I used to do  
the writing is also copyright.  However none of either program ends  
up in my novel.  So the novel is not derived from them, and their  
copyright holders have no rights over the novel.

I can happily package appropriately licensed copies of the word  
processing software with the finished book.  So that others can try  
their hand at the same thing.  In no way does such an operation give  
the software copyright holders any rights over the book.

This is exactly analogous to including the GPL tools with your source  
distribution.  You must comply with the GPL in respect of the GPL  
code.  So you must include copyright notices and and make any  
modifications or improvements freely available.  However, if you  
build a binary from the sources which does not include or link GPL  
code then the FSF have no rights over it and you are not obliged to  
acknowledge them or include their copyright notice.

A Python binary is no more derived from the autotools than the book  
is derived from the word processing software.

Bill Northcott


From w.northcott at unsw.edu.au  Sun Jan 29 03:04:54 2006
From: w.northcott at unsw.edu.au (Bill Northcott)
Date: Sun, 29 Jan 2006 13:04:54 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
References: <ADA29999-DC79-4062-80C3-A96029B09E49@unsw.edu.au>
Message-ID: <7F4CBD32-9093-4F9C-AC4B-BFD2675C2885@unsw.edu.au>

On 29/01/2006, at 5:48 AM, Martin v. L?wis wrote:
> Yes, but your conclusion is wrong. Python uses autoconf, but not
> aclocal/automake. The generated configure is explicitly not covered by
> the GPL;

What makes you think that?  I can see no such concession in the  
autoconf source distribution.  A configure script is built up from  
lots of code fragments out of the autoconf and automake M4 files, and  
would clearly be covered by GPL.

Bill Northcott


From w.northcott at unsw.edu.au  Sun Jan 29 12:15:18 2006
From: w.northcott at unsw.edu.au (Bill Northcott)
Date: Sun, 29 Jan 2006 22:15:18 +1100
Subject: [Python-Dev] (libffi) Re: Copyright issue
In-Reply-To: <43DC760D.2020308@v.loewis.de>
References: <1137071518.12975.6.camel@devel002><m3zmlzzwn2.fsf@localhost.localdomain><1137186878.3098.87.camel@localhost.localdomain><wtglmz6e.fsf_-_@python.net><1138378937.3090.255.camel@localhost.localdomain><lkx1k3gh.fsf_-_@python.net><1138381908.3090.259.camel@localhost.localdomain>	<zmlha5d5.fsf@python.net>
	<074501c62373$df526cd0$bf03030a@trilan>
	<43DAAFC7.2060603@v.loewis.de>
	<4509FB68-6882-409E-9CFF-D5ADDA6ED210@internode.on.net>
	<43DB3381.7040002@v.loewis.de>
	<B6417D46-E45C-4397-BD0A-7F2959A352AD@internode.on.net>
	<43DC760D.2020308@v.loewis.de>
Message-ID: <CB1662FF-994F-4E26-8786-432D6E81156E@unsw.edu.au>

On 29/01/2006, at 7:00 PM, Martin v. L?wis wrote:

> Again: What matters is what ends up in the source distribution,
> http://www.python.org/ftp/python/2.4/Python-2.4.tgz

No really that is wrong.  What matters is what is in the Python  
executables, but you don't want to know that.  So I will bow out of  
the discussion.

Cheers
Bill

PS Linking with GNU libreadline really is a problem.


From guido at python.org  Mon Jan 30 20:20:18 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 11:20:18 -0800
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DE51F7.4010007@colorstudy.com>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com>
Message-ID: <ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>

On 1/30/06, Ian Bicking <ianb at colorstudy.com> wrote:
> I don't think enhancing ConfigParser significantly is a good way
> forward.  Because of ConfigParser's problems people have made all sorts
> of workarounds, and so I don't think there's any public interface that
> we can maintain while changing the internals without breaking lots of
> code.  In practice, everything is a public interface.  So I think the
> implementation as it stands should stay in place, and if anything it
> should be deprecated instead of being enhanced in-place.

Somehow that's not my experience. What's so bad about ConfigParser?
What would break if we rewrote the save functionality to produce a
predictable order?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Mon Jan 30 21:19:49 2006
From: fuzzyman at voidspace.org.uk (Fuzzyman)
Date: Mon, 30 Jan 2006 20:19:49 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
Message-ID: <43DE74E5.7000001@voidspace.org.uk>

Guido van Rossum wrote:
> On 1/30/06, Ian Bicking <ianb at colorstudy.com> wrote:
>   
>> I don't think enhancing ConfigParser significantly is a good way
>> forward.  Because of ConfigParser's problems people have made all sorts
>> of workarounds, and so I don't think there's any public interface that
>> we can maintain while changing the internals without breaking lots of
>> code.  In practice, everything is a public interface.  So I think the
>> implementation as it stands should stay in place, and if anything it
>> should be deprecated instead of being enhanced in-place.
>>     
>
> Somehow that's not my experience. What's so bad about ConfigParser?
> What would break if we rewrote the save functionality to produce a
> predictable order?
>   
Well, what I'm suggesting is not merely enhancing ConfigParser in place
- but replacing it with the ConfigObj and a compatibility layer to
support legacy code.

As I mentioned, it would require *some* changes to the parser
(ConfigObj) to be fully compatible with the existing syntax, but that is
well achievable. The aim would be that no existing code breaks, and few
(if any) config files break. I could flesh this out in a PEP if it was
likely that it would be accepted.

Issues with ConfigParser that ConfigObj *immediately* fixes :

* Simpler API - uses dictionary syntax/methods for
adding/deleting/accessing keys and sections (Each section - including
the ConfigObj itself - is a dictionary subclass with all methods available)
* Nested sub-sections to any depth.
* Fully integrated (extensible) validation schema system, with type
conversion. No complexity overhead for not using it.
* List values parsed by default (handling quotes sanely)
* Retains order of keys/section for iteration and writing out config file.
* Retains *all* comments and allows inline comments

The next enhancement will add full unicode support, which for a text
based format really makes sense and I should have implemented it earlier.

Additionally ConfigObj allows values in the root section - meaning that
for simple config files you aren't *forced* to have an arbitrary 'section'.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/configobj.html
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>   


From ianb at colorstudy.com  Mon Jan 30 21:23:16 2006
From: ianb at colorstudy.com (Ian Bicking)
Date: Mon, 30 Jan 2006 14:23:16 -0600
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	
	<loom.20060130T004747-592@post.gmane.org>	
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
Message-ID: <43DE75B4.7000502@colorstudy.com>

Guido van Rossum wrote:
>>I don't think enhancing ConfigParser significantly is a good way
>>forward.  Because of ConfigParser's problems people have made all sorts
>>of workarounds, and so I don't think there's any public interface that
>>we can maintain while changing the internals without breaking lots of
>>code.  In practice, everything is a public interface.  So I think the
>>implementation as it stands should stay in place, and if anything it
>>should be deprecated instead of being enhanced in-place.
> 
> 
> Somehow that's not my experience. What's so bad about ConfigParser?
> What would break if we rewrote the save functionality to produce a
> predictable order?

That's a fairly minor improvement, and I can't see how that would break 
anything.  But Michael (aka Fuzzyman -- sorry, I just can't refer to you 
as Fuzzyman without feeling absurd ;) was proposing ConfigObj 
specifically (http://www.voidspace.org.uk/python/configobj.html).  I 
assume the internals of ConfigObj bear no particular resemblence to 
ConfigParser, even if ConfigObj can parse the same syntax (plus some, 
and with different failure cases) and provide the same public API.

While ConfigParser is okay for simple configuration, it is (IMHO) not a 
very good basis for anyone who wants to build better systems, like 
config files that can be changed programmatically, or error messages 
that point to file and line numbers.  Those aren't necessarily features 
we need to expose in the standard library, but it'd be nice if you could 
implement that kind of feature without having to ignore the standard 
library entirely.

That said, I'm not particularly enthused about a highly featureful 
config file *format* in the standard library, even if I would like a 
much more robust implementation.

 From my light reading on ConfigObj, it looks like it satisfies my 
personal goals (though I haven't used it), but maybe has too many 
features, like nested sections.  And it seems like maybe the API can be 
reduced in size with a little high-level refactoring -- APIs generally 
grow over time so as to preserve backward compatibility, but I think if 
it was introduced into the standard library that might be an opportunity 
to trim the API back again before it enters the long-term API freeze 
that the standard library demands.

-- 
Ian Bicking  /  ianb at colorstudy.com  /  http://blog.ianbicking.org

From fuzzyman at voidspace.org.uk  Mon Jan 30 21:50:53 2006
From: fuzzyman at voidspace.org.uk (Fuzzyman)
Date: Mon, 30 Jan 2006 20:50:53 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DE51F7.4010007@colorstudy.com>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
Message-ID: <43DE7C2D.5060804@voidspace.org.uk>

Ian Bicking wrote:
> Fuzzyman wrote:
>> The resolution I'm suggesting means that people can continue to use
>> ConfigParser, with major feature enhancements. *Or* they can migrate
>> to a slightly different API that is easier to use - without needing
>> to switch between incompatible modules.
>
> I don't think enhancing ConfigParser significantly is a good way
> forward.  Because of ConfigParser's problems people have made all
> sorts of workarounds, and so I don't think there's any public
> interface that we can maintain while changing the internals without
> breaking lots of code.  In practice, everything is a public
> interface.  So I think the implementation as it stands should stay in
> place, and if anything it should be deprecated instead of being
> enhanced in-place.
>
> Another class or module could be added that fulfills the documented
> interface to ConfigParser.  This would provide an easy upgrade path,
> without calling it a backward-compatible interface.  I personally
> would like if any new config system included a parser, and then an
> interface to the configuration that was read (ConfigParser is only the
> latter). Then people who want to do their own thing can work with just
> the parser, without crudely extending and working around the
> configuration interface.
>
But to implement a parser you still need to agree a basic syntax.

For example ConfigObj supports list values, handles quotes sanely, uses
triple quotes for multiline values, allows inline comments, and has a
syntax for nested sections.

Once you have agreed these and a *basic* API spec for accessing the data
then you've implemented most of your config file handler.

In order to implement a write (save) method you also have to agree on
how to handle non string values (like lists).

There's not an *awful* lot left to do. In ConfigObj if you switch off
list handling when parsing and interpolation when fetching values, you
have access to the raw values and are free to extend value handling any
way you choose.

In my opinion dictionary syntax is the most natural way to represent a
config file - it is a data structure with named members. This means the
API for accessing the data - whether from a subclass that does
additional value processing or from code that just accesses the data.

I may be misunderstanding you of course (or more likely not fully
understanding). :-)

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

From fuzzyman at voidspace.org.uk  Mon Jan 30 22:02:59 2006
From: fuzzyman at voidspace.org.uk (Fuzzyman)
Date: Mon, 30 Jan 2006 21:02:59 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DE75B4.7000502@colorstudy.com>
References: <43D930E0.6070805@voidspace.org.uk>		<loom.20060130T004747-592@post.gmane.org>		<43DDE661.4020808@voidspace.org.uk>	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<43DE75B4.7000502@colorstudy.com>
Message-ID: <43DE7F03.1080202@voidspace.org.uk>

Ian Bicking wrote:
> Guido van Rossum wrote:
>   
>>> I don't think enhancing ConfigParser significantly is a good way
>>> forward.  Because of ConfigParser's problems people have made all sorts
>>> of workarounds, and so I don't think there's any public interface that
>>> we can maintain while changing the internals without breaking lots of
>>> code.  In practice, everything is a public interface.  So I think the
>>> implementation as it stands should stay in place, and if anything it
>>> should be deprecated instead of being enhanced in-place.
>>>       
>> Somehow that's not my experience. What's so bad about ConfigParser?
>> What would break if we rewrote the save functionality to produce a
>> predictable order?
>>     
>
> That's a fairly minor improvement, and I can't see how that would break 
> anything.  But Michael (aka Fuzzyman -- sorry, I just can't refer to you 
> as Fuzzyman without feeling absurd ;) 
Don't worry, absurdity is a common reaction I inspire in people.
Actually Fuzzyman is more of a description than a nickname. I understand
that I may not be the only one worthy of such a title of course. ;-)

> was proposing ConfigObj 
> specifically (http://www.voidspace.org.uk/python/configobj.html).  I 
> assume the internals of ConfigObj bear no particular resemblence to 
> ConfigParser, even if ConfigObj can parse the same syntax (plus some, 
> and with different failure cases) and provide the same public API.
>
> While ConfigParser is okay for simple configuration, it is (IMHO) not a 
> very good basis for anyone who wants to build better systems, like 
> config files that can be changed programmatically, or error messages 
> that point to file and line numbers.  Those aren't necessarily features 
> we need to expose in the standard library, but it'd be nice if you could 
> implement that kind of feature without having to ignore the standard 
> library entirely.
>
>   
Can you elaborate on what kinds of programattic changes you envisage ?
I'm just wondering if there are classes of usage not covered by
ConfigObj. Of course you can pretty much do anything to a ConfigObj
instance programattically, but even so...

> That said, I'm not particularly enthused about a highly featureful 
> config file *format* in the standard library, even if I would like a 
> much more robust implementation.
>
>   
I don't see how you can easily separate the format from the parser -
unless you just leave raw values. (As I said in the other email, I don't
think I fully understand you.)

If accessing raw values suits your purposes, why not subclass
ConfigParser and do magic in the get* methods ?

>  From my light reading on ConfigObj, it looks like it satisfies my 
> personal goals (though I haven't used it), but maybe has too many 
> features, like nested sections.  And it seems like maybe the API can be 
>   
I personally think nested sections are very useful and would be sad to
not see them included. Grouping additional configuration options as a
sub-section can be *very* handy.

> reduced in size with a little high-level refactoring -- APIs generally 
> grow over time so as to preserve backward compatibility, but I think if 
> it was introduced into the standard library that might be an opportunity 
> to trim the API back again before it enters the long-term API freeze 
> that the standard library demands.
Hmmm.... possibly. :-)

walk, decode and encode are the obvious extraneous methods. walk can be
re-implemented as a recursive function and encode/decode will be
unnecessary when unicode support is complete.

Additional complexity comes from retaining all comments and supporting
validation. I think retaining comments is worthwhile, because it allows
a program to modify a config file (e.g. from a GUI) without losing user
comments in a handwritten config file.

The validation support has several levels to it - but it can be
completely ignored with no complexity for the user. Some validation
support is integrated into instantiation, but this could *probably* be
broken out into a subclass.

Anyway, ho hum - possibly. All for future hypothetical discussion I guess...

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/configobj.html

From tim.peters at gmail.com  Mon Jan 30 22:44:47 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Mon, 30 Jan 2006 16:44:47 -0500
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
	<1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com>
	<873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp>
Message-ID: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com>

[Stephen J. Turnbull]
> ...
> Aladdin took a position similar to Martin's, and only yanked the
> offending Makefile stanza when the FSF called them and said "we're
> ready to go to court; are you?"

> ...

> It's not theoretical; it's almost identical to the Aladdin case.
> Legally the PSF is, if anything, in a weaker position than Aladdin
> (which did not distribute the module that interfaced to libreadline in
> Ghostscript, but merely a makefile stanza that used it if it were
> found).

I'm not making myself clear.  The FSF (like most organizations,
including the PSF) has agendas that are more aptly described as
political than as legal -- legalities are more of a club to try to
force what they want.  If the FSF merely _says_ they want to make it
difficult, or even impossible, to link Python and/or Python apps with
GNU readline, that's fine by me, and they don't have to make up
creative license readings or threats to get that.  I'm happy to accede
to their wishes in the matter regardless of what the license says, or
is claimed to say.

OTOH, I have no reason to _presume_ that this is their hoped-for
outcome wrt Python, neither to presume that the politics shaping their
tussle with Aladdin are relevant to the PSF.  "The law" is rarely
applied uniformly, in large part because it usually is means rather
than end.  Python is a high-profile project that hasn't been hiding
its readline module, and if I presume anything here it's that the FSF
would have complained by now if they really didn't want this.

In the meantime, I'm satisfied that the people involved with Python's
readline module acted on good-faith readings of the licenses involved.
 If someone cares enough to rip it out, and/or to supply an
alternative, fine by me too (but it won't _be_ me).

From skip at pobox.com  Mon Jan 30 22:17:28 2006
From: skip at pobox.com (skip at pobox.com)
Date: Mon, 30 Jan 2006 15:17:28 -0600
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
Message-ID: <17374.33384.775422.175195@montanaro.dyndns.org>


    Guido> What's so bad about ConfigParser?

My guess is that people find its limited nesting, well, limiting.  Python
containers being the arbitrary nesting little devils that they are, I can
understand the desire to push the envelope.

It's my opinion that ConfigParser should stay pretty much as it is other
than perhaps adding round trip capability.  It is pretty good at reading and
writing Windows INI files, which was what it was designed to do.  My guess
is there are a fair number of people who wouldn't want to lose that.  Mixing
INI file handling with something more powerful seems doomed.  If we want
more sophisticated functionality a new module should be written, or one of
the existing shootout candidates on the Wiki should be chosen and perhaps
enhanced.  I have a horse in that race (actually, it's more like a pony).

Skip

From guido at python.org  Mon Jan 30 22:56:37 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 13:56:37 -0800
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <17374.33384.775422.175195@montanaro.dyndns.org>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
Message-ID: <ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>

On 1/30/06, skip at pobox.com <skip at pobox.com> wrote:
>
>     Guido> What's so bad about ConfigParser?
>
> My guess is that people find its limited nesting, well, limiting.  Python
> containers being the arbitrary nesting little devils that they are, I can
> understand the desire to push the envelope.

To the contrary. Config file syntax should be limited by design so as
to avoid confusing the people who have to edit them without knowing
Python, and if an app is successful, that's most of the app's users.
(If the app is only targeted at Python users, why bother with a config
file? You could just have one designated .py file where all the
configuration code is concentrated, like faqwizard's faqconf.py.)

> It's my opinion that ConfigParser should stay pretty much as it is other
> than perhaps adding round trip capability.  It is pretty good at reading and
> writing Windows INI files, which was what it was designed to do.  My guess
> is there are a fair number of people who wouldn't want to lose that.  Mixing
> INI file handling with something more powerful seems doomed.  If we want
> more sophisticated functionality a new module should be written, or one of
> the existing shootout candidates on the Wiki should be chosen and perhaps
> enhanced.  I have a horse in that race (actually, it's more like a pony).

Aha. I am beginning to understand. When people say "ConfigParser is
hopeless" they mean ".INI files are hopeless". I happen to disagree.
(There's also a meme that says that every aspect of an app should be
configurable. I disagree with that too. As Joel Spolski points out in
his book on UI design, most configuration options are signs of
indecisive developers, and most users never change the defaults.)

Regarding the claim that dicts are the best kind of API for
configuration, I disagree -- the dict API doesn't maintain order, and
doesn't provide access to comments and other metadata. And it has many
operations that make little sense for configuration option values.

(And would Michael please start signing with his full name, since he
doesn't seem to care much for true anonymity? It's okay if your email
says

  From: Michael So-and-so <fuzzyman at wherever.com>

It's suboptimal if it says

  From: Fuzzyman <fuzzyman at wherever.com>

You might also start signing your posts "Michael So-and-so" rather
than the alias. I don't sign my email "BDFL" either. :-)  Since there
are many Michaels, signing just "Michael" doesn't really help. I'm too
old fogey to get used to using IRC handles to refer to people; I can
never keep the mapping in my head.)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fuzzyman at voidspace.org.uk  Mon Jan 30 23:17:10 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 30 Jan 2006 22:17:10 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
Message-ID: <43DE9066.2040100@voidspace.org.uk>

Guido van Rossum wrote:
> On 1/30/06, skip at pobox.com <skip at pobox.com> wrote:
>   
>>     Guido> What's so bad about ConfigParser?
>>
>> My guess is that people find its limited nesting, well, limiting.  Python
>> containers being the arbitrary nesting little devils that they are, I can
>> understand the desire to push the envelope.
>>     
>
> To the contrary. Config file syntax should be limited by design so as
> to avoid confusing the people who have to edit them without knowing
> Python, and if an app is successful, that's most of the app's users.
> (If the app is only targeted at Python users, why bother with a config
> file? You could just have one designated .py file where all the
> configuration code is concentrated, like faqwizard's faqconf.py.)
>
>   
>> It's my opinion that ConfigParser should stay pretty much as it is other
>> than perhaps adding round trip capability.  It is pretty good at reading and
>> writing Windows INI files, which was what it was designed to do.  My guess
>> is there are a fair number of people who wouldn't want to lose that.  Mixing
>> INI file handling with something more powerful seems doomed.  If we want
>> more sophisticated functionality a new module should be written, or one of
>> the existing shootout candidates on the Wiki should be chosen and perhaps
>> enhanced.  I have a horse in that race (actually, it's more like a pony).
>>     
>
> Aha. I am beginning to understand. When people say "ConfigParser is
> hopeless" they mean ".INI files are hopeless". I happen to disagree.
> (There's also a meme that says that every aspect of an app should be
> configurable. I disagree with that too. As Joel Spolski points out in
> his book on UI design, most configuration options are signs of
> indecisive developers, and most users never change the defaults.)
>
>   
But like it or not, configuration files are often used to store data
about what a program does - not just the UI options. Storing this in a
human readable and editable format is of great advantage.

Yes, a lot of the entries will never be modified by a user - but many
will need to be edited and read by a developer. In a production
environment importing from python modules is not likely to be safe.

I don't dislike the INI format. ``key = value`` is intuitive. Nesting of
sections is a logical extension that provides additional flexibility
without breaking backwards compatibility. It needn't be added
complexity, because you don't need to use it. There are plenty of people
who will though.

> Regarding the claim that dicts are the best kind of API for
> configuration, I disagree -- the dict API doesn't maintain order, and
> doesn't provide access to comments and other metadata. And it has many
> operations that make little sense for configuration option values.
>
>   
An ordered dictionary is a fairly trivial modification of the dictionary
(if you don't mind the performance hit), and yes there is other metadata
also needs to be stored. Hey, since Python 2.2 some folk put a lot of
work in allowing us to subclass dict. ;-)

Perhaps a better way of saying this is that the dictionary API is an
intuitive way to access config file data (including modifying and
creating).  You are mapping values to names, it's a logical match.

As soon as you generate/manipulate config files programattically a lot
of the dictionary methods become useful. Perhaps not all though.
``get``, ``update``, ``has_key`` are all useful.

> (And would Michael please start signing with his full name, since he
> doesn't seem to care much for true anonymity? It's okay if your email
> says
>
>   From: Michael So-and-so <fuzzyman at wherever.com>
>
>   
Michael So-and-so ??? Hey, that's fighting talk. ;-)

> It's suboptimal if it says
>
>   From: Fuzzyman <fuzzyman at wherever.com>
>
> You might also start signing your posts "Michael So-and-so" rather
> than the alias. I don't sign my email "BDFL" either. :-)  Since there
> are many Michaels, signing just "Michael" doesn't really help. I'm too
> old fogey to get used to using IRC handles to refer to people; I can
> never keep the mapping in my head.)
>   
Wow, my first rebuke from the BDFL. May it be the first of many. ;-)

I've changed this client, but I use this account from multiple
locations. Forgive me if I forget to change others (but feel free to
remind me).

Michael Foord
(Chastened but Fuzzy nonetheless.)

> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>   


From guido at python.org  Mon Jan 30 23:18:25 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 14:18:25 -0800
Subject: [Python-Dev] [Doc-SIG] that library reference, again
In-Reply-To: <05663E18-CDFB-4B18-A515-BAA21C48C1FD@lag.net>
References: <dope79$flv$2@sea.gmane.org> <dp0qrh$r78$1@sea.gmane.org>
	<4335d2c40512291238l6b1d44b7vaf0963b631f3bd89@mail.gmail.com>
	<dp1jd8$p4a$1@sea.gmane.org> <43B4A238.2030101@python.org>
	<51A7717B-3832-477E-BED5-3C36DCA20336@lag.net>
	<dp2mmh$45b$1@sea.gmane.org>
	<05663E18-CDFB-4B18-A515-BAA21C48C1FD@lag.net>
Message-ID: <ca471dc20601301418g761af34dme4d0a6a4a46cf3ac@mail.gmail.com>

On 1/26/06, Robey Pointer <robey at lag.net> wrote:
[quoting /F]
> > moving all of (or parts of) the reference documentation in to the
> > library source code would be an alternative, of course [1], but that
> > would basically mean starting over from scratch.

Bad idea. Putting the full docs in the source adds more coupling
between the files and makes it harder instead of easier to edit the
docs, especially for C code. Also, the needs for interactive help are
different from those for off-line docs.

Also note that several object types (e.g. lists and dicts, like used
for sys.path and sys.modules) don't have __doc__ properties so
documenting them in the source is problematic.

The list goes on; e.g. the source code ordering may not be the same as
the ideal documentation ordering; where do you put subsection
introductions, etc.

> I think that would be a good thing to do no matter what markup is
> used.  It always irritates me when I do 'help(sys.something)' and get
> one line of ASCII art that's probably useful to the module author but
> nobody else.

Wrong guess. That string was put there by the author specifically so
that it shows up in help(sys.whatever), and the author probably
(mistakenly) though that whatever he wrote was sufficient. If you ever
find a docstring that is IYO insufficient, please submit a patch or
bug. Until we have a better mechanism, simply submitting the new text
of the docstring, without bothering to make it into a context-diff or
to use the proper markup, is fine.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From thomas at xs4all.net  Mon Jan 30 23:23:21 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Mon, 30 Jan 2006 23:23:21 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
	2006-01-15
In-Reply-To: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com>
References: <BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
	<1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com>
	<873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp>
	<1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com>
Message-ID: <20060130222321.GP18916@xs4all.nl>

On Mon, Jan 30, 2006 at 04:44:47PM -0500, Tim Peters wrote:

> Python is a high-profile project that hasn't been hiding its readline
> module, and if I presume anything here it's that the FSF would have
> complained by now if they really didn't want this.

In fact, we can be absolutely certain the FSF knows (or people in charge
have known, rather) of the readline module, since it was an important reason
to get Python's license GPL-compatible. Had they, at that time, considered
code-that-wraps-libreadline a problem, they would have made that known. As
it is, it was clear that the resulting binary (of non-GPL-compatible source
and a GPL library) was a GPL violation, but the source itself was not.

Not that they can't change their minds, of course. But I wouldn't go and ask
them if they changed their minds yet; it might make them ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From guido at python.org  Mon Jan 30 23:40:04 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 14:40:04 -0800
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DE9066.2040100@voidspace.org.uk>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
	<43DE9066.2040100@voidspace.org.uk>
Message-ID: <ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>

> Guido van Rossum wrote:
> > Aha. I am beginning to understand. When people say "ConfigParser is
> > hopeless" they mean ".INI files are hopeless". I happen to disagree.
> > (There's also a meme that says that every aspect of an app should be
> > configurable. I disagree with that too. As Joel Spolski points out in
> > his book on UI design, most configuration options are signs of
> > indecisive developers, and most users never change the defaults.)

On 1/30/06, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> But like it or not, configuration files are often used to store data
> about what a program does - not just the UI options. Storing this in a
> human readable and editable format is of great advantage.
>
> Yes, a lot of the entries will never be modified by a user - but many
> will need to be edited and read by a developer. In a production
> environment importing from python modules is not likely to be safe.

Ah. This definitely isn't what ConfigParser was meant to do. I'd think
for this you should use some kind of XML pickle though. That's
horrible if end users must edit it, but great for saving
near-arbitrary persistent data in a readable and occasionally editable
(for the developer) form.

You should probably not mix the end user config data with the
program's persistent store.

> I don't dislike the INI format. ``key = value`` is intuitive. Nesting of
> sections is a logical extension that provides additional flexibility
> without breaking backwards compatibility. It needn't be added
> complexity, because you don't need to use it. There are plenty of people
> who will though.

And I'm trying to say that (for config files) they probably shouldn't,
because most of their users won't understand it. Of course this
doesn't apply to your use case, so maybe we're just violently agreeing
-- except you seem to be confusing matters by talking about
"configuration" (which suggests end-user-editable) when you really
mean "persistent state" (which is under full program control).

> > Regarding the claim that dicts are the best kind of API for
> > configuration, I disagree -- the dict API doesn't maintain order, and
> > doesn't provide access to comments and other metadata. And it has many
> > operations that make little sense for configuration option values.
> >
> An ordered dictionary is a fairly trivial modification of the dictionary
> (if you don't mind the performance hit), and yes there is other metadata
> also needs to be stored. Hey, since Python 2.2 some folk put a lot of
> work in allowing us to subclass dict. ;-)

Yeah, but that was more as a matter of principle than to encourage that use...

Narrow APIs are typically better than wide APIs, and the dict API is
the ultimate wide API. (I guess strings are just as bad; I haven't
contributed to that thread yet but I think the new path module should
not inherit from str or unicode either.)

> Perhaps a better way of saying this is that the dictionary API is an
> intuitive way to access config file data (including modifying and
> creating).  You are mapping values to names, it's a logical match.
>
> As soon as you generate/manipulate config files programattically a lot
> of the dictionary methods become useful. Perhaps not all though.
> ``get``, ``update``, ``has_key`` are all useful.

has_key is obsolete though -- use 'in'.

Anyway, we have a name for this -- we call it the "mapping protocol".
In most cases it's better to implement a protocol than inherit a
standard object. (This is perhaps unique to Python -- other OO
languages often encourage inheritance, so I can understand the
confusion.)

> Wow, my first rebuke from the BDFL. May it be the first of many. ;-)
>
> I've changed this client, but I use this account from multiple
> locations. Forgive me if I forget to change others (but feel free to
> remind me).
>
> Michael Foord
> (Chastened but Fuzzy nonetheless.)

Thanks much!

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Mon Jan 30 23:50:20 2006
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon, 30 Jan 2006 23:50:20 +0100
Subject: [Python-Dev] Extension to ConfigParser
References: <43D930E0.6070805@voidspace.org.uk><loom.20060130T004747-592@post.gmane.org><43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com><ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com><17374.33384.775422.175195@montanaro.dyndns.org><ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com><43DE9066.2040100@voidspace.org.uk>
	<ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
Message-ID: <drm57e$isn$1@sea.gmane.org>

Guido van Rossum wrote:

> Ah. This definitely isn't what ConfigParser was meant to do. I'd think
> for this you should use some kind of XML pickle though. That's
> horrible if end users must edit it, but great for saving
> near-arbitrary persistent data in a readable and occasionally editable
> (for the developer) form.

fwiw, I've *never* used INI files to store program state, and I've
never used the save support in ConfigParser.

on the other hand, earlier today, Firefox messed up my user profile,
and I'm not sure I had managed to sort that out if they hadn't used
an INI file to keep track of available profiles...

(especially not if they'd used Mork ;-)

</F>




From fuzzyman at voidspace.org.uk  Tue Jan 31 00:05:13 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 30 Jan 2006 23:05:13 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	
	<loom.20060130T004747-592@post.gmane.org>	
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>	
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	
	<17374.33384.775422.175195@montanaro.dyndns.org>	
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>	
	<43DE9066.2040100@voidspace.org.uk>
	<ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
Message-ID: <43DE9BA9.3040807@voidspace.org.uk>

Guido van Rossum wrote:
>
> On 1/30/06, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>   
>> But like it or not, configuration files are often used to store data
>> about what a program does - not just the UI options. Storing this in a
>> human readable and editable format is of great advantage.
>>
>> Yes, a lot of the entries will never be modified by a user - but many
>> will need to be edited and read by a developer. In a production
>> environment importing from python modules is not likely to be safe.
>>     
>
> Ah. This definitely isn't what ConfigParser was meant to do. I'd think
> for this you should use some kind of XML pickle though. That's
> horrible if end users must edit it, but great for saving
> near-arbitrary persistent data in a readable and occasionally editable
> (for the developer) form.
>
> You should probably not mix the end user config data with the
> program's persistent store.
>
>   
I have a feeling that for many complex applications the developer is the
user. Agreed that giving complex configuration data along with program
data to the end-user is not good.

Despite this, data storage that needs to be edited by developers and
system administration is a very common usage.

I'm saying that for this purpose (up to a point) an INI file like syntax
is much *better* than XML.

>> I don't dislike the INI format. ``key = value`` is intuitive. Nesting of
>> sections is a logical extension that provides additional flexibility
>> without breaking backwards compatibility. It needn't be added
>> complexity, because you don't need to use it. There are plenty of people
>> who will though.
>>     
>
> And I'm trying to say that (for config files) they probably shouldn't,
> because most of their users won't understand it. Of course this
> doesn't apply to your use case, so maybe we're just violently agreeing
> -- except you seem to be confusing matters by talking about
> "configuration" (which suggests end-user-editable) when you really
> mean "persistent state" (which is under full program control).
>
>   
I'm not really sure there is such a clear distinction. Especially for
programs without a GUI, things like IP addresses, network details, paths
to data files, plugged in components, device names, (etc) seem to be both.

>>> Regarding the claim that dicts are the best kind of API for
>>> configuration, I disagree 
[snip..]
>>  Hey, since Python 2.2 some folk put a lot of
>> work in allowing us to subclass dict. ;-)
>>     
>
> Yeah, but that was more as a matter of principle than to encourage that use...
>
> Narrow APIs are typically better than wide APIs, and the dict API is
> the ultimate wide API. (I guess strings are just as bad; I haven't
> contributed to that thread yet but I think the new path module should
> not inherit from str or unicode either.)
>
>   
I am a firm fan of the new path module but agnostic on whether it should
inherit from basestring in some way. Wrong thread though.

[snip...]
>
> Anyway, we have a name for this -- we call it the "mapping protocol".
> In most cases it's better to implement a protocol than inherit a
> standard object. (This is perhaps unique to Python -- other OO
> languages often encourage inheritance, so I can understand the
> confusion.)
>
>   
Mapping protocol is fine. Subclassing dict has the advantage that some
aspects of the API are already implemented in C.

Anyway - I modify my statement accordingly. For accessing configuration
type data, the mapping protocol is an intuitive match. Named members
mapping to values. Whether this is done via subclassing dict is an
implementation detail. (Although I guess it implies that the full
dictionary API will be available and you seem to be saying that you
would rather that *only* relevant methods are available).

All the best,

Uhm... Michael Foord I guess




From t-meyer at ihug.co.nz  Mon Jan 30 23:56:09 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue, 31 Jan 2006 11:56:09 +1300
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <17374.33384.775422.175195@montanaro.dyndns.org>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
Message-ID: <37371F19-FA7B-470F-ABB4-32B9E7E65A85@ihug.co.nz>

[Guido]
>> What's so bad about ConfigParser?

[Skip Montanaro]
> It's my opinion that ConfigParser should stay pretty much as it is  
> other
> than perhaps adding round trip capability.
[...]
> If we want more sophisticated functionality a new module should be  
> written,
> or one of the existing shootout candidates on the Wiki should be  
> chosen and perhaps
> enhanced.

+1.

=Tony.Meyer


From guido at python.org  Tue Jan 31 00:15:45 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 15:15:45 -0800
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DE9BA9.3040807@voidspace.org.uk>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
	<43DE9066.2040100@voidspace.org.uk>
	<ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
	<43DE9BA9.3040807@voidspace.org.uk>
Message-ID: <ca471dc20601301515u346390c4ned3805db213acdce@mail.gmail.com>

On 1/30/06, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
> I have a feeling that for many complex applications the developer is the
> user. Agreed that giving complex configuration data along with program
> data to the end-user is not good.
>
> Despite this, data storage that needs to be edited by developers and
> system administration is a very common usage.
>
> I'm saying that for this purpose (up to a point) an INI file like syntax
> is much *better* than XML.

Sure. But for the same reasons, arbitrary nesting is probably bad.

> I'm not really sure there is such a clear distinction. Especially for
> programs without a GUI, things like IP addresses, network details, paths
> to data files, plugged in components, device names, (etc) seem to be both.

Just try to keep it simple.

I still think it's a typical programmer's mistake to assume that
there's no clear distinction. For a user who's looking to change one
of the three simple properties that make sense for him to configure,
having to hunt these down in a file containing hundreds of stuff she
shouldn't tuch is a pain, plus there's the risk that an occasional
slip of the finger causes damage to stuff they don't understand.

> [snip...]
> Mapping protocol is fine. Subclassing dict has the advantage that some
> aspects of the API are already implemented in C.

And many you don't need. You'll never be able to change the API
because it's constrained by all existing uses, which means every dict
API you've never heard of.

If you're using this so much that speed's an issue perhaps you're overusing it.

> Anyway - I modify my statement accordingly. For accessing configuration
> type data, the mapping protocol is an intuitive match. Named members
> mapping to values. Whether this is done via subclassing dict is an
> implementation detail. (Although I guess it implies that the full
> dictionary API will be available and you seem to be saying that you
> would rather that *only* relevant methods are available).

I can see several improvements on the dict protocol. For example, a
handy API would be soemthing that produces a default value that is
specific to the key you are requesting. Using dict.get(key, default)
has the problem that if you ever decide to change the default you'll
have to modify every place that references it. If would be nice if you
could use dict.get(key) or even dict[key] and it gave you a default
that was specified once when the the file was read. Another idea: have
a list of multiple dict objects representing different config files
(e.g. ./.appconfig, ../appconfig, ../../appconfig, ..., ~/.appconfig,
/etc/appconfig, /opt/app/config). The dict semantics get in the way
when making changes -- which file should be updated?

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From t-meyer at ihug.co.nz  Tue Jan 31 00:13:03 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Tue, 31 Jan 2006 12:13:03 +1300
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
Message-ID: <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz>

[Guido van Rossum]
> What would break if we rewrote the save functionality to produce a
> predictable order?

As a reminder to anyone interested, there are three patches on SF  
that provide this (each in a different way):

ConfigParser to accept a custom dict to allow ordering
     http://python.org/sf/1371075
Adds an optional argument to the constructor, the value of which will  
be the class used to store the configuration data (so that a third- 
party order-preserving dictionary class can be used).

ConfigParser to save with order
     http://python.org/sf/1399309
Does a sort() of the sections and options before writing them.

Add 'surgical editing' to ConfigParser
     http://python.org/sf/1410680
Adds an update_file method (no other changes) that updates the given  
file to match the current configuration, preserving blank lines,  
comments, and order.  [Disclaimer: this is my patch]

=Tony.Meyer

From fuzzyman at voidspace.org.uk  Tue Jan 31 00:31:35 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Mon, 30 Jan 2006 23:31:35 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301515u346390c4ned3805db213acdce@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	
	<loom.20060130T004747-592@post.gmane.org>	
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>	
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	
	<17374.33384.775422.175195@montanaro.dyndns.org>	
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>	
	<43DE9066.2040100@voidspace.org.uk>	
	<ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>	
	<43DE9BA9.3040807@voidspace.org.uk>
	<ca471dc20601301515u346390c4ned3805db213acdce@mail.gmail.com>
Message-ID: <43DEA1D7.6070208@voidspace.org.uk>

Guido van Rossum wrote:
> On 1/30/06, Michael Foord <fuzzyman at voidspace.org.uk> wrote:
>   
>> I have a feeling that for many complex applications the developer is the
>> user. Agreed that giving complex configuration data along with program
>> data to the end-user is not good.
>>
>> Despite this, data storage that needs to be edited by developers and
>> system administration is a very common usage.
>>
>> I'm saying that for this purpose (up to a point) an INI file like syntax
>> is much *better* than XML.
>>     
>
> Sure. But for the same reasons, arbitrary nesting is probably bad.
>
>   
Interesting. It's one of the most frequent complaints (lack of nesting)
I hear about ConfigParser. I guess I hang out in different circles.

I still think it's a convenient way of grouping related options.
Breaking them out into separate files is another way.

>> I'm not really sure there is such a clear distinction. Especially for
>> programs without a GUI, things like IP addresses, network details, paths
>> to data files, plugged in components, device names, (etc) seem to be both.
>>     
>
> Just try to keep it simple.
>
> I still think it's a typical programmer's mistake to assume that
> there's no clear distinction. For a user who's looking to change one
> of the three simple properties that make sense for him to configure,
> having to hunt these down in a file containing hundreds of stuff she
> shouldn't tuch is a pain, plus there's the risk that an occasional
> slip of the finger causes damage to stuff they don't understand.
>
>   
Right, I guess I'm still thinking more of the developer than user. For
end-user simpler is definitely better.

I still see use cases for ConfigObj where complex editable data
(relatively - not database record sets) needs to be stored, and
COnfigObj syntax is preferable to XML.

> [snip..]
>> Anyway - I modify my statement accordingly. For accessing configuration
>> type data, the mapping protocol is an intuitive match. Named members
>> mapping to values. Whether this is done via subclassing dict is an
>> implementation detail. (Although I guess it implies that the full
>> dictionary API will be available and you seem to be saying that you
>> would rather that *only* relevant methods are available).
>>     
>
> I can see several improvements on the dict protocol. For example, a
> handy API would be soemthing that produces a default value that is
> specific to the key you are requesting. Using dict.get(key, default)
> has the problem that if you ever decide to change the default you'll
> have to modify every place that references it. If would be nice if you
> could use dict.get(key) or even dict[key] and it gave you a default
> that was specified once when the the file was read. 
We do this through the schema. You specify a default value for keys
(optionally). This is supplied if the value is missing, but isn't
written out again unless modified.

For a *simpler* case (no possibility of a write) you can just merge in
two config files, user data over the defaults.

It doesn't keep a track of which file values come from though (your use
case below).

> Another idea: have
> a list of multiple dict objects representing different config files
> (e.g. ./.appconfig, ../appconfig, ../../appconfig, ..., ~/.appconfig,
> /etc/appconfig, /opt/app/config). The dict semantics get in the way
> when making changes -- which file should be updated?
>
>   
Hmmm.... all in all, I guess this is no. ConfigObj is fine as a separate
module, and it's features are well used - so to that extent the status
quo is fine.

Have we got anywhere on an updated spec for ConfigParser
replacement/enhancement ?

Suggested features seem to include :

* Round tripping (preserving order and possibly comments)
(http://python.org/sf/1410680 maybe)
* Mapping protocol for access is not a bad idea
* List values seem a no brainer good idea to me
* Better support for default values
* Keeping track of values from multiple files, and writing changed
values to the correct file

We haven't discussed unicode of course, seems like a good idea for a
text format. (It was one of the motivations for an alternative
implementation Dict4ini.)

I'd also like to see values allowed without a section. Flat config files
with a few values are appropriate for many applications. (Maybe
section=None to access, this was the approach for ConfigObj 3).

Maybe the shooutout needs an update.

All the best,

Michael Foord
> --
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
>   


From martin at v.loewis.de  Tue Jan 31 01:07:58 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 31 Jan 2006 01:07:58 +0100
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <200601310256.31337.anthony@interlink.com.au>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>	<43DD0BE2.4050306@v.loewis.de>	<87r76qz0c5.fsf@tleepslib.sk.tsukuba.ac.jp>
	<200601310256.31337.anthony@interlink.com.au>
Message-ID: <43DEAA5E.2080300@v.loewis.de>

Anthony Baxter wrote:
> Rather than the back-n-forth about what the FSF might or might not do, 
> can we just ask them for an official opinion and settle the matter?

We can ask, sure. Whether this settles the matter depends on the
answer.

Regards,
Martin

From martin at v.loewis.de  Tue Jan 31 01:50:46 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 31 Jan 2006 01:50:46 +0100
Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno
Message-ID: <43DEB466.8030206@v.loewis.de>

I have a new implementation of stat/fstat/wstat which directly uses
Win32 API, rather than using msvcrt. This works fine so far, except
that error handling turns out to be tricky.

mscvcrt maps errors (GetLastError()) into errno.h values, and also
preserves the original error code in _doserrno. Currently, stat()
will raise OSError, with errno set to the errno.h value.

Because the Win32 error codes are much more fine-grained, this
conversion loses information. Python raises WindowsError in
some cases (e.g. os.listdir); WindowsError inherits from OSError,
but the errno attribute now must be interpreted as a Win32 error.
This is unfortunate, because the values overlap, and somebody
handling OSError might confuse the error codes for errno.h
(msvcrt) values.

So what should I do in the new stat implementation? Try to map
error codes also? Raise WindowsErrors instead? Do something else
entirely?

Comments appreciated.

Regards,
Martin

From guido at python.org  Tue Jan 31 02:09:20 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 17:09:20 -0800
Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno
In-Reply-To: <43DEB466.8030206@v.loewis.de>
References: <43DEB466.8030206@v.loewis.de>
Message-ID: <ca471dc20601301709n19f77779sdfa7a6afa604d457@mail.gmail.com>

What a mess. :-(

WindowsError should have used a different name for the Windows-native
error code, so we could have defined both separately without
confusion.

Is it too late to change WindowsError in that way?

Unhelpfully,

--Guido

On 1/30/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> I have a new implementation of stat/fstat/wstat which directly uses
> Win32 API, rather than using msvcrt. This works fine so far, except
> that error handling turns out to be tricky.
>
> mscvcrt maps errors (GetLastError()) into errno.h values, and also
> preserves the original error code in _doserrno. Currently, stat()
> will raise OSError, with errno set to the errno.h value.
>
> Because the Win32 error codes are much more fine-grained, this
> conversion loses information. Python raises WindowsError in
> some cases (e.g. os.listdir); WindowsError inherits from OSError,
> but the errno attribute now must be interpreted as a Win32 error.
> This is unfortunate, because the values overlap, and somebody
> handling OSError might confuse the error codes for errno.h
> (msvcrt) values.
>
> So what should I do in the new stat implementation? Try to map
> error codes also? Raise WindowsErrors instead? Do something else
> entirely?
>
> Comments appreciated.
>
> Regards,
> Martin
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/guido%40python.org
>


--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Jan 31 05:54:22 2006
From: guido at python.org (Guido van Rossum)
Date: Mon, 30 Jan 2006 20:54:22 -0800
Subject: [Python-Dev] from __future__ syntax changed
Message-ID: <ca471dc20601302054m706b9453n7874776a528626e9@mail.gmail.com>

It looks like the syntax for "from __future__ import ..." changes in
Python 2.5.  I was playing around with Cheetah, and it stumbled over a
file with the following structure:

  # comments
  """ docstring """
  __author__ = "..."
  __version__ = "..."
  from __future__ import generators

Python 2.2 throug 2.4 accept this fine.

Perhaps the AST tree enforces stricter syntax for what can precede a
future statement? I think this should be fixed.

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From mhammond at skippinet.com.au  Tue Jan 31 06:03:01 2006
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Tue, 31 Jan 2006 16:03:01 +1100
Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno
In-Reply-To: <ca471dc20601301709n19f77779sdfa7a6afa604d457@mail.gmail.com>
Message-ID: <DAELJHBGPBHPJKEBGGLNCELEKJAD.mhammond@skippinet.com.au>

Guido:
> What a mess. :-(
>
> WindowsError should have used a different name for the Windows-native
> error code, so we could have defined both separately without
> confusion.
>
> Is it too late to change WindowsError in that way?

I guess "too late" is purely a judgement call about breaking existing code.
One thing to our advantage is that I believe the most common errno
explicitly checked for will be ENOENT, which happily has the same value as
ERROR_FILE_NOT_FOUND.  [Actually, checking 2 *or* 3 (ERROR_PATH_NOT_FOUND)
is also common - but ESRCH==3, which sounds reasonable]

Another way forward may be to issue a warning whenever '.errno' or '[0]' is
referenced from WindowsError, noting that the semantics are soon to change,
and to use the new attribute if they really do want a win32 error code.  I'm
not sure how practical that is though...

Mark


>
> Unhelpfully,
>
> --Guido
>
> On 1/30/06, "Martin v. L?wis" <martin at v.loewis.de> wrote:
> > I have a new implementation of stat/fstat/wstat which directly uses
> > Win32 API, rather than using msvcrt. This works fine so far, except
> > that error handling turns out to be tricky.
> >
> > mscvcrt maps errors (GetLastError()) into errno.h values, and also
> > preserves the original error code in _doserrno. Currently, stat()
> > will raise OSError, with errno set to the errno.h value.
> >
> > Because the Win32 error codes are much more fine-grained, this
> > conversion loses information. Python raises WindowsError in
> > some cases (e.g. os.listdir); WindowsError inherits from OSError,
> > but the errno attribute now must be interpreted as a Win32 error.
> > This is unfortunate, because the values overlap, and somebody
> > handling OSError might confuse the error codes for errno.h
> > (msvcrt) values.
> >
> > So what should I do in the new stat implementation? Try to map
> > error codes also? Raise WindowsErrors instead? Do something else
> > entirely?
> >


From tim.peters at gmail.com  Tue Jan 31 06:53:26 2006
From: tim.peters at gmail.com (Tim Peters)
Date: Tue, 31 Jan 2006 00:53:26 -0500
Subject: [Python-Dev] from __future__ syntax changed
In-Reply-To: <ca471dc20601302054m706b9453n7874776a528626e9@mail.gmail.com>
References: <ca471dc20601302054m706b9453n7874776a528626e9@mail.gmail.com>
Message-ID: <1f7befae0601302153n701d732ek79307c509962d115@mail.gmail.com>

[Guido van Rossum]
> It looks like the syntax for "from __future__ import ..." changes in
> Python 2.5.  I was playing around with Cheetah, and it stumbled over a
> file with the following structure:
>
>   # comments
>   """ docstring """
>   __author__ = "..."
>   __version__ = "..."
>   from __future__ import generators
>
> Python 2.2 throug 2.4 accept this fine.
>
> Perhaps the AST tree enforces stricter syntax for what can precede a
> future statement?  I think this should be fixed.

Peculiar.  It wasn't intended that be allowed; PEP 236 spelled this out:

""""
In addition, all future_statments must appear near the top of the
module.  The only lines that can appear before a future_statement are:

    + The module docstring (if any).
    + Comments.
    + Blank lines.
    + Other future_statements.
""""

That isn't frivolous, since a __future__ gimmick may affect the
legality of binding statements (as in your example) or other import
statements, etc -- it was deliberate that only blank lines, comments
and (maybe) a docstring could appear before the first future statment.
 I'd call this a bug in 2.2-2.4.

Note too that Tools/scripts/cleanfuture.py in 2.4 doesn't change that
example, because it doesn't believe it contains any __future__
statements (cleanfuture.py follows the PEP's syntax).

From martin at v.loewis.de  Tue Jan 31 07:46:26 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 31 Jan 2006 07:46:26 +0100
Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno
In-Reply-To: <ca471dc20601301709n19f77779sdfa7a6afa604d457@mail.gmail.com>
References: <43DEB466.8030206@v.loewis.de>
	<ca471dc20601301709n19f77779sdfa7a6afa604d457@mail.gmail.com>
Message-ID: <43DF07C2.5080302@v.loewis.de>

Guido van Rossum wrote:
> WindowsError should have used a different name for the Windows-native
> error code, so we could have defined both separately without
> confusion.
> 
> Is it too late to change WindowsError in that way?

We could define a different exception, say, Win32Error which inherits
from OSError, and has a POSIX errno in the errno attribute, and also
an additional attribute win32_error. We could then stop raising
WindowsError. So code that expects WindowsError will never see it, but
will see OSError instead (which it also should handle if it was written
portable).

Or we could just break the errno value in WindowsError, which would
make code that catches WindowsError continue to work, but break
code that looks for specific errno values in WindowsError. Code aiming
for backwards compatibility should then write

 except WindowsError, e:
   try:
      windows_error = e.windows_error
   except NameError:
      # Python 2.4 and earlier
      windows_error = e.errno

As for naming, I would like to avoid naming new things Win32: Microsoft
just changed the name of the API to "Windows API" (pointing out that
it isn't necessarily 32 bits anymore).

Regards,
Martin

From martin at v.loewis.de  Tue Jan 31 07:58:25 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 31 Jan 2006 07:58:25 +0100
Subject: [Python-Dev] "DOS" error codes, WindowsError, and errno
In-Reply-To: <DAELJHBGPBHPJKEBGGLNCELEKJAD.mhammond@skippinet.com.au>
References: <DAELJHBGPBHPJKEBGGLNCELEKJAD.mhammond@skippinet.com.au>
Message-ID: <43DF0A91.9020407@v.loewis.de>

Mark Hammond wrote:
> I guess "too late" is purely a judgement call about breaking existing code.
> One thing to our advantage is that I believe the most common errno
> explicitly checked for will be ENOENT, which happily has the same value as
> ERROR_FILE_NOT_FOUND.  [Actually, checking 2 *or* 3 (ERROR_PATH_NOT_FOUND)
> is also common - but ESRCH==3, which sounds reasonable]

The strerror for ESRCH is "No such process", which isn't really that
reasonable. Breakage would occur, because people might currently check
for ENOENT (or any other specific error code); if they don't see that
specific error code, they will assume it is a "real" error.

That said: I would actually expect that it is infrequent that people
do look at the errno of a WindowsError, so breakage might be small.

> Another way forward may be to issue a warning whenever '.errno' or '[0]' is
> referenced from WindowsError, noting that the semantics are soon to change,
> and to use the new attribute if they really do want a win32 error code.  I'm
> not sure how practical that is though...

It would need to get suppressed when the value is merely displayed,
atleast in the default __str__ implementation.

Regards,
Martin

From stephen at xemacs.org  Tue Jan 31 10:51:12 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 31 Jan 2006 18:51:12 +0900
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01 through
 2006-01-15
In-Reply-To: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com> (Tim
	Peters's message of "Mon, 30 Jan 2006 16:44:47 -0500")
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
	<1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com>
	<873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp>
	<1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com>
Message-ID: <87vew0ohdb.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Tim" == Tim Peters <tim.peters at gmail.com> writes:

    Tim> I'm not making myself clear.

Whatever makes you think that?<wink> In fact, everything you've said
about your criteria for behavior was quite clear from the first, and
it was fairly easy to infer your beliefs about the implications of
history.

I just wanted to state that there is a clear precedent of an FSF
complaint about exactly what Python is doing.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From stephen at xemacs.org  Tue Jan 31 11:09:26 2006
From: stephen at xemacs.org (Stephen J. Turnbull)
Date: Tue, 31 Jan 2006 19:09:26 +0900
Subject: [Python-Dev] / as path join operator
In-Reply-To: <bb8868b90601301006s4d9a6ea7y8221e713430499b4@mail.gmail.com>
	(Jason Orendorff's message of "Mon, 30 Jan 2006 13:06:16 -0500")
References: <740c3aec0601241222s25123daerd85ac2e6a9d0b920@mail.gmail.com>
	<43D6BA85.8070007@colorstudy.com>
	<bb8868b90601250839g4a264695kdb9b7e41d90eb61c@mail.gmail.com>
	<740c3aec0601251237j422c274dx32667261a83df9b1@mail.gmail.com>
	<63642A9B-BE9A-4E85-8C54-5060FE32007F@ihug.co.nz>
	<Pine.LNX.4.58.0601252331300.6232@alice>
	<d11dcfba0601251645l346d08c2v38a9d09ae765ed51@mail.gmail.com>
	<87k6cnaabt.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601271519j2a89dd05qb1ffe677ef93b169@mail.gmail.com>
	<87vew44z8k.fsf@tleepslib.sk.tsukuba.ac.jp>
	<bb8868b90601301006s4d9a6ea7y8221e713430499b4@mail.gmail.com>
Message-ID: <87r76oogix.fsf@tleepslib.sk.tsukuba.ac.jp>

>>>>> "Jason" == Jason Orendorff <jason.orendorff at gmail.com> writes:

    Jason> You seem to think that because I said "operating systems",
    Jason> I'm talking about kernel algorithms and such.

I can see how you'd get that impression, but it's not true.  My reason
for mentioning OS-level filesystem was to show that even in that
limited domain, treating paths as strings leads to bugs.

    Jason> I think paths are used more for communication, less for
    Jason> computation.

True.  For that purpose it is absolutely essential to have a string
represention.  However, I was discussing the use of "/" to invoke path
composition, which is a computation.  Nobody will use that for
communication (except to describe a path expression in graph theoretic
terms), and I don't think it's a good idea to use that particular
symbol for that operation.


-- 
School of Systems and Information Engineering http://turnbull.sk.tsukuba.ac.jp
University of Tsukuba                    Tennodai 1-1-1 Tsukuba 305-8573 JAPAN
               Ask not how you can "do" free software business;
              ask what your business can "do for" free software.

From p.f.moore at gmail.com  Tue Jan 31 11:52:00 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 31 Jan 2006 10:52:00 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
Message-ID: <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com>

On 1/30/06, Guido van Rossum <guido at python.org> wrote:
> Aha. I am beginning to understand. When people say "ConfigParser is
> hopeless" they mean ".INI files are hopeless". I happen to disagree.
> (There's also a meme that says that every aspect of an app should be
> configurable. I disagree with that too. As Joel Spolski points out in
> his book on UI design, most configuration options are signs of
> indecisive developers, and most users never change the defaults.)

While you're right that a lot of the comments people make about
ConfigParser are actually about INI files (e.g., no nested sections,
no "top-level" values), I do find that when I use ConfigParser, I
often need to write a wrapper around it to make it more usable.

A couple of examples, from a current application:

* No way to merge files or sections. Usually to provide default
values. I have a suite of applications, all using the same framework.
I have a hardcoded DEFAULT_CONFIG in the code, overriden by a
<suite>.ini, overridden again by a <app>.ini. OK, maybe it's
overengineered, but I do use the various levels, so it's also
useful... (The defaults parameter to the constructor is misnamed,
AFAICT, as it's for interpolation defaults, not value defaults).

* A dictionary interface, or even attribute access (where the key
names are Python identifiers, which is usually the case), is usually
far cleaner to work with in code. For example, conf.log.level vs
conf.get('log', 'level'). But that may just be me - it's certainly a
style issue.

Nothing major, but worth considering.

I agree entirely that once you go beyond INI format, you should have a
new module. Of course, ConfigParser already allows more than basic INI
format ("key: value" lines, continuations, and substitution strings)
but we'll gloss over that... :-)

Paul.

From thomas at xs4all.net  Tue Jan 31 11:59:20 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 31 Jan 2006 11:59:20 +0100
Subject: [Python-Dev] Compiler warnings
Message-ID: <20060131105920.GQ18916@xs4all.nl>


I noticed a few compiler warnings, when I compile Python on my amd64 with
gcc 4.0.3:

Objects/longobject.c: In function ?PyLong_AsDouble?:
Objects/longobject.c:655: warning: ?e? may be used uninitialized in this function
Objects/longobject.c: In function ?long_true_divide?:
Objects/longobject.c:2263: warning: ?aexp? may be used uninitialized in this function
Objects/longobject.c:2263: warning: ?bexp? may be used uninitialized in this function

Modules/linuxaudiodev.c: In function ?lad_obuffree?:
Modules/linuxaudiodev.c:392: warning: ?ssize? may be used uninitialized in this function
Modules/linuxaudiodev.c: In function ?lad_bufsize?:
Modules/linuxaudiodev.c:348: warning: ?ssize? may be used uninitialized in this function
Modules/linuxaudiodev.c: In function ?lad_obufcount?:
Modules/linuxaudiodev.c:369: warning: ?ssize? may be used uninitialized in this function

Those are all fairly harmless, just the compiler can't figure out that they
are never actually used when they aren't explicitly initialized, because the
initialization happens a few functioncalls deeper into the callstack. This
one:

Python/marshal.c: In function ?PyMarshal_ReadShortFromFile?:
Python/marshal.c:410: warning: ?rf.end? is used uninitialized in this function
Python/marshal.c:410: warning: ?rf.ptr? is used uninitialized in this function

(The linenumber is off, it should be 884)
is more of the same, except you can't tell from the function that it is a
"can never happen" situation: if PyMarshal_ReadShortFromFile() was called
with NULL as fp, it would actually use the uninitialized 'end' and 'ptr'
struct members. An assert() is probably in place here.

Should these warnings be fixed? I know Tim has always argued to fix them, in
the past (and I agree,) and it doesn't look like doing so, by initializing
the variables, wouldn't be too big a performance hit.

I also noticed test_logging is spuriously failing, and not just on my
machine (according to buildbot logs.) Is anyone (Vinay?) looking at that
yet?

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From fuzzyman at voidspace.org.uk  Tue Jan 31 13:27:25 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 31 Jan 2006 12:27:25 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	<17374.33384.775422.175195@montanaro.dyndns.org>	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
	<79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com>
Message-ID: <43DF57AD.5010007@voidspace.org.uk>

Paul Moore wrote:
> On 1/30/06, Guido van Rossum <guido at python.org> wrote:
>   
>> Aha. I am beginning to understand. When people say "ConfigParser is
>> hopeless" they mean ".INI files are hopeless". I happen to disagree.
>> (There's also a meme that says that every aspect of an app should be
>> configurable. I disagree with that too. As Joel Spolski points out in
>> his book on UI design, most configuration options are signs of
>> indecisive developers, and most users never change the defaults.)
>>     
>
> While you're right that a lot of the comments people make about
> ConfigParser are actually about INI files (e.g., no nested sections,
> no "top-level" values), I do find that when I use ConfigParser, I
> often need to write a wrapper around it to make it more usable.
>
> A couple of examples, from a current application:
>
> * No way to merge files or sections. Usually to provide default
> values. I have a suite of applications, all using the same framework.
> I have a hardcoded DEFAULT_CONFIG in the code, overriden by a
> <suite>.ini, overridden again by a <app>.ini. OK, maybe it's
> overengineered, but I do use the various levels, so it's also
> useful... (The defaults parameter to the constructor is misnamed,
> AFAICT, as it's for interpolation defaults, not value defaults).
>
> * A dictionary interface, or even attribute access (where the key
> names are Python identifiers, which is usually the case), is usually
> far cleaner to work with in code. For example, conf.log.level vs
> conf.get('log', 'level'). But that may just be me - it's certainly a
> style issue.
>
> Nothing major, but worth considering.
>
> I agree entirely that once you go beyond INI format, you should have a
> new module. Of course, ConfigParser already allows more than basic INI
> format ("key: value" lines, continuations, and substitution strings)
> but we'll gloss over that... :-)
>   
Well, ConfigObj is listed as an XML alternative at :

    http://www.pault.com/pault/pxml/xmlalternatives.html

For complex configuration situations, many people feel that handcrafting 
XML is not a fun alternative.

One of the driving forces behind ConfigObj was a firm that *used* to 
have their system administrators using ZConfig, but wanted something 
more friendly.

They have configuration files nested a few levels deep configuring 
services, plugins, network details, etc. This is all properly 
'application configuration'. (Not data persistence.)

The file format (and config structure) is easily visible from the files. 
The config files are automatically validated from the schema 
(configspecs) which also facilitate the automatic creation of new sections.

I *personally* think that this is a common use case and the format is a 
natural extension of the 'INI-like' format. I'm happy for it to remain a 
separate module (as indeed ConfigObj will). There are several valid 
complaints about ConfigParser - and it is *possible* (although possibly 
not desirable) to largely solve them all without breaking backwards 
compatibility with ConfigParser.

Is it likely that an *additional* module could be added ? The suggestion 
to narrow the ConfigObj API and not subclass dict (but still use the 
mapping protocol) are interesting, and certainly not out of the 
question. Nested sections however, are a significant part of the reason 
for it's existence.

All the best,

Fuzzyman
http://www.voidspace.org.uk/python/index.shtml

> Paul.
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>   


From fuzzyman at voidspace.org.uk  Tue Jan 31 13:38:19 2006
From: fuzzyman at voidspace.org.uk (Michael Foord)
Date: Tue, 31 Jan 2006 12:38:19 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz>
Message-ID: <43DF5A3B.7010705@voidspace.org.uk>

Tony Meyer wrote:
> [Guido van Rossum]
>   
>> What would break if we rewrote the save functionality to produce a
>> predictable order?
>>     
>
> As a reminder to anyone interested, there are three patches on SF  
> that provide this (each in a different way):
>
> ConfigParser to accept a custom dict to allow ordering
>      http://python.org/sf/1371075
> Adds an optional argument to the constructor, the value of which will  
> be the class used to store the configuration data (so that a third- 
> party order-preserving dictionary class can be used).
>
> ConfigParser to save with order
>      http://python.org/sf/1399309
> Does a sort() of the sections and options before writing them.
>
>   
These first two patches don't seem to address the main issue, which is 
preserving the *original* order of the file.

> Add 'surgical editing' to ConfigParser
>      http://python.org/sf/1410680
> Adds an update_file method (no other changes) that updates the given  
> file to match the current configuration, preserving blank lines,  
> comments, and order.  [Disclaimer: this is my patch]
>
>   
This one however does, and if it does what it says on the box - is 
definitely worth accepting.

All the best,

Michael Foord
http://www.voidspace.org.uk/python/index.shtml

> =Tony.Meyer
> _______________________________________________
> Python-Dev mailing list
> Python-Dev at python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/fuzzyman%40voidspace.org.uk
>
>   


From vinay_sajip at yahoo.co.uk  Tue Jan 31 17:13:19 2006
From: vinay_sajip at yahoo.co.uk (Vinay Sajip)
Date: Tue, 31 Jan 2006 16:13:19 +0000 (UTC)
Subject: [Python-Dev] Extension to ConfigParser
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<43DE7C2D.5060804@voidspace.org.uk>
Message-ID: <loom.20060131T163900-859@post.gmane.org>

Fuzzyman <fuzzyman <at> voidspace.org.uk> writes:

> In my opinion dictionary syntax is the most natural way to represent a
> config file - it is a data structure with named members. This means the
> API for accessing the data - whether from a subclass that does
> additional value processing or from code that just accesses the data.

I agree with this, and also with Michael's comments that configuration is not
just for the end user. (Alternatively, you could say that developers are end
users too!)

As a developer who works for many different clients, I find it saves me a fair
amount of time when I have configurable software modules which I can put
together to meet client requirements. The client doesn't need to see this
"internal" configuration stuff - it's for developers and maintainers of the
software, but very useful nonetheless!

When I implemented my own config module (see
http://www.red-dove.com/python_config.html), I decided not to use an INI-like
format, as I found it wanting. The syntax I used is like Python and like JSON,
but it is not executable (i.e. not Python) and allows bits of the configuration
to refer to other bits of the configuration, provides some restricted evaluation
of values such as "sys.stderr" but in a general way, readily allows defaults at
different levels (e.g. user, program, suite), handles repeating values and
mappings, uses dict or attribute syntax etc. - and all in a single 1500-line
module, since I didn't have to implement INI support.

What I'd like to know is, what happens how? There are a whole lot of
alternatives put forward in the ConfigParserShootout wiki page, but how do we
move things forward from here?

Regards,


Vinay Sajip


From martin at v.loewis.de  Tue Jan 31 18:04:45 2006
From: martin at v.loewis.de (=?UTF-8?B?Ik1hcnRpbiB2LiBMw7Z3aXMi?=)
Date: Tue, 31 Jan 2006 18:04:45 +0100
Subject: [Python-Dev] Compiler warnings
In-Reply-To: <20060131105920.GQ18916@xs4all.nl>
References: <20060131105920.GQ18916@xs4all.nl>
Message-ID: <43DF98AD.3050602@v.loewis.de>

Thomas Wouters wrote:
> I also noticed test_logging is spuriously failing, and not just on my
> machine (according to buildbot logs.) Is anyone (Vinay?) looking at that
> yet?

I looked at it, and I couldn't figure it out. It appears that the last
line of communication is somehow lost, but I could not find out why that
might happen.

Regards,
Martin

From barry at python.org  Mon Jan 30 23:23:18 2006
From: barry at python.org (Barry Warsaw)
Date: Mon, 30 Jan 2006 17:23:18 -0500
Subject: [Python-Dev] DRAFT: python-dev Summary for 2006-01-01
	through	2006-01-15
In-Reply-To: <1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com>
References: <6c63de570601252031y1227ada2n8fce87321de87a73@mail.gmail.com>
	<BFF7D3D6-18F0-4734-872D-69AE72EDDEB9@fuhm.net>
	<99EE6EB7-33B5-47DC-A6E2-6BF1F80ABE7C@mac.com>
	<43DAB283.70207@v.loewis.de>
	<90733CAD-8AB7-4749-9BF8-2386B12F0FAB@mac.com>
	<43DB4A7A.8070606@v.loewis.de>
	<87oe1v2ctz.fsf@tleepslib.sk.tsukuba.ac.jp>
	<43DD0BE2.4050306@v.loewis.de>
	<1f7befae0601291131jd2f7c52j4356efd889b070ff@mail.gmail.com>
	<873bj6yu80.fsf@tleepslib.sk.tsukuba.ac.jp>
	<1f7befae0601301344u14547b7fw44d5d64eb841c1b5@mail.gmail.com>
Message-ID: <1138659798.10363.24.camel@geddy.wooz.org>

On Mon, 2006-01-30 at 16:44 -0500, Tim Peters wrote:

> OTOH, I have no reason to _presume_ that this is their hoped-for
> outcome wrt Python, neither to presume that the politics shaping their
> tussle with Aladdin are relevant to the PSF.  "The law" is rarely
> applied uniformly, in large part because it usually is means rather
> than end.  Python is a high-profile project that hasn't been hiding
> its readline module, and if I presume anything here it's that the FSF
> would have complained by now if they really didn't want this.

I agree, and suggest we do nothing unless the FSF asks us to.

-Barry

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 307 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20060130/9230701f/attachment.pgp 

From evdo.hsdpa at gmail.com  Tue Jan 31 19:45:09 2006
From: evdo.hsdpa at gmail.com (Robert Kim Wireless Internet Advisor)
Date: Tue, 31 Jan 2006 10:45:09 -0800
Subject: [Python-Dev] (no subject)
Message-ID: <1ec620e90601311045g6fd0d206x29b998a83b6f28d4@mail.gmail.com>

anybody here?
bob









2611 s highway 101
san diego, ca 92007
209 984 0880
http://evdo-coverage.com/cell-phone-antenna-booster.html


--
Robert Q Kim, Wireless Internet Advisor
http://evdo-coverage.com/cellular-repeater.html
http://hsdpa-coverage.com

2611 S. Pacific Coast Highway 101
Suite 102
Cardiff by the Sea, CA 92007
206 984 0880

From thomas at xs4all.net  Tue Jan 31 20:16:20 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 31 Jan 2006 20:16:20 +0100
Subject: [Python-Dev] Compiler warnings
In-Reply-To: <43DF98AD.3050602@v.loewis.de>
References: <20060131105920.GQ18916@xs4all.nl> <43DF98AD.3050602@v.loewis.de>
Message-ID: <20060131191620.GR18916@xs4all.nl>

On Tue, Jan 31, 2006 at 06:04:45PM +0100, "Martin v. L?wis" wrote:
> Thomas Wouters wrote:
> > I also noticed test_logging is spuriously failing, and not just on my
> > machine (according to buildbot logs.) Is anyone (Vinay?) looking at that
> > yet?

> I looked at it, and I couldn't figure it out. It appears that the last
> line of communication is somehow lost, but I could not find out why that
> might happen.

Not just the last line, see below. It never happens, as far as I can tell,
when the test_logging test is run alone; it never once broke in several
hundred solo tests. When I ran all combinations of a non-logging test
followed by test_logging, it seems to break fairly randomly with almost all
tests. Certainly not just tests that deal with signals, subprocesses,
threads or sockets, also with tests like test_dircache, test_cookielib,
test_coding, test_filecmp, and others. Just about the only correlation that
makes sense is the duration of the first test. Not all of the other tests
trigger test_logging's failure in any degree of reliability, although I get
the feeling tests that take longer trigger the bug more often -- except that
a test consisting of 'time.sleep(30)' never triggered it.

All in all, it seems like a timing issue -- the timing is different because
the previous test already imported some modules. The test_logging test uses
threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at
strategic places, but it doesn't seem to matter. Regardless of my
pinpointing efforts, much more than just the last line is sometimes missing,
though:

$ ./python -E -tt Lib/test/regrtest.py -uall test_dircache test_logging
test_dircache
test_logging
test test_logging produced unexpected output:
**********************************************************************
*** line 515 of expected output missing:
- DEB -> WARNING: Message 16 (via logrecv.tcp.DEB)
*** line 521 of expected output missing:
- UNDEF -> INFO: Message 22 (via logrecv.tcp.UNDEF)
*** line 524 of expected output missing:
- INF -> INFO: Finish up, it's closing time. Messages should bear numbers 0 through 24. (via logrecv.tcp.INF)
**********************************************************************

I've 'seen' at least all messages between 16 and 22 disappear, although not
more than 3 or 4 in total. There's only ever messages missing, never out of
order or garbled. I'm starting to think there might actually be a bug in the
logging module ;P

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From scott+python-dev at scottdial.com  Tue Jan 31 20:43:00 2006
From: scott+python-dev at scottdial.com (Scott Dial)
Date: Tue, 31 Jan 2006 14:43:00 -0500
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DF5A3B.7010705@voidspace.org.uk>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	<6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz>
	<43DF5A3B.7010705@voidspace.org.uk>
Message-ID: <43DFBDC4.6020804@scottdial.com>

Michael Foord wrote:
> Tony Meyer wrote:
>> [Guido van Rossum]
>> Add 'surgical editing' to ConfigParser
>>      http://python.org/sf/1410680
>> Adds an update_file method (no other changes) that updates the given  
>> file to match the current configuration, preserving blank lines,  
>> comments, and order.  [Disclaimer: this is my patch]
>>
>>   
> This one however does, and if it does what it says on the box - is 
> definitely worth accepting.
> 

I've spent a small amount of time playing with this patch, and the 
intent is there, but it appears to have some obvious bugs with adding 
blank lines and (at least) making an empty [DEFAULT] section appear and 
disappear. I'm not sure that this is the place to make these comments, 
so I will stop with that.

In case it wasn't clear, I am new here.. Hi.

-- 
Scott Dial
scott at scottdial.com
dialsa at rose-hulman.edu

From mattheww at chiark.greenend.org.uk  Tue Jan 31 20:46:39 2006
From: mattheww at chiark.greenend.org.uk (mattheww at chiark.greenend.org.uk)
Date: Tue, 31 Jan 2006 19:46:39 +0000 (UTC)
Subject: [Python-Dev] Octal literals
References: <bbaeab100601181850k3c885e0dq3d6cdc8ead4d77a7@mail.gmail.com>
	<ca471dc20601190931g39f89a6pe24789d043c7a31c@mail.gmail.com>
	<dqolm4$4l5$1@sea.gmane.org>
	<ca471dc20601191112g4b68a127q16f23ee7fce7fd80@mail.gmail.com>
Message-ID: <droeqv$4u5$2@sea.gmane.org>

Guido van Rossum  <guido at python.org> wrote:
>Right. And this is not a hypothetical issue either -- in Perl, hex and
>oct *do* work the other way I believe. More reasons to get rid of
>these in Python 3000. Perhaps we should also get rid of hex/oct
>lterals?


I would like to argue for removing octal literals.

This feature has a very bad property: it can cause obscure problems for
people who do not know or care about it. I have seen people try to use
leading zeroes to make integer literals line up in a table. If they are
lucky, they will get a syntax error. If they are unlucky, their program
will silently do the wrong thing.

It would be rather offputting to have to warn about this in the
tutorial. But at present, a learner who isn't familiar with another
language using this convention would have no reason to suspect it
exists. As far as I can tell, it's documented only in the BNF.

I think the safe thing in Python 3000 would be for literals with leading
0 to be syntax errors.

Possibly os.chmod and os.umask could be extended to take a string
argument so we could write chmod(path, "0640").

-M-


From ark at acm.org  Tue Jan 31 20:55:33 2006
From: ark at acm.org (Andrew Koenig)
Date: Tue, 31 Jan 2006 14:55:33 -0500
Subject: [Python-Dev] Octal literals
In-Reply-To: <droeqv$4u5$2@sea.gmane.org>
Message-ID: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop>

> Possibly os.chmod and os.umask could be extended to take a string
> argument so we could write chmod(path, "0640").

-1.

Would you really want chmod(path, 0640) and chmod(path, "0640") to have
different meanings?
 



From g.brandl at gmx.net  Tue Jan 31 20:57:18 2006
From: g.brandl at gmx.net (Georg Brandl)
Date: Tue, 31 Jan 2006 20:57:18 +0100
Subject: [Python-Dev] YAML (was Re: Extension to ConfigParser)
In-Reply-To: <ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	<17374.33384.775422.175195@montanaro.dyndns.org>	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>	<43DE9066.2040100@voidspace.org.uk>
	<ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
Message-ID: <drofeu$9g9$1@sea.gmane.org>

Guido van Rossum wrote:

>> But like it or not, configuration files are often used to store data
>> about what a program does - not just the UI options. Storing this in a
>> human readable and editable format is of great advantage.
>>
>> Yes, a lot of the entries will never be modified by a user - but many
>> will need to be edited and read by a developer. In a production
>> environment importing from python modules is not likely to be safe.
> 
> Ah. This definitely isn't what ConfigParser was meant to do. I'd think
> for this you should use some kind of XML pickle though. That's
> horrible if end users must edit it, but great for saving
> near-arbitrary persistent data in a readable and occasionally editable
> (for the developer) form.

While we're at it, is the Python library going to incorporate some YAML
parser in the future? YAML seems like a perfectly matching data format
for Python.

Georg


From t-meyer at ihug.co.nz  Tue Jan 31 20:49:02 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed, 1 Feb 2006 08:49:02 +1300
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <43DFBDC4.6020804@scottdial.com>
References: <43D930E0.6070805@voidspace.org.uk>	<loom.20060130T004747-592@post.gmane.org>	<43DDE661.4020808@voidspace.org.uk>	<43DE51F7.4010007@colorstudy.com>	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>	<6E8262A4-17F8-404E-8EA4-773BA2639100@ihug.co.nz>
	<43DF5A3B.7010705@voidspace.org.uk>
	<43DFBDC4.6020804@scottdial.com>
Message-ID: <4127C17B-01A0-4E47-93C4-71B1C138F8E7@ihug.co.nz>

[Scott Dial]

[Re: http://python.org/sf/1410680]
> I've spent a small amount of time playing with this patch, and the
> intent is there, but it appears to have some obvious bugs with adding
> blank lines and (at least) making an empty [DEFAULT] section appear  
> and
> disappear. I'm not sure that this is the place to make these comments,
> so I will stop with that.

Could you please add whatever information you have to the tracker?   
If you don't have a sf account and don't want to get one, I suppose  
you could just email it to me and I'll add it.

=Tony.Meyer


From t-meyer at ihug.co.nz  Tue Jan 31 20:54:42 2006
From: t-meyer at ihug.co.nz (Tony Meyer)
Date: Wed, 1 Feb 2006 08:54:42 +1300
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
	<79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com>
Message-ID: <B190FD4D-FCBE-4CF1-B113-BFF12561824D@ihug.co.nz>

[Paul Moore]
> * No way to merge files or sections. Usually to provide default
> values. I have a suite of applications, all using the same framework.
> I have a hardcoded DEFAULT_CONFIG in the code, overriden by a
> <suite>.ini, overridden again by a <app>.ini. OK, maybe it's
> overengineered, but I do use the various levels, so it's also
> useful... (The defaults parameter to the constructor is misnamed,
> AFAICT, as it's for interpolation defaults, not value defaults).

Why doesn't this work?  It does here:

$ cat suite.ini
[sect]
opt1 = 1
opt2 = 2
$ cat app.ini
[sect]
opt1 = 3
opt4 = 5
$ python
Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
 >>> import ConfigParser
 >>> c = ConfigParser.ConfigParser()
 >>> c.read(("suite.ini", "app.ini"))
['suite.ini', 'app.ini']
 >>> c.sections()
['sect']
 >>> c.options("sect")
['opt4', 'opt2', 'opt1']
 >>> c.get("sect", "opt1")
'3'

Or do you mean something else?

=Tony.Meyer


From mattheww at chiark.greenend.org.uk  Tue Jan 31 21:03:36 2006
From: mattheww at chiark.greenend.org.uk (mattheww at chiark.greenend.org.uk)
Date: Tue, 31 Jan 2006 20:03:36 +0000 (UTC)
Subject: [Python-Dev] Octal literals
References: <droeqv$4u5$2@sea.gmane.org>
	<00ec01c626a0$5afdc080$6402a8c0@arkdesktop>
Message-ID: <drofqo$bes$1@sea.gmane.org>

Andrew Koenig <ark at acm.org> wrote:
>> Possibly os.chmod and os.umask could be extended to take a string
>> argument so we could write chmod(path, "0640").
>
>-1.
>
>Would you really want chmod(path, 0640) and chmod(path, "0640") to have
>different meanings?


I want the former to be a syntax error, as I said in the preceding paragraph.

-M-


From martin at v.loewis.de  Tue Jan 31 21:04:39 2006
From: martin at v.loewis.de (=?ISO-8859-1?Q?=22Martin_v=2E_L=F6wis=22?=)
Date: Tue, 31 Jan 2006 21:04:39 +0100
Subject: [Python-Dev] Compiler warnings
In-Reply-To: <20060131191620.GR18916@xs4all.nl>
References: <20060131105920.GQ18916@xs4all.nl> <43DF98AD.3050602@v.loewis.de>
	<20060131191620.GR18916@xs4all.nl>
Message-ID: <43DFC2D7.5080506@v.loewis.de>

Thomas Wouters wrote:
> All in all, it seems like a timing issue -- the timing is different because
> the previous test already imported some modules. The test_logging test uses
> threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at
> strategic places, but it doesn't seem to matter. Regardless of my
> pinpointing efforts, much more than just the last line is sometimes missing,
> though:

Can you create a truss/strace dump of a failed run (I tried, but it
always succeeded for me when run under strace).

Regards,
Martin

From phd at mail2.phd.pp.ru  Tue Jan 31 21:07:48 2006
From: phd at mail2.phd.pp.ru (Oleg Broytmann)
Date: Tue, 31 Jan 2006 23:07:48 +0300
Subject: [Python-Dev] JSON (was: YAML)
In-Reply-To: <drofeu$9g9$1@sea.gmane.org>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk>
	<43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
	<43DE9066.2040100@voidspace.org.uk>
	<ca471dc20601301440u2a2113a3qe521e3c0662c5a46@mail.gmail.com>
	<drofeu$9g9$1@sea.gmane.org>
Message-ID: <20060131200748.GA19860@phd.pp.ru>

On Tue, Jan 31, 2006 at 08:57:18PM +0100, Georg Brandl wrote:
> While we're at it, is the Python library going to incorporate some YAML
> parser in the future? YAML seems like a perfectly matching data format
> for Python.

   JSON is even better!

Oleg.
-- 
     Oleg Broytmann            http://phd.pp.ru/            phd at phd.pp.ru
           Programmers don't die, they just GOSUB without RETURN.

From fdrake at acm.org  Tue Jan 31 21:06:29 2006
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue, 31 Jan 2006 15:06:29 -0500
Subject: [Python-Dev] Octal literals
In-Reply-To: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop>
References: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop>
Message-ID: <200601311506.29633.fdrake@acm.org>

On Tuesday 31 January 2006 14:55, Andrew Koenig wrote:
 > Would you really want chmod(path, 0640) and chmod(path, "0640") to have
 > different meanings?

Actually, the proposal that suggested this also proposed that 0640 would raise 
a SyntaxError, since it was all about getting rid of octal literals.


  -Fred

-- 
Fred L. Drake, Jr.   <fdrake at acm.org>

From p.f.moore at gmail.com  Tue Jan 31 21:29:04 2006
From: p.f.moore at gmail.com (Paul Moore)
Date: Tue, 31 Jan 2006 20:29:04 +0000
Subject: [Python-Dev] Extension to ConfigParser
In-Reply-To: <B190FD4D-FCBE-4CF1-B113-BFF12561824D@ihug.co.nz>
References: <43D930E0.6070805@voidspace.org.uk>
	<loom.20060130T004747-592@post.gmane.org>
	<43DDE661.4020808@voidspace.org.uk> <43DE51F7.4010007@colorstudy.com>
	<ca471dc20601301120i74c52677u7a816815a87c25d4@mail.gmail.com>
	<17374.33384.775422.175195@montanaro.dyndns.org>
	<ca471dc20601301356pd263aevcccaaf390cb544fb@mail.gmail.com>
	<79990c6b0601310252t2bff85cfkc11273ea646bcc31@mail.gmail.com>
	<B190FD4D-FCBE-4CF1-B113-BFF12561824D@ihug.co.nz>
Message-ID: <79990c6b0601311229o4a88bd1evf279c7bfcaa7db4a@mail.gmail.com>

On 1/31/06, Tony Meyer <t-meyer at ihug.co.nz> wrote:
> Why doesn't this work?  It does here:
>
> $ cat suite.ini
> [sect]
> opt1 = 1
> opt2 = 2
> $ cat app.ini
> [sect]
> opt1 = 3
> opt4 = 5
> $ python
> Python 2.4.1 (#2, Mar 31 2005, 00:05:10)
> [GCC 3.3 20030304 (Apple Computer, Inc. build 1666)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
>  >>> import ConfigParser
>  >>> c = ConfigParser.ConfigParser()
>  >>> c.read(("suite.ini", "app.ini"))
> ['suite.ini', 'app.ini']
>  >>> c.sections()
> ['sect']
>  >>> c.options("sect")
> ['opt4', 'opt2', 'opt1']
>  >>> c.get("sect", "opt1")
> '3'
>
> Or do you mean something else?

Err. Because I missed the fact that read() method takes multiple
filenames? There's even a specific explanation of how to load defaults
and then override them with optional files.

I don't know how I missed that. Thanks for pointing it out.

(The whole day's been like that - I'm not sure why I get out of bed
sometimes....:-)

Paul.

From thomas at xs4all.net  Tue Jan 31 21:42:57 2006
From: thomas at xs4all.net (Thomas Wouters)
Date: Tue, 31 Jan 2006 21:42:57 +0100
Subject: [Python-Dev] Compiler warnings
In-Reply-To: <43DFC2D7.5080506@v.loewis.de>
References: <20060131105920.GQ18916@xs4all.nl> <43DF98AD.3050602@v.loewis.de>
	<20060131191620.GR18916@xs4all.nl> <43DFC2D7.5080506@v.loewis.de>
Message-ID: <20060131204257.GS18916@xs4all.nl>

On Tue, Jan 31, 2006 at 09:04:39PM +0100, "Martin v. L?wis" wrote:
> Thomas Wouters wrote:
> > All in all, it seems like a timing issue -- the timing is different because
> > the previous test already imported some modules. The test_logging test uses
> > threads, *ugh*. I tried adding a few Event-waits and time.sleep()'s at
> > strategic places, but it doesn't seem to matter. Regardless of my
> > pinpointing efforts, much more than just the last line is sometimes missing,
> > though:

> Can you create a truss/strace dump of a failed run (I tried, but it
> always succeeded for me when run under strace).

Nope, they always succeed when run under strace. I haven't been able to
capture the supposedly-TCP traffic either, not even when binding and
connecting to my actual IP address. I wonder if glibc is doing trickery
because it sees both endpoints of the socket are in the same process.

-- 
Thomas Wouters <thomas at xs4all.net>

Hi! I'm a .signature virus! copy me into your .signature file to help me spread!

From guido at python.org  Tue Jan 31 21:47:12 2006
From: guido at python.org (Guido van Rossum)
Date: Tue, 31 Jan 2006 12:47:12 -0800
Subject: [Python-Dev] Octal literals
In-Reply-To: <00ec01c626a0$5afdc080$6402a8c0@arkdesktop>
References: <droeqv$4u5$2@sea.gmane.org>
	<00ec01c626a0$5afdc080$6402a8c0@arkdesktop>
Message-ID: <ca471dc20601311247w5c40579t42d0923bb4caa290@mail.gmail.com>

On 1/31/06, Andrew Koenig <ark at acm.org> wrote:
> > Possibly os.chmod and os.umask could be extended to take a string
> > argument so we could write chmod(path, "0640").
>
> -1.
>
> Would you really want chmod(path, 0640) and chmod(path, "0640") to have
> different meanings?

Apart from making 0640 a syntax error (which I think is wrong too),
could this be solved by *requiring* the argument to be a string? (Or
some other data type, but that's probably overkill.)

--
--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark at acm.org  Tue Jan 31 23:17:22 2006
From: ark at acm.org (Andrew Koenig)
Date: Tue, 31 Jan 2006 17:17:22 -0500
Subject: [Python-Dev] Octal literals
In-Reply-To: <ca471dc20601311247w5c40579t42d0923bb4caa290@mail.gmail.com>
Message-ID: <011c01c626b4$2d6a0750$6402a8c0@arkdesktop>

> Apart from making 0640 a syntax error (which I think is wrong too),
> could this be solved by *requiring* the argument to be a string? (Or
> some other data type, but that's probably overkill.)

That solves the problem only in that particular context.

I would think that if it is deemed undesirable for a leading 0 to imply
octal, then it would be best to decide on a different syntax for octal
literals and use that syntax consistently everywhere.

I am personally partial to allowing an optional radix (in decimal) followed
by the letter r at the beginning of a literal, so 19, 8r23, and 16r13 would
all represent the same value.