From anthony at ekit-inc.com  Mon Dec  1 01:15:46 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Mon Dec  1 01:16:10 2003
Subject: [Python-Dev] backticks delenda est 
In-Reply-To: Message from "Raymond Hettinger" <python@rcn.com> of "Sun,
	30 Nov 2003 23:42:55 CDT." <003401c3b7c5$96b39e20$e841fea9@oemcomputer>
Message-ID: <200312010615.hB16FlJ8019622@maxim.off.ekorp.com>


>>> "Raymond Hettinger" wrote
> Advisory from a micro-performance hawk:  Backticks are faster than
> repr()
> 
> >>> from timeit import Timer
> >>> min(Timer('`x`', 'x=1').repeat(3))
> 1.4857213496706265
> >>> min(Timer('repr(x)', 'x=1').repeat(3))
> 1.7748914665012876

Presumably because the backtick version doesn't have to look up 
'repr', then call it. 

>>> def f1(a):
...   repr(a)
... 
>>> 
>>> def f2(a):
...   `a`  
... 
>>> import dis
>>> dis.dis(f1)
  2           0 LOAD_GLOBAL              0 (repr)
              3 LOAD_FAST                0 (a)
              6 CALL_FUNCTION            1
              9 POP_TOP             
             10 LOAD_CONST               0 (None)
             13 RETURN_VALUE        
>>> dis.dis(f2)
  2           0 LOAD_FAST                0 (a)
              3 UNARY_CONVERT       
              4 POP_TOP             
              5 LOAD_CONST               0 (None)
              8 RETURN_VALUE        

This is presumably something that a future optimiser could "fix",
once we have the no-shadowing-builtins rule enforced.

I don't think this is a serious enough problem to merit leaving the 
backticks in the std lib. I find them annoying, and hard to read.
In general, the backticks are in debugging and error handling code, 
so the performance penalty should be negligible.

Anthony

From theller at python.net  Mon Dec  1 06:43:43 2003
From: theller at python.net (Thomas Heller)
Date: Mon Dec  1 06:43:57 2003
Subject: [Python-Dev] "groupby" iterator
In-Reply-To: <200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Sun, 30 Nov 2003 16:18:37 -0800")
References: <20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.net>
Message-ID: <ptf8yd8w.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

>> How about:
>> 
>> extract(attr='grade')
>> extract(item=2)
>> extract(method='foo')   # returns the result of calling 'ob.foo()'
>> 
>> And following the pattern of Zope's old "query" package:
>> 
>> extract(extract(attr='foo'), attr='bar')   # extracts ob.foo.bar
>> extract(extract(item=10), method='spam')   # extracts ob[10].spam()
>> 
>> i.e., the first (optional) positional argument to extract is a function 
>> that's called on the outer extract's argument, and the return value is then 
>> used to perform the main extract operation on.
>
> I'm not sure what the advantage of this is.  It seems more typing,
> more explanation, probably more code to implement (to check for
> contradicting keyword args).

Hm, couldn't "lambda ob: ob.foo.bar" return exactly the same thing as

"extract(extract(attr='foo'), attr='bar')"

? In other words: return specialized C implemented functions for simple
lambda expressions?

Thomas


From walter at livinglogic.de  Mon Dec  1 07:59:44 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Dec  1 07:59:52 2003
Subject: [Python-Dev] Banishing apply(), buffer(), coerce(), and intern()
In-Reply-To: <000201c3b71b$9facae40$e841fea9@oemcomputer>
References: <000201c3b71b$9facae40$e841fea9@oemcomputer>
Message-ID: <3FCB3B40.5070703@livinglogic.de>

Raymond Hettinger wrote:

> [...]
>>As always, be careful with doing peephole changes to the standard
>>library -- historically, we've seen a 1-5% error rate in these change
>>sets that persists for months or years afterwards.
> 
> FWIW, Walter and I did a bunch of these for Py2.3 and had excellent
> success because of a good process.

If you're willing to review the patch, I'm going to give it a try.

Bye,
    Walter D?rwald



From python at rcn.com  Mon Dec  1 08:59:01 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec  1 08:59:49 2003
Subject: [Python-Dev] Banishing apply(), buffer(), coerce(), and intern()
In-Reply-To: <3FCB3B40.5070703@livinglogic.de>
Message-ID: <000501c3b813$46249d80$e841fea9@oemcomputer>

> Raymond Hettinger wrote:
> 
> > [...]
> >>As always, be careful with doing peephole changes to the standard
> >>library -- historically, we've seen a 1-5% error rate in these
change
> >>sets that persists for months or years afterwards.
> >
> > FWIW, Walter and I did a bunch of these for Py2.3 and had excellent
> > success because of a good process.

[Walter]
> If you're willing to review the patch, I'm going to give it a try.

The honor of second review should go to Anthony ;-)  This is his itch.
If a third review is needed to make it error free, I would be happy to
accommodate.


Raymond


From anthony at ekit-inc.com  Mon Dec  1 09:16:36 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Mon Dec  1 09:16:48 2003
Subject: [Python-Dev] Banishing apply(), buffer(), coerce(), and intern() 
In-Reply-To: Message from "Raymond Hettinger" <python@rcn.com> of "Mon,
	01 Dec 2003 08:59:01 CDT." <000501c3b813$46249d80$e841fea9@oemcomputer>
Message-ID: <200312011416.hB1EGaDC023723@maxim.off.ekorp.com>


> [Walter]
> > If you're willing to review the patch, I'm going to give it a try.
> 
> The honor of second review should go to Anthony ;-)  This is his itch.
> If a third review is needed to make it error free, I would be happy to
> accommodate.

I'll see how I go for time - with the 2.3.3 release coming up, I'm
unlikely to have a whole lot of spare cycles.

Anthony

From anthony at interlink.com.au  Mon Dec  1 09:22:36 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Mon Dec  1 09:22:50 2003
Subject: [Python-Dev] 2.3.3 cycle
Message-ID: <200312011422.hB1EMa9d023801@maxim.off.ekorp.com>


Ok, my email mailbox is still on tape at the moment, so I'm going
from memory here. My current plan (assuming this meets with everyone's
approval) is to shoot for a 2.3.3rc1 either Friday this week or else
on this weekend, and a final 2.3.3 release around the 17th. How's this
fit in with other people's plans? In particular Thomas or Tim, whoever's
going to need to do the Windows release.

I'll be spending Wednesday working on autoconf magic - I'll see how the
latest autoconf-generated stuff works on the HP testdrive farm. If it 
shows no problems and fixes a bunch of the existing build problems, I'll
look at checking it in. I'm not entirely sold on this, though, so could
probably be convinced to leave it until 2.3.4, whenever we decide to do
that. Given the long time until 2.4, I could see a 2.3.4 around May next
year.

I'm going to start hanging out on #python-dev for the next few weeks
if people want more real-time communications.

Anthony

From skip at pobox.com  Mon Dec  1 09:56:36 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec  1 09:56:49 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Libary
In-Reply-To: <000901c3b45f$b600bba0$5ab0958d@oemcomputer>
References: <000901c3b45f$b600bba0$5ab0958d@oemcomputer>
Message-ID: <16331.22180.131561.468695@montanaro.dyndns.org>


    Raymond> Some other candidates (let's pick just a two or three):

    Raymond> - csv (basic tool for sharing data with other applications)

-0.  I think basic usage is covered pretty well in the libref.  At best, I'd
just mention that it exists and link to the libref.

    Raymond> - datetime (comes up frequently in real apps and admin tasks)

+1.  Discussing how datetime and time integrate would be useful.

    Raymond> - ftplib (because the examples are so brief)

-1.  I would think it's rarely used.

    Raymond> - getopt or optparse (because the task is common)

+1.

    Raymond> - operator (because otherwise, the functionals can be a PITA)

-1.  The most common case people needs is now covered by a builtin (sum).

    Raymond> - pprint (because beauty counts)

+0.  Brief mention at best.

    Raymond> - struct (because fixed record layouts are common)

-0.  Only for propeller heads.

    Raymond> - threading/Queue (because without direction people grab thread
    Raymond>   and mutexes)

+0.  I agree that stumbling on thread is too common.  OTOH, threads in
general are a pretty advanced topic, and probably not real suitable for the
tutorial.

    Raymond> - timeit (because it answers most performance questions in a
    Raymond>   jiffy)

-1.  Newbies should probably not be worried about performance too much.  In
addition, I think most performance questions are deeper than those which can
be answered by timeit (think naive O(n^2) algorithms).

    Raymond> - unittest (because TDD folks like myself live by it)

+1.  There's no time like the present to start adding tests.

    Raymond> I've avoided XML because it is a can of worms and because short
    Raymond> examples don't do it justice.  OTOH, it *is* the hot topic of
    Raymond> the day and seems to be taking over the world one angle bracket
    Raymond> at a time.

-1.  XML is too complex for tutorial material.

Skip

From skip at pobox.com  Mon Dec  1 10:12:25 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec  1 10:12:49 2003
Subject: [Python-Dev] possible backward incompatibility in test.regrtest
In-Reply-To: <200311300124.hAU1O1N21082@grad.sccs.chukyo-u.ac.jp>
References: <200311300124.hAU1O1N21082@grad.sccs.chukyo-u.ac.jp>
Message-ID: <16331.23129.160126.321827@montanaro.dyndns.org>


    Tamito> It seems that the test.regrtest module has a possible backward
    Tamito> incompatibility with regard to pre-Python 2.3 releases.  I have
    Tamito> a test suit implemented using the test.regrtest module.  

    ...

    Tamito> Unless I miss something trivial (I hope so), I'd have to give up
    Tamito> using the test.regrtest module.

I think this is the best course.  The advertised (and supported) way to
build test suites in Python is with the unittest module.

Skip

From pycon at python.org  Mon Dec  1 10:55:36 2003
From: pycon at python.org (Steve Holden)
Date: Mon Dec  1 10:55:38 2003
Subject: [Python-Dev] PyCon DC 2004 - Registration about to open!
Message-ID: <E1AQqOe-0003J2-9d@mail.python.org>


Dear Python User:

You have probably been wondering why you haven't yet heard
that PyCon has opened its doors for registration. Well, we
are a volunteer organization, and it often takes more time
than anticipated to put things together. Welcome to the
world of open source :-)

I am happy to be able to tell you that thanks to sterling
work by Trevor Toenjes we are now in a position to accept
your registrations. All you need to do is navigate to

    http://dc2004reg.pycon.org/

and make the appropriate entries. This year we all have the
opportunity to add a PSF donation to the registration fee,
and I hope many of you will follow my example and donate a
little extra to assist the Foundation in its goals.

There is still lots to be done -- for example, we are *also*
late with the system to accept submissions, so the original
December 1 deadline will be extended by at least a month to
ensure that everyone gets plenty of chance to submit papers
telling us what they've been up to with Python.

We are also encouraging students to attend the conference by
giving them a significant reduction from the standard fee,
and we hope that this will result in even wider participation
by the educational community.

Last year was an exciting departure from the traditional
"professional" conference format, and I hope that we will be
able to improve on it next time. I look forward to seeing you
there!

Sincerely

Steve Holden
Chairman
PyCon DC 2004

From guido at python.org  Mon Dec  1 11:15:20 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 11:15:27 2003
Subject: [Python-Dev] "groupby" iterator
In-Reply-To: Your message of "Sun, 30 Nov 2003 21:20:16 EST."
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com> 
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com> 
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com> 
Message-ID: <200312011615.hB1GFKf30570@c-24-5-183-134.client.comcast.net>

> The chaining part, or the idea at all?

Sorry, I didn't even see the chaining idea, so it was the keyword
attribute instead of two different functions I disliked.  The chaining
idea seems unnecessary, you can do this with a general currying
facility.  (I'm not so keen on copying too many such ideas from Zope
in to Python -- together they feel more like a bag of clever tricks
than like a well-thought-out language.)

Anyway, it's all moot -- Raymond just added operator.itemgetter and
operator.attrgetter.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec  1 11:18:20 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 11:18:33 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Sun, 30 Nov 2003 23:35:56 EST."
	<003301c3b7c4$b9f1a400$e841fea9@oemcomputer> 
References: <003301c3b7c4$b9f1a400$e841fea9@oemcomputer> 
Message-ID: <200312011618.hB1GILD30610@c-24-5-183-134.client.comcast.net>

> P.S.  I'm leaning toward Alex's suggested argument order.  Having a
> default identity function is too attractive to pass up.  So the choice
> is between a style like map(None, s) or something closer to
> list.sorted(s, key=).   Though the latter is not consistent with other
> itertools, it wins in the beauty department and its similarity with the
> key= is a accurate, helpful analogy.

Good.  Have you thought about the name yet?  Will it be grouped() or
grouby()?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From gerrit at nl.linux.org  Mon Dec  1 13:30:30 2003
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Mon Dec  1 13:32:20 2003
Subject: [Python-Dev] adding save() and load() function for interactive use
Message-ID: <20031201183030.GA5043@nl.linux.org>

Hello,

I have a proposal to enhance the usefulness of Python for learning
purposes. RFE#852222:

 I think that for learning Python, it would be nice to
 save the current namespace to a file using a builtin
 save() function (and corresponding load()). For
 beginning programmers, it would be nice to be able to
 tinker interactively with the shell, defining functions
 and maybe even classes, with which they can continue
 later on. save() is easily implemented by pickling
 locals(); load() is easily implemented by updating
 locals() while unpickling it. The functions would take
 a filename (string) as an argument. In Idle, they could be
 in the menu. Like the _ variable, load() and save() are
 only present in the interactive shell (some global
 .pythonrc?). Example usage:

 >>> def foo():
 ...    raw_input("Do you like it?" )
 ...    print "Great!"
 >>> save("~/session")


 >>> load("~/session")
 >>> foo()
 Do you like it? yes
 Great!

Note that this code is untested.

I think this would be a nice addition to Python 2.4.

yours,
Gerrit.

-- 
105. If the agent is careless, and does not take a receipt for the
money which he gave the merchant, he can not consider the unreceipted
money as his own.
          -- 1780 BC, Hammurabi, Code of Law
-- 
Asperger's Syndrome - a personal approach:
	http://people.nl.linux.org/~gerrit/english/

From mcherm at mcherm.com  Mon Dec  1 14:00:30 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Dec  1 14:00:47 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Library
Message-ID: <1070305230.3fcb8fce50934@mcherm.com>

Raymond> - timeit (because it answers most performance questions in a
Raymond>   jiffy)

Skip> -1.  Newbies should probably not be worried about performance too
Skip> much.  In addition, I think most performance questions are deeper
Skip> than those which can be answered by timeit (think naive O(n^2)
Skip> algorithms).

Skip is right on both counts: newbies DO worry too much about performance,
and they WON'T be able to think deeply about performance issues like how
an algorithm scales (unless they're new to Python but not to programming).
But I still agree with Raymond that timeit should be included.

The fact is, newbies ARE going to worry too much about performance,
whether we want them to or not. And a VERY frequently asked question
on c.l.py is "what's the fastest way to do X". IMHO, learning to answer
this kind of question themselves is MORE important than learning not to
ask the question when it isn't needed -- just like learning to use the
interactive interpreter to answer "does the syntax X work?", it makes
them more self sufficient, and therefore more self confident.

So I'd say DO include a (brief!) tutorial example on using timeit. But
perhaps craft the example so it shows two approaches, one readable
and one obscure, and the obscure one is shown to be FASTER but not
MUCH faster, and is therefore DROPPED in favor of the more readable.
(Hey... rigging the problem set in order to model good practices is an
old teacher's trick I've used often.)

And having piped up with my own 2c, I'd better pay up. Raymond... if
you're working on this and need help, send me an example or two of other
modules (so I can try to follow your style) and I'll volunteer to write
one for timeit.

-- Michael Chermside

From guido at python.org  Mon Dec  1 14:07:11 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 14:07:32 2003
Subject: [Python-Dev] adding save() and load() function for interactive use
In-Reply-To: Your message of "Mon, 01 Dec 2003 19:30:30 +0100."
	<20031201183030.GA5043@nl.linux.org> 
References: <20031201183030.GA5043@nl.linux.org> 
Message-ID: <200312011907.hB1J7BU31066@c-24-5-183-134.client.comcast.net>

>  I think that for learning Python, it would be nice to
>  save the current namespace to a file using a builtin
>  save() function (and corresponding load()). For
>  beginning programmers, it would be nice to be able to
>  tinker interactively with the shell, defining functions
>  and maybe even classes, with which they can continue
>  later on. save() is easily implemented by pickling
>  locals(); load() is easily implemented by updating
>  locals() while unpickling it. The functions would take
>  a filename (string) as an argument. In Idle, they could be
>  in the menu. Like the _ variable, load() and save() are
>  only present in the interactive shell (some global
>  .pythonrc?). Example usage:
[omitted]

Saving classes and functions with pickle doesn't work; it *appeared*
to work only because you didn't actually quit your session.

IDLE has a feature designed to further the same goal without having to
save a "workspace" (a vague concept at best and one that doesn't scale
to larger Python apps).  You can configure IDLE to come up with a
blank untitled Python file instead of a Python Shell window.  Then the
student's work cycle is intended to be as follows:

  - Type some code into the file.

  - Save it (the first time the student has to pick a filename).

  - Hit F5 which will run the file (which is now a module); this will
    open a Python Shell (if one isn't already open) so you can
    experiment with the code defined in the file, but these
    experiments won't be saved.

  - Go back to the top to edit the file and run it again.

There's also an option to avoid the save step (F5 will save
automatically once a filename is picked).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Mon Dec  1 14:06:57 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec  1 14:08:32 2003
Subject: [Python-Dev] adding save() and load() function for interactive use
In-Reply-To: <20031201183030.GA5043@nl.linux.org>
Message-ID: <5.1.1.6.0.20031201140528.024c3b90@telecommunity.com>

At 07:30 PM 12/1/03 +0100, Gerrit Holl wrote:
>save() is easily implemented by pickling
>  locals(); load() is easily implemented by updating
>  locals() while unpickling it.

Functions and classes are pickled as references to a module, so this 
wouldn't actually work.  When you try to load(), the unpickling will fail 
if the function or class doesn't already exist in the module.


From python at rcn.com  Mon Dec  1 14:22:32 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec  1 14:23:22 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312011618.hB1GILD30610@c-24-5-183-134.client.comcast.net>
Message-ID: <002a01c3b840$783a24c0$e841fea9@oemcomputer>

[David Eppstein]
> My implementation will skip or partially consume the subiterators,
with
> only a very temporary additional use of memory, if you don't keep a
> reference to them.
> But I can see your arguments about visible vs silent failure modes and
> code
> complexity.

I'll add note saying that the behavior may change.  Then, if I was
wrong, we can add support for delinquent subiterators.



[Guido]
>Have you thought about the name yet?  Will it be grouped() or
> grouby()?

This question is still open.

The grouped() name fits well with its list.sorted() and reversed().
Also, it is more natural in context of a default identity function.  

The groupby() name better suggests what's going on under the hood.  The
strong association with SQL is a plus because the analogy is accurate.

I'm leaning away from grouped() because it seems vague and lifeless.
Other tools like izip() and the prospective window() and roundrobin()
functions could also be said to return groups.


[After Guido suggests introducing an SQLlike itertools.count()]
I think this belongs in a new module for accumulator functions including
average, standard deviation, and such.  In the meantime, len(list(grp))
will suffice.

That is even more reasonable when multiple accumulation functions are
used because a list has to be created anyway:

for k, g in groupby(s, key=keyfunc):
     data = list(g)
     print s, len(data), sum(data), average(data)


[Guido on chained attribute lookups]
>Anyway, it's all moot -- Raymond just added operator.itemgetter 
> and operator.attrgetter.

The implementation does not preclude someone adding support for chained
attribute lookups.  IMO, the non-chained version covers the most common
use cases simply and directly.  For complex structures, lambda already
has plenty of versatility: 

    key = lambda r: r.roster[3].personalinfo.physicaladdress.zipcode

IIRC, Tim recently yearned for the days when the lack of support for
nested structures pushed people towards flatter designs.  With him
already looking forward to death, we dare not add to his angst.



Raymond Hettinger



From gerrit at nl.linux.org  Mon Dec  1 14:30:55 2003
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Mon Dec  1 14:31:25 2003
Subject: [Python-Dev] adding save() and load() function for interactive use
In-Reply-To: <5.1.1.6.0.20031201140528.024c3b90@telecommunity.com>
References: <20031201183030.GA5043@nl.linux.org>
	<5.1.1.6.0.20031201140528.024c3b90@telecommunity.com>
Message-ID: <20031201193054.GA5336@nl.linux.org>

Phillip J. Eby wrote:
> At 07:30 PM 12/1/03 +0100, Gerrit Holl wrote:
> >save() is easily implemented by pickling
> > locals(); load() is easily implemented by updating
> > locals() while unpickling it.
> 
> Functions and classes are pickled as references to a module, so this 
> wouldn't actually work.  When you try to load(), the unpickling will fail 
> if the function or class doesn't already exist in the module.

Ouch. Of course... I should have thought it through better... sorry!

yours,
Gerrit.

-- 
219. If a physician make a large incision in the slave of a freed man,
and kill him, he shall replace the slave with another slave.
          -- 1780 BC, Hammurabi, Code of Law
-- 
Asperger's Syndrome - a personal approach:
	http://people.nl.linux.org/~gerrit/english/

From mwh at python.net  Mon Dec  1 14:35:13 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec  1 14:35:21 2003
Subject: [Python-Dev] CVS bustage?
Message-ID: <2mwu9gxrf2.fsf@starship.python.net>

I'm seeing test_trace dump core from current CVS.  Anyone else?

Slightly worryingly, working backwards in CVS isn't making it stop, so
it might just be something silly at my end...

Cheers,
mwh

-- 
  I have no disaster recovery plan for black holes, I'm afraid.  Also
  please be aware that if it one looks imminent I will be out rioting
  and setting fire to McDonalds (always wanted to do that) and
  probably not reading email anyway.                     -- Dan Barlow

From guido at python.org  Mon Dec  1 14:47:17 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 14:48:46 2003
Subject: [Python-Dev] CVS bustage?
In-Reply-To: Your message of "Mon, 01 Dec 2003 19:35:13 GMT."
	<2mwu9gxrf2.fsf@starship.python.net> 
References: <2mwu9gxrf2.fsf@starship.python.net> 
Message-ID: <200312011947.hB1JlHo31202@c-24-5-183-134.client.comcast.net>

> I'm seeing test_trace dump core from current CVS.  Anyone else?

Not here (Red Hat 9, production build).

> Slightly worryingly, working backwards in CVS isn't making it stop, so
> it might just be something silly at my end...

Most likely.  What platform?  What build (debug or prod.)?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Mon Dec  1 14:54:11 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec  1 14:54:15 2003
Subject: [Python-Dev] CVS bustage?
In-Reply-To: <200312011947.hB1JlHo31202@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 01 Dec 2003 11:47:17 -0800")
References: <2mwu9gxrf2.fsf@starship.python.net>
	<200312011947.hB1JlHo31202@c-24-5-183-134.client.comcast.net>
Message-ID: <2m8ylwjov0.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> I'm seeing test_trace dump core from current CVS.  Anyone else?
>
> Not here (Red Hat 9, production build).

Nor anyone on #python, either.

>> Slightly worryingly, working backwards in CVS isn't making it stop, so
>> it might just be something silly at my end...
>
> Most likely.  

Yes, sigh.  I think maybe I should give up on today.

> What platform? 

Redhat 9...

> What build (debug or prod.)?

Both.

Cheers,
mwh

-- 
  If your telephone company installs a system in the woods with no
  one around to see them, do they still get it wrong?
                                 -- Robert Moir, alt.sysadmin.recovery

From tim.one at comcast.net  Mon Dec  1 15:07:03 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec  1 15:07:13 2003
Subject: [Python-Dev] CVS bustage?
In-Reply-To: <2mwu9gxrf2.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEPEHHAB.tim.one@comcast.net>

[Michael Hudson]
> I'm seeing test_trace dump core from current CVS.  Anyone else?

Works on Windows, under release and debug builds.

test_winreg fails, though:

    http://www.python.org/sf/852281

From mcherm at mcherm.com  Mon Dec  1 15:12:08 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Dec  1 15:13:13 2003
Subject: [Python-Dev] possible backward incompatibility in test.regrtest
Message-ID: <1070309528.3fcba09899c9e@mcherm.com>

> Unless I miss something trivial (I hope so), I'd have to give up
> using the test.regrtest module.  I appreciate any comment.

Maybe the thing to do is to change:
>  if test.startswith('test.'):
>      abstest = test
>  else:
>      # Always import it from the test package
>      abstest = 'test.' + test
>  the_package = __import__(abstest, globals(), locals(), [])

to this:

>  if test.startswith('mytest.'):
>      abstest = test
>  else:
>      # Always import it from the test package
>      abstest = 'mytest.' + test
>  the_package = __import__(abstest, globals(), locals(), [])

in your own copy of regrtest. Converting to unittest would be better,
but this is a fix that would at least prevent immediate breakage.

-- Michael Chermside


From skip at pobox.com  Mon Dec  1 15:35:33 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec  1 15:35:44 2003
Subject: [Python-Dev] save()/load()
Message-ID: <16331.42517.265132.36606@montanaro.dyndns.org>

I lost the original thread, but my first thought was to use the readline
module's history capability to save and reload a "session".  This hack
partially works for what little testing I've done (place it in
sitecustomize.py):

    import readline
    import os
    import sys

    _start_len = len(open(os.path.expanduser("~/.pyhist")).readlines())

    _session_file = os.path.expanduser("~/.pysession")

    def save(f=_session_file, start=_start_len):
        sf = open(f, 'w')
        while True:
            line = readline.get_history_item(start)
            start += 1
            if line is None:
                break
            if line.startswith('save(') or line.startswith('load('):
                continue
            sf.write(line+"\n")

    def load(f=_session_file):
        execfile(f, sys._getframe(-1).f_globals)

    import __builtin__
    __builtin__.save = save
    __builtin__.load = load

My readline history is stored in ~/.pyhist.  I can't use
readline.get_history_length() to mark the start of a session for reasons
which are not obvious to me.  When I first started futzing around with it,
readline.get_history_length() returned 9934.  Now, even though my ~/.pyhist
file has more than 9960 lines, get_history_length() seems to always return
-1, thus making it useless in this context.

Here's a simple session showing save() and load() within the same session:

    >>> import math
    >>> print math.sin(47)
    0.123573122745
    >>> def f():
    ...   return math.sin(48)
    ... 
    >>> save()
    >>> load()
    0.123573122745
    >>> f
    <function f at 0x431830>
    >>> f()
    -0.76825466132366682

Note that f may not be a new f, however.  A call to load() in a fresh
session fails to place 'f' in the session's globals.  I'm not sure why.  Any
ideas?

Skip

From pje at telecommunity.com  Mon Dec  1 16:04:07 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec  1 16:07:09 2003
Subject: [Python-Dev] save()/load()
In-Reply-To: <16331.42517.265132.36606@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20031201160258.01e12700@telecommunity.com>

At 02:35 PM 12/1/03 -0600, Skip Montanaro wrote:

>My readline history is stored in ~/.pyhist.  I can't use
>readline.get_history_length() to mark the start of a session for reasons
>which are not obvious to me.

get_history_length() is the length you've set for truncation, not the 
actual length.  You want get_current_history_length(), IIRC.


From python at rcn.com  Mon Dec  1 16:30:19 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec  1 16:31:26 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Library
In-Reply-To: <1070305230.3fcb8fce50934@mcherm.com>
Message-ID: <003101c3b852$51f2c6c0$e841fea9@oemcomputer>

 [Michael Chermside]
> So I'd say DO include a (brief!) tutorial example on using timeit. But
> perhaps craft the example so it shows two approaches, one readable
> and one obscure, and the obscure one is shown to be FASTER but not
> MUCH faster, and is therefore DROPPED in favor of the more readable.
> (Hey... rigging the problem set in order to model good practices is an
> old teacher's trick I've used often.)

Especially, in the tutorial, stacking the deck is fair play.


> And having piped up with my own 2c, I'd better pay up. Raymond... if
> you're working on this and need help, send me an example or two of
other
> modules (so I can try to follow your style) and I'll volunteer to
write
> one for timeit.

I expect to be able to wrap it up this weekend.


Raymond


From perky at i18n.org  Mon Dec  1 16:38:18 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Mon Dec  1 16:39:32 2003
Subject: [Python-Dev] "groupby" iterator
In-Reply-To: <200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0311271535490.1066-100000@tane.cfa.harvard.edu>
	<200311272149.hARLneU16201@c-24-5-183-134.client.comcast.net>
	<20031128195420.GA63319@i18n.org>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
Message-ID: <20031201213818.GA20382@i18n.org>

On Fri, Nov 28, 2003 at 01:46:53PM -0800, Guido van Rossum wrote:
> I would make one change: after looking at another use case, I'd like
> to change the outer iterator to produce (key, grouper) tuples.  This
> way, you can write things like
> 
>   totals = {}
>   for key, group in sequence:
>       totals[key] = sum(group)
> 

I think this would be helpful for lazy coders if some function of
itertools cover the use case: (`LIMIT' keyword of SQL)

>>> from groupby import groupby
>>> alwaystrue = lambda n: True
>>> for k, g in groupby(alwaystrue, range(20), 5):
...     print list(g)
...
[0, 1, 2, 3, 4]
[5, 6, 7, 8, 9]
[10, 11, 12, 13, 14]
[15, 16, 17, 18, 19]

prototype of this case is groupby(keyfunc, iterable, limit=None).
Either, groupby(5, range(20)) is okay but 5 is not a sort of `key'. :)


Hye-Shik

From walter at livinglogic.de  Mon Dec  1 16:42:09 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Dec  1 16:42:26 2003
Subject: [Python-Dev] Banishing apply(), buffer(), coerce(), and intern()
In-Reply-To: <000501c3b813$46249d80$e841fea9@oemcomputer>
References: <000501c3b813$46249d80$e841fea9@oemcomputer>
Message-ID: <3FCBB5B1.8000602@livinglogic.de>

Raymond Hettinger wrote:

> [...]
>>>FWIW, Walter and I did a bunch of these for Py2.3 and had excellent
>>>success because of a good process.
> 
> [Walter]
> 
>>If you're willing to review the patch, I'm going to give it a try.
> 
> The honor of second review should go to Anthony ;-)  This is his itch.
> If a third review is needed to make it error free, I would be happy to
> accommodate.

OK, the patch is done (http://www.python.org/sf/852334) and assigned
to Anthony.

I couldn't upload the patch itself, as it's too big, so the patch can
be found at http://styx.livinglogic.de/~walter/backticks2repr.txt

Bye,
    Walter D?rwald



From skip at pobox.com  Mon Dec  1 17:10:15 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec  1 17:11:21 2003
Subject: [Python-Dev] save()/load()
In-Reply-To: <5.1.1.6.0.20031201160258.01e12700@telecommunity.com>
References: <16331.42517.265132.36606@montanaro.dyndns.org>
	<5.1.1.6.0.20031201160258.01e12700@telecommunity.com>
Message-ID: <16331.48199.502155.614605@montanaro.dyndns.org>


    >> My readline history is stored in ~/.pyhist.  I can't use
    >> readline.get_history_length() to mark the start of a session for
    >> reasons which are not obvious to me.

    Phillip> get_history_length() is the length you've set for truncation,
    Phillip> not the actual length.  You want get_current_history_length(),
    Phillip> IIRC.

Yes, thanks.  I decided to dispense with accessing the "start" of the
session and force the user to simply mark() where a session is to start.
That's one extra function to remember, but seems a lot more flexible.

Appended is my current incarnation.  Still doesn't load() right, though
save() seems to work.  Usage is:

    mark()
    ... bunch of commands at the interpreter prompt ...
    save()

Any clues about what I'm doing wrong in load()?  I want to stuff the
evaluated objects into the globals() of the caller (that is, the globals for
the interactive session in most situations).

Skip

import readline
import os
import sys

# needs to match the filename the user uses!
_histfile = os.path.expanduser("~/.pyhist")

# where we save sessions
_session_file = os.path.expanduser("~/.pysession")

# mark with something valid at the interpreter but unlikely to be executed
# by the user
_marker = '((1.0+999.0j, "mark", 999.0-1.0j))'

def save():
    end = readline.get_current_history_length() - 1
    session = []
    item = readline.get_history_item(end)
    while item != _marker:
        session.insert(0, item+"\n")
        end -= 1
        item = readline.get_history_item(end)
        
    file(_session_file, 'w').writelines(session)
    print >> sys.stderr, "saved session to", _session_file

def load():
    execfile(_session_file, sys._getframe(-1).f_globals)
    print >> sys.stderr, "loaded session from", _session_file

def mark():
    readline.add_history(_marker)

import __builtin__
__builtin__.save = save
__builtin__.load = load
__builtin__.mark = mark

From pje at telecommunity.com  Mon Dec  1 17:26:47 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec  1 17:28:31 2003
Subject: [Python-Dev] save()/load()
In-Reply-To: <16331.48199.502155.614605@montanaro.dyndns.org>
References: <5.1.1.6.0.20031201160258.01e12700@telecommunity.com>
	<16331.42517.265132.36606@montanaro.dyndns.org>
	<5.1.1.6.0.20031201160258.01e12700@telecommunity.com>
Message-ID: <5.1.1.6.0.20031201172602.024c1cd0@telecommunity.com>

At 04:10 PM 12/1/03 -0600, Skip Montanaro wrote:
>def load():
>     execfile(_session_file, sys._getframe(-1).f_globals)
>     print >> sys.stderr, "loaded session from", _session_file

What does getframe with a negative number do?  I've only ever used positive 
numbers when calling getframe.  I think maybe you want 1 here, not -1.


From guido at python.org  Mon Dec  1 18:57:28 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 18:57:47 2003
Subject: [Python-Dev] "groupby" iterator
In-Reply-To: Your message of "Tue, 02 Dec 2003 06:38:18 +0900."
	<20031201213818.GA20382@i18n.org> 
References: <Pine.LNX.4.44.0311271535490.1066-100000@tane.cfa.harvard.edu>
	<200311272149.hARLneU16201@c-24-5-183-134.client.comcast.net>
	<20031128195420.GA63319@i18n.org>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net> 
	<20031201213818.GA20382@i18n.org> 
Message-ID: <200312012357.hB1NvSI32310@c-24-5-183-134.client.comcast.net>

> I think this would be helpful for lazy coders if some function of
> itertools cover the use case: (`LIMIT' keyword of SQL)
> 
> >>> from groupby import groupby
> >>> alwaystrue = lambda n: True
> >>> for k, g in groupby(alwaystrue, range(20), 5):
> ...     print list(g)
> ...
> [0, 1, 2, 3, 4]
> [5, 6, 7, 8, 9]
> [10, 11, 12, 13, 14]
> [15, 16, 17, 18, 19]
> 
> prototype of this case is groupby(keyfunc, iterable, limit=None).
> Either, groupby(5, range(20)) is okay but 5 is not a sort of `key'. :)

I'd rather not weigh down groupby() with more options.  If you really
want only the first 5 of each group, you can use itertools.islice():

  for k, g in groupby(alwaystrue, range(20)):
     print list(itertools.islice(g, 0, 5))

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Mon Dec  1 20:00:23 2003
From: aahz at pythoncraft.com (Aahz)
Date: Mon Dec  1 20:00:26 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200311300006.hAU06Lp19846@c-24-5-183-134.client.comcast.net>
References: <200311240434.hAO4Y4L06979@c-24-5-183-134.client.comcast.net>
	<20031125203258.GA29814@i92.ryd.student.liu.se>
	<200311300006.hAU06Lp19846@c-24-5-183-134.client.comcast.net>
Message-ID: <20031202010023.GA4439@panix.com>

On Sat, Nov 29, 2003, Guido van Rossum wrote:
>
> (2) PEP 237 promises that repr() of a long should no longer show a
>     trailing 'L'.  This is not yet implemented (i.e., repr() of a long
>     still has a trailing 'L').  First, past experience suggests that
>     quite a bit of end user code will break, and it may easily break
>     silently: there used to be code that did str(x)[:-1] (knowing x
>     was a long) to strip the 'L', which broke when str() of a long no
>     longer returned a trailing 'L'.  Apparently some of this code was
>     "fixed" by changing str() into repr(), and this code will now
>     break again.  Second, I *like* seeing a trailing L on longs,
>     especially when there's no reason for it to be a long: if some
>     expression returns 1L, I know something fishy may have gone on.

That makes sense to me; there should be an easy way from Python to
detect what kind of object you've got (as a string representation), and
repr() is precisely the place for it.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From guido at python.org  Mon Dec  1 20:10:25 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 20:10:33 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Mon, 01 Dec 2003 20:00:23 EST."
	<20031202010023.GA4439@panix.com> 
References: <200311240434.hAO4Y4L06979@c-24-5-183-134.client.comcast.net>
	<20031125203258.GA29814@i92.ryd.student.liu.se>
	<200311300006.hAU06Lp19846@c-24-5-183-134.client.comcast.net> 
	<20031202010023.GA4439@panix.com> 
Message-ID: <200312020110.hB21APb32451@c-24-5-183-134.client.comcast.net>

[me]
> > (2) PEP 237 promises that repr() of a long should no longer show a
> >     trailing 'L'.  This is not yet implemented (i.e., repr() of a long
> >     still has a trailing 'L').  First, past experience suggests that
> >     quite a bit of end user code will break, and it may easily break
> >     silently: there used to be code that did str(x)[:-1] (knowing x
> >     was a long) to strip the 'L', which broke when str() of a long no
> >     longer returned a trailing 'L'.  Apparently some of this code was
> >     "fixed" by changing str() into repr(), and this code will now
> >     break again.  Second, I *like* seeing a trailing L on longs,
> >     especially when there's no reason for it to be a long: if some
> >     expression returns 1L, I know something fishy may have gone on.

[Aahz]
> That makes sense to me; there should be an easy way from Python to
> detect what kind of object you've got (as a string representation), and
> repr() is precisely the place for it.

OK.  I've got one weak (-0 from Raymond) opposition to this idea, and
two strong agreements (+1 from Tim and Aahz), so I'm going to go ahead
and make this change to the PEP.  This pretty much makes PEP 237
finished except for the complete and utter eradication of the trailing
'L' (and probably of the 'long' type altogether) in Python 3.0.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Mon Dec  1 20:27:05 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Dec  1 20:27:12 2003
Subject: [Python-Dev] save()/load()
In-Reply-To: <5.1.1.6.0.20031201172602.024c1cd0@telecommunity.com>
Message-ID: <200312020127.hB21R5626908@oma.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>

> What does getframe with a negative number do? 

Obviously it uses the time machine to look into the
future a little and return a frame that is *going*
to be called from the current function...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From ozabluda at yahoo.com  Mon Dec  1 21:53:18 2003
From: ozabluda at yahoo.com (Oleg Zabluda)
Date: Mon Dec  1 21:53:21 2003
Subject: [Python-Dev] Possible bugs and security holes in getpass.py
Message-ID: <20031202025318.97602.qmail@web12208.mail.yahoo.com>

I am comparing getpass.py
(http://cvs.osafoundation.org/index.cgi/osaf/chandler/python/Lib/getpass.py)
and getpass.c from glibc
(http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/misc/getpass.c?rev=1.18&content-type=text/x-cvsweb-markup&cvsroot=glibc).

Here are the differences that I identified, some of which may or may not
be bugs in getpass.py:

1. getpass.c sets ~ISIG in addition to ~ECHO.
2. getpass.c locks "stdin".
3. getpass.c makes sure the "stdin" is closed even if the thread is cancelled.
4. getpass.c explicitly flushes "stdin" after outputting the promt and before
   reading the password.
5. getpass.c opens "stdin" in "c" mode. This sets _IO_FLAGS2_NOTCANCEL,
   whatever that means. Maybe it has something to do with thread cancellation,
   maybe not.
(http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/libio/fileops.c?rev=1.101&content-type=text/x-cvsweb-markup&cvsroot=glibc)

1,2,3,5 are possible security holes in addition to being possible bugs.

Although I don't completely understand all the details, it appears to me
that getpass.c is more correct then getpass.py.

Suggestion/RFC: either implement the same functionality or a portion
thereof in getpass.py, or implement it using getpass(3) directly, at least
when linking with glibc.


More references:
http://www.python.org/doc/current/lib/module-getpass.html
http://www.opengroup.org/onlinepubs/007908799/xsh/getpass.html
'info getpass'

It would help if we knew whether login, passwd, chfn, su, sudo, yppasswd,
etc, use getpass(3) or not. If yes, the answer would be a no-brainer to me.

Oleg.


From jepler at unpythonic.net  Mon Dec  1 22:55:16 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec  1 22:55:27 2003
Subject: [Python-Dev] Possible bugs and security holes in getpass.py
In-Reply-To: <20031202025318.97602.qmail@web12208.mail.yahoo.com>
References: <20031202025318.97602.qmail@web12208.mail.yahoo.com>
Message-ID: <20031202035508.GA4023@unpythonic.net>

On Mon, Dec 01, 2003 at 06:53:18PM -0800, Oleg Zabluda wrote:
> I am comparing getpass.py
> (http://cvs.osafoundation.org/index.cgi/osaf/chandler/python/Lib/getpass.py)
> and getpass.c from glibc
> (http://sources.redhat.com/cgi-bin/cvsweb.cgi/libc/misc/getpass.c?rev=1.18&content-type=text/x-cvsweb-markup&cvsroot=glibc).

Many of the differences you cite seem related to glibc-specific
internals (I don't know what it means to 'lock "stdin"', open 'in "c"
mode', or 'set _IO_FLAGS2_NOTCANCEL', when talking about portable stdio
code, though I'm not a posix/SuS expert).  glibc-specific code is
unlikely to be included in Python, for obvious reasons.

> 4. getpass.c explicitly flushes "stdin" after outputting the promt and before
>    reading the password.

In 2.2, I think the marked lines are supposed to perform those
functions:
    try:
        termios.tcsetattr(fd, termios.TCSADRAIN, new) ### HERE
        passwd = _raw_input(prompt)
    finally:
        termios.tcsetattr(fd, termios.TCSADRAIN, old) ### HERE

> Suggestion/RFC: either implement the same functionality or a portion
> thereof in getpass.py, or implement it using getpass(3) directly, at least
> when linking with glibc.

I think that a _getpass module on systems that provide getpass(3) would
be appropriate.  The block at the bottom of getpass.py would have
another level added to deal with the import of _getpass and its absence:

try:
   import _getpass
except:
   existing block, but indented more
else:
   getpass = _getpass.getpass

There's probably nothing subtle about writing _getpassmodule.c, and
distutils should be able to handle its absence gracefully (?).

Before I'm +1 on doing this, though, here's what the linux (redhat 9)
manpage has to say about getpass:
        SYNOPSIS
               #include <unistd.h>

               char *getpass( const char * prompt );

        DESCRIPTION
               This function is obsolete. Do not use it.
                                ^^^^^^^^
The opengroup web page you mention says it is "LEGACY", with this
explanation:
        This function was marked LEGACY since it provides no functionality
        which a user could not easily implement, and its name is
        misleading.
... how important is getpass, and did the opengroup really
underestimate the subtlety of its implementation that greatly?

Jeff

From pje at telecommunity.com  Mon Dec  1 23:11:20 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec  1 23:09:29 2003
Subject: [Python-Dev] save()/load()
In-Reply-To: <200312020127.hB21R5626908@oma.cosc.canterbury.ac.nz>
References: <5.1.1.6.0.20031201172602.024c1cd0@telecommunity.com>
Message-ID: <5.1.0.14.0.20031201231000.02c97b60@mail.telecommunity.com>

At 02:27 PM 12/2/03 +1300, Greg Ewing wrote:
>"Phillip J. Eby" <pje@telecommunity.com>
>
> > What does getframe with a negative number do?
>
>Obviously it uses the time machine to look into the
>future a little and return a frame that is *going*
>to be called from the current function...

Actually, it appears to return the same result as sys._getframe(0).  Maybe 
it should raise an exception for a negative value.  (OTOH, maybe leaving it 
so that it silently does something meaningless will help discourage people 
from using it so often... ;) )


From skip at pobox.com  Mon Dec  1 23:16:46 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec  1 23:16:51 2003
Subject: [Python-Dev] save()/load()
In-Reply-To: <5.1.0.14.0.20031201231000.02c97b60@mail.telecommunity.com>
References: <5.1.1.6.0.20031201172602.024c1cd0@telecommunity.com>
	<5.1.0.14.0.20031201231000.02c97b60@mail.telecommunity.com>
Message-ID: <16332.4654.696644.838775@montanaro.dyndns.org>

    >> > What does getframe with a negative number do?

    >> Obviously it uses the time machine to look into the future...

    Phillip> Maybe it should raise an exception for a negative value.
    Phillip> (OTOH, maybe leaving it so that it silently does something
    Phillip> meaningless will help discourage people from using it so
    Phillip> often... ;) )

Yeah, whatever. ;-)

At any rate, now that Phillip and Tim straightened me out on my number line
basics, there's a simple session save/restore available at

    http://www.musi-cal.com/~skip/python/save_session.py

Making it work on Windows (where it's probably needed more) is left as an
exercise for the reader.

Skip

From guido at python.org  Mon Dec  1 23:31:56 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  1 23:32:04 2003
Subject: [Python-Dev] Possible bugs and security holes in getpass.py
In-Reply-To: Your message of "Mon, 01 Dec 2003 18:53:18 PST."
	<20031202025318.97602.qmail@web12208.mail.yahoo.com> 
References: <20031202025318.97602.qmail@web12208.mail.yahoo.com> 
Message-ID: <200312020431.hB24Vu932651@c-24-5-183-134.client.comcast.net>

> 1,2,3,5 are possible security holes in addition to being possible bugs.
> 
> Although I don't completely understand all the details, it appears
> to me that getpass.c is more correct then getpass.py.

Sorry, but this just doesn't make sense.  There are so many
differences between C and Python that you can't just compare a C and a
Python version of a function and pointing at the differences as
possible security holes or bugs.  If you want to be helpful, I please
try to understand the details, and then see if there are *actual* bugs
or security holes instead of just "possible" ones.

Looking for security issues is serious business.  (It pays my
bills. :-)  But people shouldn't go around pointing out "possible"
security holes without understanding what they are talking about --
spreading fear doesn't help real security.  It is unlikely that a
beginning programmer can find a security hole in a piece of software
without dumb luck.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Tue Dec  2 06:49:16 2003
From: mwh at python.net (Michael Hudson)
Date: Tue Dec  2 06:49:20 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib site.py,
	1.53.6.1, 1.53.6.2
In-Reply-To: <E1AR4Jv-00081b-00@sc8-pr-cvs1.sourceforge.net>
	(jhylton@users.sourceforge.net's
	message of "Mon, 01 Dec 2003 22:47:39 -0800")
References: <E1AR4Jv-00081b-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2m1xrnjv7n.fsf@starship.python.net>

jhylton@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Lib
> In directory sc8-pr-cvs1:/tmp/cvs-serv30842
>
> Modified Files:
>       Tag: release23-maint
> 	site.py 
> Log Message:
> A hack:  Disable import of re, because sre doesn't compile right.
>
> DON'T MERGE THIS CHANGE TO THE TRUNK.

Was this meant to be on the ast-branch?  If not, what's going on?

I don't think breaking uninstalled builds on any platform that isn't
linux/x86 is a good solution for anything.

Cheers,
mwh

-- 
7. It is easier to write an incorrect program than understand a
   correct one.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From skip at pobox.com  Tue Dec  2 07:53:23 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Dec  2 07:53:24 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib site.py,
	1.53.6.1, 1.53.6.2
In-Reply-To: <E1AR4Jv-00081b-00@sc8-pr-cvs1.sourceforge.net>
References: <E1AR4Jv-00081b-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <16332.35651.448681.927287@montanaro.dyndns.org>


    jhylton> Update of /cvsroot/python/python/dist/src/Lib
    jhylton> In directory sc8-pr-cvs1:/tmp/cvs-serv30842

    jhylton> Modified Files:
    jhylton>       Tag: release23-maint
                        ^^^^^^^^^^^^^^^

    jhylton>    site.py 
    jhylton> Log Message:
    jhylton> A hack:  Disable import of re, because sre doesn't compile right.

    jhylton> DON'T MERGE THIS CHANGE TO THE TRUNK.

Jeremy,

Should this be on the release23-maint branch?

Skip

From mmclay at comcast.net  Tue Dec  2 08:27:46 2003
From: mmclay at comcast.net (Michael McLay)
Date: Tue Dec  2 08:24:37 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312020110.hB21APb32451@c-24-5-183-134.client.comcast.net>
References: <200311240434.hAO4Y4L06979@c-24-5-183-134.client.comcast.net>
	<20031202010023.GA4439@panix.com>
	<200312020110.hB21APb32451@c-24-5-183-134.client.comcast.net>
Message-ID: <200312020827.46280.mmclay@comcast.net>

On Monday 01 December 2003 08:10 pm, Guido van Rossum wrote:
This pretty much makes PEP 237
> finished except for the complete and utter eradication of the trailing
> 'L' (and probably of the 'long' type altogether) in Python 3.0.

Would there still be an int type and a long type in Python 3.0, or would the 
notion of a long be be dropped. If it were dropped, then the int 
representation would be transparently represented as a long if the size of 
the number could not fit in an int. If long is dropped then the long function 
could be added to the list of builtins that will disappear in Python 3.0.



From pycon at python.org  Tue Dec  2 09:10:15 2003
From: pycon at python.org (Steve Holden)
Date: Tue Dec  2 09:10:17 2003
Subject: [Python-Dev] PyCon DC 2004 - Submissions Deadline Extended
Message-ID: <E1ARBEF-0003im-Hg@mail.python.org>


Dear Python User:

We have received many enquiries asking "When and how will
I be able to submit my paper to PyCon DC 2004?". This is
encouraging news - from the correspondence to date it seems
that there will be some interesting and exciting presentations
next March.

Please be patient as the system to handle submissions goes
through to completion. When it is ready, look for a further
announcement in the same place as this one.

Just a reminder, while I have your attention. Submissions
should ideally be in a single file, containing HTML or the
reStructured Text format. At a pinch we will be prepared to
accept PDF, and either tar or zip files containing sets of
coordinated documents (such as an HTML master and the graphics
to which it refers). Ulitmately we would like to publish all
accepted papers on the web, and these rules should make it
easier to do so.

If your paper is accepted and you prepare an electronic
presentation (in PDF, PythonPoint or PowerPoint) we will
also happily publish that on the web site when PyCon is over.

So, have at it! Anything describing the uses of Python, or
better ways of teaching it, neat tricks for developers and
suggestions for improvement to the language or its community
is fair game. It's likely we will have a larger proportion of
students attending in 2004, so we are particularly encouraging
presentations with an educational spin, including material
of a more introductory nature.

Don't forget, too, that even if you don't want to make a formal
presentation, the Open Space sessions and other informal activities
will give everyone plenty of chance to contribute what they know
and discuss what they need. PyCon should be *the* place to go
to learn more about Python. Please help to ensure this is so.

Sincerely

Steve Holden
Chairman
PyCon DC 2004

From jeremy at alum.mit.edu  Tue Dec  2 09:54:26 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Tue Dec  2 09:55:56 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Lib site.py,
	1.53.6.1, 1.53.6.2
In-Reply-To: <2m1xrnjv7n.fsf@starship.python.net>
References: <E1AR4Jv-00081b-00@sc8-pr-cvs1.sourceforge.net>
	<2m1xrnjv7n.fsf@starship.python.net>
Message-ID: <1070376865.2609.0.camel@localhost.localdomain>

On Tue, 2003-12-02 at 06:49, Michael Hudson wrote:
> jhylton@users.sourceforge.net writes:
> 
> > Update of /cvsroot/python/python/dist/src/Lib
> > In directory sc8-pr-cvs1:/tmp/cvs-serv30842
> >
> > Modified Files:
> >       Tag: release23-maint
> > 	site.py 
> > Log Message:
> > A hack:  Disable import of re, because sre doesn't compile right.
> >
> > DON'T MERGE THIS CHANGE TO THE TRUNK.
> 
> Was this meant to be on the ast-branch?  If not, what's going on?

Yikes.  It was meant just for the ast-branch.  Don't know how my
release23-maint checkout got mixed up with ast-branch checkout.

Jeremy



From guido at python.org  Tue Dec  2 10:10:55 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  2 10:10:59 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Tue, 02 Dec 2003 08:27:46 EST."
	<200312020827.46280.mmclay@comcast.net> 
References: <200311240434.hAO4Y4L06979@c-24-5-183-134.client.comcast.net>
	<20031202010023.GA4439@panix.com>
	<200312020110.hB21APb32451@c-24-5-183-134.client.comcast.net> 
	<200312020827.46280.mmclay@comcast.net> 
Message-ID: <200312021510.hB2FAtX01085@c-24-5-183-134.client.comcast.net>

> Would there still be an int type and a long type in Python 3.0, or
> would the notion of a long be be dropped. If it were dropped, then
> the int representation would be transparently represented as a long
> if the size of the number could not fit in an int. If long is
> dropped then the long function could be added to the list of
> builtins that will disappear in Python 3.0.

I'm not sure about that yet.  I'd *like* to find a hack that lets the
int type change representations, but the fact is that it's much easier
to use different types to indicate different representations.

But you're right, even if there are two types, there's probably no
reason to expose the 'long' type as a builtin.  So long should go on
the list as *likely* to disappear in 3.0.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pf_moore at yahoo.co.uk  Tue Dec  2 11:02:21 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Dec  2 11:02:13 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<200311302157.49205.aleaxit@yahoo.com>
Message-ID: <8ylvmcmq.fsf@yahoo.co.uk>

Alex Martelli <aleaxit@yahoo.com> writes:

> Since these functions or types are going to be in operator, I think
> we can afford to "spend" two names to distinguish functionality
> (even though attgetter and itemgetter look nowhere as neat as
> extract -- I don't have better suggestions offhand).

operator.getattr and operator.getitem?

Paul.
-- 
This signature intentionally left blank


From pf_moore at yahoo.co.uk  Tue Dec  2 11:31:11 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Dec  2 11:31:01 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
Message-ID: <4qwjmbao.fsf@yahoo.co.uk>

[Phillip J. Eby]
> Yes.  Really the whole extract thing isn't that useful, except to get
> extra speed over using 'lambda x: x.foo' or whatever, which is what
> I'd probably use in any code that wasn't composing functions or
> compiling an OO query language.  :)

[Thomas Heller]
> Hm, couldn't "lambda ob: ob.foo.bar" return exactly the same thing
> as

> "extract(extract(attr='foo'), attr='bar')"

> ? In other words: return specialized C implemented functions for
> simple lambda expressions?

I agree with Thomas - rather than adding yet more specialised
functions, it would seem more sensible to optimize lambda - probably
via special cases like this.

Paul.
-- 
This signature intentionally left blank


From theller at python.net  Tue Dec  2 12:31:21 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec  2 12:31:34 2003
Subject: [Python-Dev] 2.3.3 cycle
In-Reply-To: <200312011422.hB1EMa9d023801@maxim.off.ekorp.com> (Anthony
	Baxter's message of "Tue, 02 Dec 2003 01:22:36 +1100")
References: <200312011422.hB1EMa9d023801@maxim.off.ekorp.com>
Message-ID: <ekvn6s9i.fsf@python.net>

Anthony Baxter <anthony@interlink.com.au> writes:

> Ok, my email mailbox is still on tape at the moment, so I'm going
> from memory here. My current plan (assuming this meets with everyone's
> approval) is to shoot for a 2.3.3rc1 either Friday this week or else
> on this weekend, and a final 2.3.3 release around the 17th. How's this
> fit in with other people's plans? In particular Thomas or Tim, whoever's
> going to need to do the Windows release.

That's fine with me.

Thomas


From walter at livinglogic.de  Tue Dec  2 13:42:02 2003
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Tue Dec  2 13:42:08 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
Message-ID: <3FCCDCFA.6070800@livinglogic.de>

Currently pprint refuses to prettyprint subclasses of list,
tuple and dict. Does it make sense to change that for 2.4?

Of course once __repr__() is overwritten by a subclass,
pprint has to use that instead of its own formatting.

Bye,
    Walter D?rwald



From fdrake at acm.org  Tue Dec  2 13:45:41 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Dec  2 13:45:52 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <3FCCDCFA.6070800@livinglogic.de>
References: <3FCCDCFA.6070800@livinglogic.de>
Message-ID: <16332.56789.356676.91289@grendel.fdrake.net>


Walter D?rwald writes:
 > Currently pprint refuses to prettyprint subclasses of list,
 > tuple and dict. Does it make sense to change that for 2.4?
 > 
 > Of course once __repr__() is overwritten by a subclass,
 > pprint has to use that instead of its own formatting.

There's a patch for this that's on my list of things to look at I
think.  The discussion associated with the patch had the right idea,
but I've not had time to review the patch (which I think is
embarrassingly short); if you'd like to do so, be my guest!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From guido at python.org  Tue Dec  2 13:50:58 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  2 13:51:05 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: Your message of "Tue, 02 Dec 2003 19:42:02 +0100."
	<3FCCDCFA.6070800@livinglogic.de> 
References: <3FCCDCFA.6070800@livinglogic.de> 
Message-ID: <200312021850.hB2IowQ01754@c-24-5-183-134.client.comcast.net>

> Currently pprint refuses to prettyprint subclasses of list,
> tuple and dict. Does it make sense to change that for 2.4?
> 
> Of course once __repr__() is overwritten by a subclass,
> pprint has to use that instead of its own formatting.

Sounds like a  good plan.  Do you have a suggested patch?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at python.net  Tue Dec  2 13:59:16 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec  2 13:59:28 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <4qwjmbao.fsf@yahoo.co.uk> (Paul Moore's message of "Tue, 02
	Dec 2003 16:31:11 +0000")
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk>
Message-ID: <r7zn59mj.fsf@python.net>

Paul Moore <pf_moore@yahoo.co.uk> writes:

> [Phillip J. Eby]
>> Yes.  Really the whole extract thing isn't that useful, except to get
>> extra speed over using 'lambda x: x.foo' or whatever, which is what
>> I'd probably use in any code that wasn't composing functions or
>> compiling an OO query language.  :)
>
> [Thomas Heller]
>> Hm, couldn't "lambda ob: ob.foo.bar" return exactly the same thing
>> as
>
>> "extract(extract(attr='foo'), attr='bar')"
>
>> ? In other words: return specialized C implemented functions for
>> simple lambda expressions?
>
> I agree with Thomas - rather than adding yet more specialised
> functions, it would seem more sensible to optimize lambda - probably
> via special cases like this.

One question that remains is: do a handful of these specialized
functions make it possible to replace the remaining uses lambda
completely?

Looking at parts of my codebase nearly all uses of lambda are
'lambda self: self.someattr'.

The remaining occurences have not yet been ported to the idioms of newer
Pythons.

Thomas



From hfastjava at yahoo.com  Tue Dec  2 14:06:20 2003
From: hfastjava at yahoo.com (Hunter Peress)
Date: Tue Dec  2 14:06:25 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses  
Message-ID: <20031202190620.58700.qmail@web41302.mail.yahoo.com>

I see the arguments,but the following code works fine. so what exactly is the issue?

#!/usr/bin/python
import sys,commands,os,re,string
from pprint import pprint as ppr

class t(list):
  pass

inst = t()
inst.append(1)
inst.append({2:3})
inst.append([4,5,6,[7,8]])

ppr(inst)

__________________________________
Do you Yahoo!?
Free Pop-Up Blocker - Get it now
http://companion.yahoo.com/

From fdrake at acm.org  Tue Dec  2 14:18:54 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Dec  2 14:19:05 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses  
In-Reply-To: <20031202190620.58700.qmail@web41302.mail.yahoo.com>
References: <20031202190620.58700.qmail@web41302.mail.yahoo.com>
Message-ID: <16332.58782.958432.60132@grendel.fdrake.net>


Hunter Peress writes:
 > I see the arguments,but the following code works fine. so what
 > exactly is the issue?

The code doesn't work when the repr of the subclass instance should be
wrapped:

 > #!/usr/bin/python
 > import sys,commands,os,re,string
 > from pprint import pprint as ppr
 > 
 > class t(list):
 >   pass
 > 
 > inst = t()
 > inst.append(1)
 > inst.append({2:3})
 > inst.append([4,5,6,[7,8]])

Change the last line to:

for i in range(10):
    inst.append(range(10))

and what you get is:

[1, {2: 3}, [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7,
8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
[0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1,
2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9], [0, 1, 2, 3,
4, 5, 6, 7, 8, 9], [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]

(wrapped by pasting into my mail client), but what's expected is:

[1,
 {2: 3},
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9],
 [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]]


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From guido at python.org  Tue Dec  2 14:18:56 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  2 14:19:08 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Tue, 02 Dec 2003 19:59:16 +0100."
	<r7zn59mj.fsf@python.net> 
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net> 
Message-ID: <200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>

> > I agree with Thomas - rather than adding yet more specialised
> > functions, it would seem more sensible to optimize lambda - probably
> > via special cases like this.
> 
> One question that remains is: do a handful of these specialized
> functions make it possible to replace the remaining uses lambda
> completely?
> 
> Looking at parts of my codebase nearly all uses of lambda are
> 'lambda self: self.someattr'.

Aha.  Very interesting ideas both!

In the past, we had a similar issue with exec/eval.  We looked at the
most frequent uses of these, and found that getting an attribute with
a computed name was the most common, so we added getattr (and setattr
and delattr).  Importing a module with a computed name was also quite
common, and now we have __import__.  So now exec/eval are typically
only used when we *really* want to run code provided by an end user.
(Exception: I often use eval() to parse literals when I know it is a
literal but it can have several types, e.g. string, int, float.  Maybe
there's a restricted form of eval that could be used for this too?)

So again, here we have a mechanism that's rather generic (lambda)
which is frequently used in a few stylized patterns (to extract an
attribute or field).  So Raymond's new functions attrgetter and
itemgetter (whose names I cannot seem to remember :-) take care of
these.

But, at least for attrgetter, I am slightly unhappy with the outcome,
because the attribute name is now expressed as a string literal rather
than using attribute notation.  This makes it harder to write
automated tools that check or optimize code.  (For itemgetter it
doesn't really matter, since the index is a literal either way.)

So, while I'm not particularly keen on lambda, I'm not that keen on
attrgetter either.  But what could be better?  All I can think of are
slightly shorter but even more crippled forms of lambda; for example,
we could invent a new keyword XXX so that the expression (XXX.foo) is
equivalent to (lambda self: self.foo).  This isn't very attractive.
Maybe the idea of recognizing some special forms of lambda and
implementing them more efficiently indeed makes more sense!

Hm, I see no end to this rambling, but I've got to go, so I'll just
stop now...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Tue Dec  2 14:23:35 2003
From: mwh at python.net (Michael Hudson)
Date: Tue Dec  2 14:23:38 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Tue, 02 Dec 2003 11:18:56 -0800")
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
Message-ID: <2msmk3hvm0.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

> So again, here we have a mechanism that's rather generic (lambda)
> which is frequently used in a few stylized patterns (to extract an
> attribute or field).  So Raymond's new functions attrgetter and
> itemgetter (whose names I cannot seem to remember :-) take care of
> these.
>
> But, at least for attrgetter, I am slightly unhappy with the outcome,
> because the attribute name is now expressed as a string literal rather
> than using attribute notation.  This makes it harder to write
> automated tools that check or optimize code.  (For itemgetter it
> doesn't really matter, since the index is a literal either way.)
>
> So, while I'm not particularly keen on lambda, I'm not that keen on
> attrgetter either.  But what could be better?  All I can think of are
> slightly shorter but even more crippled forms of lambda; for example,
> we could invent a new keyword XXX so that the expression (XXX.foo) is
> equivalent to (lambda self: self.foo).  This isn't very attractive.

Doesn't have to be a keyword... I implemented something like this
years ago and then ditched it when list comps appeared.

It would let you do things like

>>> map(X + 1, range(2))
[1, 2, 3]

too, IIRC.

Cheers,
mwh

-- 
  > With Python you can start a thread, but you can't stop it.  Sorry.
  > You'll have to wait until reaches the end of execution.
  So, just the same as c.l.py, then?
                       -- Cliff Wells & Steve Holden, comp.lang.python

From guido at python.org  Tue Dec  2 14:26:40 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  2 14:26:47 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Tue, 02 Dec 2003 19:23:35 GMT."
	<2msmk3hvm0.fsf@starship.python.net> 
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net> 
	<2msmk3hvm0.fsf@starship.python.net> 
Message-ID: <200312021926.hB2JQeP01889@c-24-5-183-134.client.comcast.net>

> > So, while I'm not particularly keen on lambda, I'm not that keen on
> > attrgetter either.  But what could be better?  All I can think of are
> > slightly shorter but even more crippled forms of lambda; for example,
> > we could invent a new keyword XXX so that the expression (XXX.foo) is
> > equivalent to (lambda self: self.foo).  This isn't very attractive.
> 
> Doesn't have to be a keyword... I implemented something like this
> years ago and then ditched it when list comps appeared.
> 
> It would let you do things like
> 
> >>> map(X + 1, range(2))
> [1, 2, 3]

What was your notation like?  Did you actually use 'X'?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at python.net  Tue Dec  2 14:34:40 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec  2 14:34:53 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <2msmk3hvm0.fsf@starship.python.net> (Michael Hudson's message
	of "Tue, 02 Dec 2003 19:23:35 +0000")
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
	<2msmk3hvm0.fsf@starship.python.net>
Message-ID: <ad6b57zj.fsf@python.net>

Michael Hudson <mwh@python.net> writes:

> Guido van Rossum <guido@python.org> writes:
>
>> So again, here we have a mechanism that's rather generic (lambda)
>> which is frequently used in a few stylized patterns (to extract an
>> attribute or field).  So Raymond's new functions attrgetter and
>> itemgetter (whose names I cannot seem to remember :-) take care of
>> these.
>>
>> But, at least for attrgetter, I am slightly unhappy with the outcome,
>> because the attribute name is now expressed as a string literal rather
>> than using attribute notation.  This makes it harder to write
>> automated tools that check or optimize code.  (For itemgetter it
>> doesn't really matter, since the index is a literal either way.)
>>
>> So, while I'm not particularly keen on lambda, I'm not that keen on
>> attrgetter either.  But what could be better?  All I can think of are
>> slightly shorter but even more crippled forms of lambda; for example,
>> we could invent a new keyword XXX so that the expression (XXX.foo) is
>> equivalent to (lambda self: self.foo).  This isn't very attractive.
>
> Doesn't have to be a keyword... I implemented something like this
> years ago and then ditched it when list comps appeared.
>
> It would let you do things like
>
>>>> map(X + 1, range(2))

Something like this?

class Adder:
    def __init__(self, number):
        self._number = number
    def __call__(self, arg):
        return arg + self._number

class X:
    def __add__(self, number):
        return Adder(number)

X = X()

print map(X + 1, range(2))

> [1, 2, 3]
>

(Although the above only prints [1, 2] ;-)


From walter at livinglogic.de  Tue Dec  2 14:39:15 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Dec  2 14:39:22 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <20031202190620.58700.qmail@web41302.mail.yahoo.com>
References: <20031202190620.58700.qmail@web41302.mail.yahoo.com>
Message-ID: <3FCCEA63.2070503@livinglogic.de>

Hunter Peress wrote:

> I see the arguments,but the following code works fine. so what exactly is the issue?
> 
> #!/usr/bin/python
> import sys,commands,os,re,string
> from pprint import pprint as ppr
> 
> class t(list):
>   pass
> 
> inst = t()
> inst.append(1)
> inst.append({2:3})
> inst.append([4,5,6,[7,8]])
> 
> ppr(inst)

This uses list.__repr__(), but you won't see any difference because the
string is too short.

Bye,
    Walter D?rwald



From guido at python.org  Tue Dec  2 15:02:08 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  2 15:02:15 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Tue, 02 Dec 2003 20:34:40 +0100."
	<ad6b57zj.fsf@python.net> 
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
	<2msmk3hvm0.fsf@starship.python.net> <ad6b57zj.fsf@python.net> 
Message-ID: <200312022002.hB2K28301985@c-24-5-183-134.client.comcast.net>

> > It would let you do things like
> >
> >>>> map(X + 1, range(2))
> 
> Something like this?
> 
> class Adder:
>     def __init__(self, number):
>         self._number = number
>     def __call__(self, arg):
>         return arg + self._number
> 
> class X:
>     def __add__(self, number):
>         return Adder(number)
> 
> X = X()
> 
> print map(X + 1, range(2))

Ah, of course.  Nice.  This can be extended to __getattr__ and
__getitem__; unfortunately __call__ would be ambiguous.  It could
probably be made quite fast with a C implementation.

Now the question remains, would it be better to hide this and simply
use it under the hood as an alternative way of generating code for
lambda, or should it be some sort of standard library module, to be
invoked explicitly?  In favor of the latter pleads that this would
solve the semantic differences with lambda when free variables are
involved: obviously X+q would evaluate q only once, while
(lamda X: X+q) evaluates q on each invocation.  Remember that for
generator expressions we've made the decision that
  (X+q for X in seq)
should evaluate q only once.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fumanchu at amor.org  Tue Dec  2 15:06:03 2003
From: fumanchu at amor.org (Robert Brewer)
Date: Tue Dec  2 15:11:12 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561AEA@opus.amorhq.net>

After:
> > > I agree with Thomas - rather than adding yet more specialised
> > > functions, it would seem more sensible to optimize lambda 
> - probably
> > > via special cases like this.
> > 
> > One question that remains is: do a handful of these specialized
> > functions make it possible to replace the remaining uses lambda
> > completely?
> > 
> > Looking at parts of my codebase nearly all uses of lambda are
> > 'lambda self: self.someattr'.
> 

Guido van Rossum wrote:
> Aha.  Very interesting ideas both!
> 
> So again, here we have a mechanism that's rather generic (lambda)
> which is frequently used in a few stylized patterns (to extract an
> attribute or field).  So Raymond's new functions attrgetter and
> itemgetter (whose names I cannot seem to remember :-) take care of
> these.
> 
> But, at least for attrgetter, I am slightly unhappy with the outcome,
> because the attribute name is now expressed as a string literal rather
> than using attribute notation.  This makes it harder to write
> automated tools that check or optimize code.  (For itemgetter it
> doesn't really matter, since the index is a literal either way.)
> 
> So, while I'm not particularly keen on lambda, I'm not that keen on
> attrgetter either.  But what could be better?  All I can think of are
> slightly shorter but even more crippled forms of lambda; for example,
> we could invent a new keyword XXX so that the expression (XXX.foo) is
> equivalent to (lambda self: self.foo).  This isn't very attractive.
> Maybe the idea of recognizing some special forms of lambda and
> implementing them more efficiently indeed makes more sense!

I swore I wouldn't post here without lurking for a couple months, but I
can't help myself.

I mentioned on c.l.p. that extract() looked like a classic case for
currying to me, using getattr() and currying the attribute name. So that
instead of (lambda self: self.foo), you'd end up with
(rightcurry(getattr, 'foo')), which might give some of the speed
increase desired. If one could assure that each object in an iteration
had that 'foo' attribute at the same mem location (for example,
instances of a class with a common function def), you could optimize
further.

> Now the question remains, would it be better to hide this and simply
> use it under the hood as an alternative way of generating code for
> lambda, or should it be some sort of standard library module, to be
> invoked explicitly?  In favor of the latter pleads that this would
> solve the semantic differences with lambda when free variables are
> involved: obviously X+q would evaluate q only once, while
> (lamda X: X+q) evaluates q on each invocation.  Remember that for
> generator expressions we've made the decision that
>   (X+q for X in seq)
> should evaluate q only once.

Which currying would do, no? Instead of (lambda X: X+q) you get
(rightcurry(operator.add, q)).

The reason I bring it up is because I see curry applying in many more
and more powerful situations than just extract(). Kill 16 birds with one
stone. Obviously, there's been some work on curry already in PEP 309,
which included the community response of "Lambda is good enough."
Perhaps it's not in this case.

Criticism welcomed, constructive or destructive. :)


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From skip at pobox.com  Tue Dec  2 15:13:34 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Dec  2 15:13:49 2003
Subject: [Python-Dev] minor interruption to service.
In-Reply-To: <200311290344.hAT3iHF5013034@maxim.off.ekorp.com>
References: <200311290344.hAT3iHF5013034@maxim.off.ekorp.com>
Message-ID: <16332.62062.305234.903783@montanaro.dyndns.org>


    Anthony> I'm going to be pretty much offline for a week or so - we got
    Anthony> burgled the other night while we were asleep and my laptop was
    Anthony> stolen. The data's backed up, but it'll be a few days til the
    Anthony> replacement laptop arrives.  

That's a pisser.  Sorry to hear about it.

    Anthony> In the meantime, if someone wants to take on the "upgrade to
    Anthony> autoconf 2.59" task, I'd appreciate it very much.

Do you mean 2.58?  I looked here:

    http://ftp.gnu.org/gnu/autoconf/

and 2.58 appears to be the latest release.

Skip

From ark-mlist at att.net  Tue Dec  2 15:14:01 2003
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue Dec  2 15:13:58 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312021510.hB2FAtX01085@c-24-5-183-134.client.comcast.net>
Message-ID: <024201c3b910$d3e855a0$6402a8c0@arkdesktop>

> I'm not sure about that yet.  I'd *like* to find a hack that lets the
> int type change representations, but the fact is that it's much easier
> to use different types to indicate different representations.

Aren't integers immutable?  If so, I would think it doesn't make sense for
them to change representation, as they don't change value.

Anyway, if you want to use type to encode representation, I would think that
the various integer types should be related by inheritance.  As a long can
always substitute for an int, at least in theory, I would think that long
should be derived from int.  Then isinstance(42L, int) would yield True.

If integers are related this way, LSP implies that converting a long to a
string should not put an L at the end.


From walter at livinglogic.de  Tue Dec  2 15:15:51 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Dec  2 15:15:55 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <16332.56789.356676.91289@grendel.fdrake.net>
References: <3FCCDCFA.6070800@livinglogic.de>
	<16332.56789.356676.91289@grendel.fdrake.net>
Message-ID: <3FCCF2F7.7090805@livinglogic.de>

Fred L. Drake, Jr. wrote:

> Walter D?rwald writes:
>  > Currently pprint refuses to prettyprint subclasses of list,
>  > tuple and dict. Does it make sense to change that for 2.4?
>  > 
>  > Of course once __repr__() is overwritten by a subclass,
>  > pprint has to use that instead of its own formatting.
> 
> There's a patch for this that's on my list of things to look at I
> think.  The discussion associated with the patch had the right idea,
> but I've not had time to review the patch (which I think is
> embarrassingly short);

AFAICT it should only change four lines.

> if you'd like to do so, be my guest!

Is this patch on SF? What's the patch id?

Bye,
    Walter D?rwald



From fdrake at acm.org  Tue Dec  2 15:18:23 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Tue Dec  2 15:18:43 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <3FCCF2F7.7090805@livinglogic.de>
References: <3FCCDCFA.6070800@livinglogic.de>
	<16332.56789.356676.91289@grendel.fdrake.net>
	<3FCCF2F7.7090805@livinglogic.de>
Message-ID: <16332.62351.960759.135367@grendel.fdrake.net>


Walter D?rwald writes:
 > Is this patch on SF? What's the patch id?

    http://www.python.org/sf/750542


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From guido at python.org  Tue Dec  2 15:28:12 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  2 15:28:27 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Tue, 02 Dec 2003 15:14:01 EST."
	<024201c3b910$d3e855a0$6402a8c0@arkdesktop> 
References: <024201c3b910$d3e855a0$6402a8c0@arkdesktop> 
Message-ID: <200312022028.hB2KSC902115@c-24-5-183-134.client.comcast.net>

> > I'm not sure about that yet.  I'd *like* to find a hack that lets the
> > int type change representations, but the fact is that it's much easier
> > to use different types to indicate different representations.
> 
> Aren't integers immutable?  If so, I would think it doesn't make sense for
> them to change representation, as they don't change value.

I was using shorthand-speak meaning that different instances of the
same class would use a different representation (which the class can
somehow recognize by looking at the instance, of course).

> Anyway, if you want to use type to encode representation, I would
> think that the various integer types should be related by
> inheritance.  As a long can always substitute for an int, at least
> in theory, I would think that long should be derived from int.  Then
> isinstance(42L, int) would yield True.

Or should int be a subclass of long?  I believe that OO theorists
consider the base class the set with the largest number of elements
(since it is the least constrained).  Now, all ints are longs, but all
longs are not ints, since ints can only represent values in
[-sys.maxint-1, sys.maxint].  According to this reasoning, long should
be the base class.

But the naming suggests different: 'int' suggests no particular size
(except to C/C++ users :-) so should be the more general class, and
that pleads for your version.

I don't particularly like either approach, because the 'long' type is
an implementation detail over which the user has no control (this is a
change of heart since Python's original design!).  I guess that means
I have to work harder and make the single int type support both
representations.  I'm sure it can be done.

> If integers are related this way, LSP implies that converting a long
> to a string should not put an L at the end.

Well, they aren't in Python 2.x, which is why the L stays until 3.0.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From walter at livinglogic.de  Tue Dec  2 15:40:05 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Dec  2 15:40:19 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <200312021850.hB2IowQ01754@c-24-5-183-134.client.comcast.net>
References: <3FCCDCFA.6070800@livinglogic.de>
	<200312021850.hB2IowQ01754@c-24-5-183-134.client.comcast.net>
Message-ID: <3FCCF8A5.1000309@livinglogic.de>

Guido van Rossum wrote:

 >>Currently pprint refuses to prettyprint subclasses of list,
 >>tuple and dict. Does it make sense to change that for 2.4?
 >>
 >>Of course once __repr__() is overwritten by a subclass,
 >>pprint has to use that instead of its own formatting.
 >
 >
 > Sounds like a  good plan.  Do you have a suggested patch?

Not yet, but Fred seems to have one.

Bye,
    Walter D?rwald



From walter at livinglogic.de  Tue Dec  2 15:40:47 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Dec  2 15:40:57 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <16332.62351.960759.135367@grendel.fdrake.net>
References: <3FCCDCFA.6070800@livinglogic.de>	<16332.56789.356676.91289@grendel.fdrake.net>	<3FCCF2F7.7090805@livinglogic.de>
	<16332.62351.960759.135367@grendel.fdrake.net>
Message-ID: <3FCCF8CF.8080009@livinglogic.de>

Fred L. Drake, Jr. wrote:

> Walter D?rwald writes:
>  > Is this patch on SF? What's the patch id?
> 
>     http://www.python.org/sf/750542

OK, I'll take a look at it.

Bye,
   Walter D?rwald



From dan at sidhe.org  Tue Dec  2 15:46:44 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Tue Dec  2 15:46:53 2003
Subject: [Python-Dev] Merry December
Message-ID: <a0601021cbbf2aa7dece5@[10.0.1.2]>

And a short reminder that it'd be a good idea to put together The 
Great Pie-thon Benchmark Program. I'd hate to think that the reason I 
got to splat Guido with a pie is because of a default... :)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From fincher.8 at osu.edu  Tue Dec  2 16:54:34 2003
From: fincher.8 at osu.edu (Jeremy Fincher)
Date: Tue Dec  2 15:56:40 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <8ylvmcmq.fsf@yahoo.co.uk>
References: <200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<200311302157.49205.aleaxit@yahoo.com> <8ylvmcmq.fsf@yahoo.co.uk>
Message-ID: <200312021654.34386.fincher.8@osu.edu>

On Tuesday 02 December 2003 11:02 am, Paul Moore wrote:
> Alex Martelli <aleaxit@yahoo.com> writes:
> > Since these functions or types are going to be in operator, I think
> > we can afford to "spend" two names to distinguish functionality
> > (even though attgetter and itemgetter look nowhere as neat as
> > extract -- I don't have better suggestions offhand).
>
> operator.getattr and operator.getitem?

I think the functions they're talking about are actually right-curried 
versions of those functions, like this:

def attrgetter(attr):
    return lambda x: getattr(x, attr)

Jeremy

From theller at python.net  Tue Dec  2 15:56:41 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec  2 15:56:56 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312022002.hB2K28301985@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Tue, 02 Dec 2003 12:02:08 -0800")
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
	<2msmk3hvm0.fsf@starship.python.net> <ad6b57zj.fsf@python.net>
	<200312022002.hB2K28301985@c-24-5-183-134.client.comcast.net>
Message-ID: <vfoz3pme.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

>> > It would let you do things like
>> >
>> >>>> map(X + 1, range(2))
>> 
>> Something like this?
>> 
>> class Adder:
>>     def __init__(self, number):
>>         self._number = number
>>     def __call__(self, arg):
>>         return arg + self._number
>> 
>> class X:
>>     def __add__(self, number):
>>         return Adder(number)
>> 
>> X = X()
>> 
>> print map(X + 1, range(2))
>
> Ah, of course.  Nice.  This can be extended to __getattr__ and
> __getitem__; unfortunately __call__ would be ambiguous.  It could
> probably be made quite fast with a C implementation.
>
> Now the question remains, would it be better to hide this and simply
> use it under the hood as an alternative way of generating code for
> lambda, or should it be some sort of standard library module, to be
> invoked explicitly?  In favor of the latter pleads that this would
> solve the semantic differences with lambda when free variables are
> involved: obviously X+q would evaluate q only once, while
> (lamda X: X+q) evaluates q on each invocation.  Remember that for
> generator expressions we've made the decision that
>   (X+q for X in seq)
> should evaluate q only once.

The latter has another advantage (or is this a disadvantage of the
former?): You can invoke

  lambda x: x.something

with a keyword arg, which would not be possible with a C implemented
function, I assume.  lambda expressions are often used to implement gui
callbacks, and they are sometimes invoked this way.

So the former would introduce incompatibilities.

Thomas


From ark-mlist at att.net  Tue Dec  2 16:25:41 2003
From: ark-mlist at att.net (Andrew Koenig)
Date: Tue Dec  2 16:25:42 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312022028.hB2KSC902115@c-24-5-183-134.client.comcast.net>
Message-ID: <001801c3b91a$d694cc70$6402a8c0@arkdesktop>

> > Aren't integers immutable?  If so, I would think it doesn't make sense
> for them to change representation, as they don't change value.

> I was using shorthand-speak meaning that different instances of the
> same class would use a different representation (which the class can
> somehow recognize by looking at the instance, of course).

Got it.

> > Anyway, if you want to use type to encode representation, I would
> > think that the various integer types should be related by
> > inheritance.  As a long can always substitute for an int, at least
> > in theory, I would think that long should be derived from int.  Then
> > isinstance(42L, int) would yield True.

> Or should int be a subclass of long?  I believe that OO theorists
> consider the base class the set with the largest number of elements
> (since it is the least constrained).  Now, all ints are longs, but all
> longs are not ints, since ints can only represent values in
> [-sys.maxint-1, sys.maxint].  According to this reasoning, long should
> be the base class.

I think that int should be the base class, because I can imagine long
supporting operations that int does not support, but not vice versa.



From walter at livinglogic.de  Tue Dec  2 18:03:36 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Dec  2 18:03:45 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <16332.62351.960759.135367@grendel.fdrake.net>
References: <3FCCDCFA.6070800@livinglogic.de>	<16332.56789.356676.91289@grendel.fdrake.net>	<3FCCF2F7.7090805@livinglogic.de>
	<16332.62351.960759.135367@grendel.fdrake.net>
Message-ID: <3FCD1A48.5000202@livinglogic.de>

Fred L. Drake, Jr. wrote:

> Walter D?rwald writes:
>  > Is this patch on SF? What's the patch id?
> 
>     http://www.python.org/sf/750542

OK, I've updated the patch.

Two other ideas for pprint:

The arguments for the PrettyPrinter constructor could
be exposed in pprint() and pformat():
"""
import pprint
print.pprint(foo, indent=3, width=40)
"""

The indent argument could be a string, which is used for indenting,
so indent=" " would be the same as indent=1, indent="   " would
be like indent=3, and we could use something like indent="\t"
or even indent="\033[1;30m...\033[0m".

Bye,
    Walter D?rwald



From greg at cosc.canterbury.ac.nz  Tue Dec  2 18:32:17 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec  2 18:32:25 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <001801c3b91a$d694cc70$6402a8c0@arkdesktop>
Message-ID: <200312022332.hB2NWHc05139@oma.cosc.canterbury.ac.nz>

Andrew Koenig <ark-mlist@att.net>

> I think that int should be the base class, because I can imagine long
> supporting operations that int does not support, but not vice versa.

Perhaps 'int' should be an abstract class, with concrete
subclasses 'long' and 'short'.

And 'bool' would fit into this family as a sort of
"extremely short int".

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Tue Dec  2 18:42:11 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec  2 18:42:20 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <vfoz3pme.fsf@python.net>
Message-ID: <200312022342.hB2NgBH05169@oma.cosc.canterbury.ac.nz>

Thomas Heller <theller@python.net>:

> The latter has another advantage (or is this a disadvantage of the
> former?): You can invoke
> 
>   lambda x: x.something
> 
> with a keyword arg, which would not be possible with a C implemented
> function, I assume.

If you're asking whether a C function can have a keyword argument
whose name is determined at runtime, I think that could be arranged --
it would just be a matter of concocting the appropriate arguments to
PyArg_ParseTupleAndKeywords dynamically.

As to a user-friendly syntax for invoking it, maybe something like

   arg.x.something

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Tue Dec  2 19:16:30 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec  2 19:16:36 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <DE1CF2B4FEC4A342BF62B6B2B334601E561AEA@opus.amorhq.net>
Message-ID: <200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>

Robert Brewer <fumanchu@amor.org>:

> So that instead of (lambda self: self.foo), you'd end up with
> (rightcurry(getattr, 'foo'))

At the expense of a large drop in readability.

We seem to be talking about two different things in this thread, speed
and readability. The nice thing about the "attrgetter.x" etc. idea is
that it has the potential to provide both at once.

The nasty thing about it is that it smells a bit too much like a
clever trick. It's really abusing the syntax to make it mean something
different from what it usually means.

I think I like the idea of optimising lambda, but that doesn't do
anything for the readability. So, how about a nicer syntax for lambda?
Maybe along the lines of

   x -> x.something

A bonus of introducing a new lambda syntax is that it would provide
the opportunity to give it early-binding semantics for free variables,
like generator expressions. The old lambda would have to be kept
around for a while for programs relying on the old semantics, but it
could be deprecated, and removed in 3.0.

Jeepers, *another* PEP to get around to writing one day...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From zack at codesourcery.com  Tue Dec  2 19:51:26 2003
From: zack at codesourcery.com (Zack Weinberg)
Date: Tue Dec  2 19:51:31 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz> (Greg
	Ewing's message of "Wed, 03 Dec 2003 13:16:30 +1300 (NZDT)")
References: <200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
Message-ID: <87y8turaep.fsf@egil.codesourcery.com>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> I think I like the idea of optimising lambda, but that doesn't do
> anything for the readability. So, how about a nicer syntax for lambda?
> Maybe along the lines of
>
>    x -> x.something
>
> A bonus of introducing a new lambda syntax is that it would provide
> the opportunity to give it early-binding semantics for free variables,
> like generator expressions. The old lambda would have to be kept
> around for a while for programs relying on the old semantics, but it
> could be deprecated, and removed in 3.0.

If you're going to mess with lambda at all, please try to come up with
a way for it not to be limited to a single expression.  I tried to do
this last summer and couldn't get rid of grammar ambiguities.

zw

From python at rcn.com  Wed Dec  3 01:56:20 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Dec  3 01:57:05 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Libary
In-Reply-To: <200311271113.05578.aleax@aleax.it>
Message-ID: <001701c3b96a$8f1cb100$e841fea9@oemcomputer>

Thank you everyone for the ideas on what to include and exclude from
the new tutorial section.

Attached is a revised draft.  Comments, suggestions, nitpicks,
complaints, and accolades are welcome.



Raymond Hettinger



-------------- next part --------------
Brief Tour of some Standard Library Modules



Operating System Interface

The os module provides many functions for interacting with the operating system:

>>> import os
>>> os.system('copy /data/mydata.fil /backup/mydata.fil')
0
>>> os.getcwd() 
'C:\\Python24'
>>> os.chdir('/server/accesslogs')

Be sure to use the "import os" style instead of "from os import *".  This will keep
os.open() from shadowing __builtin__.open()  which operates much differently.


File Wildcards

The glob module provides a function for making file lists from directory wildcard searches:

>>> glob.glob('*.py')
['primes.py', 'random.py', 'quote.py']


Command Line Arguments

Common utility scripts often invoke processing command line arguments.
These arguments are stored in the sys module's argv attribute as a list.
For instance the following output results from running
"python demo.py one two three" at the command line:

>>> import sys
>>> print sys.argv[]
['demo.py', 'one', 'two', 'three']

The getopt module processes sys.argv using the conventions of the Unix
getopt() function:

>>> import getopt
>>> # sys.argv is ['myprog.py', '-a', '-b', '-cfoo', '-d', 'bar', 'a1', 'a2']
>>> optlist, args = getopt.getopt(sys.argv[1:], 'abc:d:')
>>> optlist
[('-a', ''), ('-b', ''), ('-c', 'foo'), ('-d', 'bar')]
>>> args
['a1', 'a2']

More powerful and flexible command line processing is provided by
the optparse module.


Error Output Redirection and Program Termination

The sys module also has attributes for stdin, stdout, and stderr.  The latter
is useful for emitting warnings and error messages to make them visible
even when stdout has been redirected:

>>> sys.stderr.write('Warning, log file not found starting a new one')
Warning, log file not found starting a new one

The most direct way to terminate a script is to use sys.exit().


String Pattern Matching

The re module provides regular expression tools for advanced string processing.
When only simple capabilities are needed, string methods are preferred
because they are easier to read and debug.  For more sophisticated applications,
regular expressions can provide succinct, optimized solutions:

>>> import re
>>> re.findall(r'\bf[a-z]*', 'which foot or hand fell fastest')
['foot', 'fell', 'fastest']
>>> re.sub(r'(\b[a-z]+) \1', r'\1', 'cat in the the hat')
'cat in the hat'


Math Library

The math module gives access to the underlying C library functions for floating point math:

>>> import math
>>> math.cos(math.pi / 4.0)
0.70710678118654757
>>> math.log(1024, 2)
10.0

The random module provides tools for making random selections:

>>> import random
>>> random.choice(['apple', 'pear', 'banana'])
'apple'
>>> random.sample(xrange(100), 10)   # sampling without replacement
[30, 83, 16, 4, 8, 81, 41, 50, 18, 33]
>>> random.random()    # random float
0.17970987693706186
>>> random.randrange(6)    # random integer chosen from range(6)
4   


Internet Access

There are a number of modules for accessing the internet and processing
internet protocols. Two of the simplest are urllib2 for retrieving data
from urls and smtplib for sending mail:

>>> import urllib
>>> for line in urllib2.urlopen('http://tycho.usno.navy.mil/cgi-bin/timer.pl'):
... if 'EST' in line:      # look for Eastern Standard Time
...     print line
    
<BR>Nov. 25, 09:43:32 PM EST

>>> import smtplib
>>> server = smtplib.SMTP('localhost')
>>> server.sendmail('soothsayer@tmp.org', 'jceasar@tmp.org',
"""To: jceasar@tmp.org
From: soothsayer@tmp.org

Beware the Ides of March.
""")
>>> server.quit()


Dates and Times

The datetime module supplies classes for manipulating dates and times in both
simple and complex ways. While date and time arithmetic is supported, the focus
of the implementation is on efficient member extraction for output formatting
and manipulation.  The module also supports objects that are time zone aware.

# dates are easily constructed and formatted
>>> from datetime import date
>>> now = date.today()
>>> now
datetime.date(2003, 12, 2)
>>> now.strftime("%m-%d-%y or %d%b %Y is a %A on the %d day of %B")
'12-02-03 or 02Dec 2003 is a Tuesday on the 02 day of December'

# dates support calendar arithmetic
>>> birthday = date(1964, 7, 31)
>>> age = now - birthday
>>> age.days
14368


Data Compression

Common data archiving and compression formats are directly supported
by modules including : zlib, gzip, bz2, zipfile, and tar.

>>> import zlib
>>> s = 'witch which has which witches wrist watch'
>>> len(s)
41
>>> t = zlib.compress(s)
>>> len(t)
37
>>> zlib.decompress(t)
'witch which has which witches wrist watch'
>>> zlib.crc32(t)
-1438085031


Performance Measurement

Some Python users develop a deep interest in knowing the relative performance
between different approaches to the same problem.   Python provides a measurement
tool that answers those questions immediately.

For example, it may be tempting to use the tuple packing and unpacking feature
instead of the traditional approach to swapping arguments.  The timeit module
quickly demonstrates that the traditional approach is faster:

>>> from timeit import Timer
>>> dir(Timer)
>>> Timer('t=a; a=b; b=t', 'a=1; b=1').timeit()
0.60864915603680925
>>> Timer('a,b = b,a', 'a=1; b=1').timeit()
0.8625194857439773

In contrast to timeit's fine level of granularity, the profile and pstats modules
provide tools for identifying time critical sections of larger blocks of code.


Quality Control

One approach for developing high quality software is to write tests for each
function as it is developed and to run those tests frequently during the
development process.

The doctest module provides a tool for scanning a module and validating
tests embedded in a program's docstrings.  Test construction is as simple as
cutting-and-pasting a typical call along with its results into the docstring.
This improves the documentation by providing the user with an example
and it allows the doctest module to make sure the code remains true to the
documentation:

def average(values):
    """Computes the arithmetic mean of a list of numbers.

    >>> print average([20, 30, 70])
    40.0
    """
    return sum(values, 0.0) / len(values)

import doctest
doctest.testmod()   # automatically validate the embedded tests

The unittest module is not as effortless as the doctest module, but it allows
a more comprehensive set of tests to be maintained in a separate file:

import unittest

class TestStatisticalFunctions(unittest.TestCase):

    def test_average(self):
        self.assertEqual(average([20, 30, 70]), 40.0)
        self.assertEqual(round(average([1, 5, 7]), 1), 4.3)
        self.assertRaises(ZeroDivisionError, average, [])
        self.assertRaises(TypeError, average, 20, 30, 70)

unittest.main() # Calling from the command line invokes all tests


Batteries Included

Python has a "batteries included" philosophy.  The is best seen through the
sophisticated and robust capabilites of its larger packages:

* The xmlrpclib and SimpleXMLRPCServer modules make implementing remote
procedure calls into an almost trivial task.  Despite the names, no direct
knowledge or handling of XML is needed.

* The email package is a library for managing email messages, including MIME
and other RFC 2822-based message documents.  Unlike smtplib and poplib which
actually send and receive messages, the email package has a complete toolset
for building or decoding complex message structures (including attachments)
and for implementing internet encoding and header protocols.

* The xml.dom and xml.sax packages provide robust support for parsing this
popular data interchange format.  Likewise, the csv module supports direct
reads and writes in a common database format.  Together, these modules and
packages greatly simplify data interchange between python applications and
other tools.

* Internationalization is supported by a number of modules including gettext
and the codecs package.





From raymond.hettinger at verizon.net  Wed Dec  3 04:21:32 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Wed Dec  3 04:22:45 2003
Subject: [Python-Dev] unittest output
Message-ID: <000a01c3b97e$e8f69d80$e841fea9@oemcomputer>

When a test fails, it is accompanied by tracebacks that include
all the processing within the unittest module itself.  This makes
it darned difficult to see at a glance what went wrong.  If like
me, you use unittest as part of the development process, then you
run it hundreds of times each week and this eyesore gets to be a
real PITA.

I don't recall when or where, but someone has proposed a fix.
But, like many great ideas, it got stymied because someone,
somewhere has a use for the rest of the traceback.   Also, a
developer raised the (IMO red herring) issue -- what if there 
is a bug in the unittest module, how would you track it.

So, I would like to re-raise the issue.
Can we get this fixed?

One other nit:  When unittest.main() is run from the new IDLE
it fails on a sys.exit() attempt.   Can we trap the exit,
and just return?



Raymond Hettinger


From theller at python.net  Wed Dec  3 04:34:41 2003
From: theller at python.net (Thomas Heller)
Date: Wed Dec  3 04:34:53 2003
Subject: [Python-Dev] unittest output
In-Reply-To: <000a01c3b97e$e8f69d80$e841fea9@oemcomputer> (Raymond
	Hettinger's message of "Wed, 3 Dec 2003 04:21:32 -0500")
References: <000a01c3b97e$e8f69d80$e841fea9@oemcomputer>
Message-ID: <d6b6453i.fsf@python.net>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> When a test fails, it is accompanied by tracebacks that include
> all the processing within the unittest module itself.  This makes
> it darned difficult to see at a glance what went wrong.  If like
> me, you use unittest as part of the development process, then you
> run it hundreds of times each week and this eyesore gets to be a
> real PITA.
>
> I don't recall when or where, but someone has proposed a fix.
> But, like many great ideas, it got stymied because someone,
> somewhere has a use for the rest of the traceback.   Also, a
> developer raised the (IMO red herring) issue -- what if there 
> is a bug in the unittest module, how would you track it.

http://www.python.org/sf/722638

>
> So, I would like to re-raise the issue.
> Can we get this fixed?

Yes, please.

Thomas


From theller at python.net  Wed Dec  3 04:36:59 2003
From: theller at python.net (Thomas Heller)
Date: Wed Dec  3 04:37:09 2003
Subject: [Python-Dev] unittest output
In-Reply-To: <000a01c3b97e$e8f69d80$e841fea9@oemcomputer> (Raymond
	Hettinger's message of "Wed, 3 Dec 2003 04:21:32 -0500")
References: <000a01c3b97e$e8f69d80$e841fea9@oemcomputer>
Message-ID: <8ylu44zo.fsf@python.net>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> When a test fails, it is accompanied by tracebacks that include
> all the processing within the unittest module itself.  This makes
> it darned difficult to see at a glance what went wrong.  If like
> me, you use unittest as part of the development process, then you
> run it hundreds of times each week and this eyesore gets to be a
> real PITA.
>
> I don't recall when or where, but someone has proposed a fix.
> But, like many great ideas, it got stymied because someone,
> somewhere has a use for the rest of the traceback.   Also, a
> developer raised the (IMO red herring) issue -- what if there 
> is a bug in the unittest module, how would you track it.

Well, distutils being mainly a user tool tries it's best to hide
tracebacks from the user, and only prints a single 'error: ...'
line.  But it pays attention to a DISTUTILS_DEBUG env var, which will
enable showing the full tracebacks.  IIRC.

Thomas


From python at rcn.com  Wed Dec  3 04:56:23 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Dec  3 04:57:05 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
Message-ID: <000e01c3b983$b5ca6360$e841fea9@oemcomputer>

[Thomas Heller]
> > Looking at parts of my codebase nearly all uses of lambda are
> > 'lambda self: self.someattr'.

Yes, they are everywhere.
Getting rid of those lambdas was part of the attraction for
attrgetter().


> But, at least for attrgetter, I am slightly unhappy with the outcome,
> because the attribute name is now expressed as a string literal rather
> than using attribute notation.  This makes it harder to write
> automated tools that check or optimize code.  (For itemgetter it
> doesn't really matter, since the index is a literal either way.)
> 
> So, while I'm not particularly keen on lambda, I'm not that keen on
> attrgetter either.  But what could be better?  

I don't know if you like this, but there is a way to change the
interface to attrgetter() so that the dot notation can be used 
instead of a string.  It produces the same result and is neater,
but I find it somewhat harder to explain:



import operator

class NewAttrGetter(object):
    def __getattribute__(self, name):
        return operator.attrgetter(name)

newattrgetter = NewAttrGetter()

class A:
    pass
a = A()
a.score = 10

getscore = operator.attrgetter('score')
print getscore(a)

getscore = newattrgetter.score
print getscore(a)



new-style-classes-rock-ly yours,



Raymond Hettinger


From python at rcn.com  Wed Dec  3 05:02:03 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Dec  3 05:02:46 2003
Subject: [Python-Dev] unittest output
In-Reply-To: <d6b6453i.fsf@python.net>
Message-ID: <000f01c3b984$80bb8360$e841fea9@oemcomputer>

> > When a test fails, it is accompanied by tracebacks that include
> > all the processing within the unittest module itself.  This makes
> > it darned difficult to see at a glance what went wrong.  If like
> > me, you use unittest as part of the development process, then you
> > run it hundreds of times each week and this eyesore gets to be a
> > real PITA.
> >
> > I don't recall when or where, but someone has proposed a fix.
> > But, like many great ideas, it got stymied because someone,
> > somewhere has a use for the rest of the traceback.   Also, a
> > developer raised the (IMO red herring) issue -- what if there
> > is a bug in the unittest module, how would you track it.
> 
> http://www.python.org/sf/722638

Arghh.  I had forgotten that Steve Purcell weighed in on this.
It's his module.  So unless he can be persuaded, the point is moot :-(


> > So, I would like to re-raise the issue.
> > Can we get this fixed?
> 
> Yes, please.



Raymond Hettinger


From python at rcn.com  Wed Dec  3 05:30:59 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Dec  3 05:31:41 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <000e01c3b983$b5ca6360$e841fea9@oemcomputer>
Message-ID: <001001c3b988$8b1054e0$e841fea9@oemcomputer>

 [Guido]
> > But, at least for attrgetter, I am slightly unhappy with the
outcome,
> > because the attribute name is now expressed as a string literal
rather
> > than using attribute notation.  This makes it harder to write
> > automated tools that check or optimize code.  (For itemgetter it
> > doesn't really matter, since the index is a literal either way.)
> >
> > So, while I'm not particularly keen on lambda, I'm not that keen on
> > attrgetter either.  But what could be better?
 
[Me, with mildly wacky idea]
> I don't know if you like this, but there is a way to change the
> interface to attrgetter() so that the dot notation can be used
> instead of a string.  It produces the same result and is neater,
> but I find it somewhat harder to explains

Carried to the limit, the idea turns into something that is
either sublime or severely bonkers.  The good points are that
Guido gets his dotted access and I get to trade in the two ugly
names and for a single beautiful "extract".  And there's no performance
cost, the inner loop is the same.  The downside is I still don't
know how to explain it (AFAICT, super() is the closest thing to it):



import operator

class ExtractorClass(object):
    def __getattribute__(self, attr):
        return operator.attrgetter(attr)
    def __getitem__(self, key):
        return operator.itemgetter(key)

extract = ExtractorClass()

class A:
    pass
a = A()
a.score = 10

getscore = extract.score      # Houston, we have dotted access
print getscore(a)


b = [10, 20, 30, 40]

getsecond = extract[1]        # and, we have the bracketed lookups 
print getsecond(b)


animal_weights = [('cat', 10), ('dog', 90), ('human', 150), ('goldfish',
0.1), ('unladen_sparrow', 3)]
list.sorted(animal_weights, key=extract[1])


list.sorted(students, key=extract.score)  


So, now we have a weird bird that is faster and better looking than
lambda.


Raymond


From mwh at python.net  Wed Dec  3 05:39:05 2003
From: mwh at python.net (Michael Hudson)
Date: Wed Dec  3 05:39:08 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312021926.hB2JQeP01889@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Tue, 02 Dec 2003 11:26:40 -0800")
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
	<2msmk3hvm0.fsf@starship.python.net>
	<200312021926.hB2JQeP01889@c-24-5-183-134.client.comcast.net>
Message-ID: <2m7k1eyyly.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> Doesn't have to be a keyword... I implemented something like this
>> years ago and then ditched it when list comps appeared.
>> 
>> It would let you do things like
>> 
>> >>> map(X + 1, range(2))
>> [1, 2, 3]
>
> What was your notation like?  Did you actually use 'X'?

Um, I think so.  I defined it in my $PYTHONSTARTUP file as an instance
of a class _X, or something like that.

Cheers,
mwh

-- 
  In many ways, it's a dull language, borrowing solid old concepts
  from many other languages & styles:  boring syntax, unsurprising
  semantics, few  automatic coercions, etc etc.  But that's one of
  the things I like about it.                 -- Tim Peters, 16 Sep 93

From mwh at python.net  Wed Dec  3 05:42:44 2003
From: mwh at python.net (Michael Hudson)
Date: Wed Dec  3 05:42:46 2003
Subject: [Python-Dev] minor interruption to service.
In-Reply-To: <16332.62062.305234.903783@montanaro.dyndns.org> (Skip
	Montanaro's message of "Tue, 2 Dec 2003 14:13:34 -0600")
References: <200311290344.hAT3iHF5013034@maxim.off.ekorp.com>
	<16332.62062.305234.903783@montanaro.dyndns.org>
Message-ID: <2m3cc2yyfv.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Anthony> In the meantime, if someone wants to take on the "upgrade to
>     Anthony> autoconf 2.59" task, I'd appreciate it very much.
>
> Do you mean 2.58?

No: http://article.gmane.org/gmane.comp.sysutils.autotools.announce/27

> I looked here:
>
>     http://ftp.gnu.org/gnu/autoconf/
>
> and 2.58 appears to be the latest release.

Don't know whats going on there.

Cheers,
mwh

-- 
  > say-hi-to-the-flying-pink-elephants-for-me-ly y'rs,
  No way, the flying pink elephants are carrying MACHINE GUNS!
  Aiiee!!  Time for a kinder, gentler hallucinogen...
                               -- Barry Warsaw & Greg Ward, python-dev

From mwh at python.net  Wed Dec  3 05:52:38 2003
From: mwh at python.net (Michael Hudson)
Date: Wed Dec  3 05:52:41 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312022002.hB2K28301985@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Tue, 02 Dec 2003 12:02:08 -0800")
References: <Your message of "Sun, 30 Nov 2003 18:48:25 EST."
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<20031130073109.GA1560@hishome.net>
	<200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<002701c3b606$c61304a0$e841fea9@oemcomputer>
	<20031130073109.GA1560@hishome.net>
	<5.1.0.14.0.20031130184217.02e3c1d0@mail.telecommunity.com>
	<200312010018.hB10IbS29532@c-24-5-183-134.client.comcast.ne t>
	<5.1.0.14.0.20031130210357.02f62d80@mail.telecommunity.com>
	<4qwjmbao.fsf@yahoo.co.uk> <r7zn59mj.fsf@python.net>
	<200312021918.hB2JIuQ01834@c-24-5-183-134.client.comcast.net>
	<2msmk3hvm0.fsf@starship.python.net> <ad6b57zj.fsf@python.net>
	<200312022002.hB2K28301985@c-24-5-183-134.client.comcast.net>
Message-ID: <2my8tuxjex.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> > It would let you do things like
>> >
>> >>>> map(X + 1, range(2))
>> 
>> Something like this?
>> 
>> class Adder:
>>     def __init__(self, number):
>>         self._number = number
>>     def __call__(self, arg):
>>         return arg + self._number
>> 
>> class X:
>>     def __add__(self, number):
>>         return Adder(number)
>> 
>> X = X()
>> 
>> print map(X + 1, range(2))
>
> Ah, of course.  Nice.  This can be extended to __getattr__ and
> __getitem__; unfortunately __call__ would be ambiguous.

Yes.  I think I used .c() for that.  IIRC correctly, I also
complicated things to the extent that there was a special object, say
N, and you could write

>>> map(X.split(N), ['a a', 'b b\nb'], [None, '\n'])
[['a', 'a'], ['b b', 'b']]

This might be going a bit far...

> It could probably be made quite fast with a C implementation.

Would be tedious to write though :-)

> Now the question remains, would it be better to hide this and simply
> use it under the hood as an alternative way of generating code for
> lambda, or should it be some sort of standard library module, to be
> invoked explicitly?  In favor of the latter pleads that this would
> solve the semantic differences with lambda when free variables are
> involved: obviously X+q would evaluate q only once, while
> (lamda X: X+q) evaluates q on each invocation.  Remember that for
> generator expressions we've made the decision that
>   (X+q for X in seq)
> should evaluate q only once.

I am not a fan of the idea of compiling certain kinds of lambdas
differently.

Cheers,
mwh

-- 
  While preceding your entrance with a grenade is a good tactic in
  Quake, it can lead to problems if attempted at work.    -- C Hacking
               -- http://home.xnet.com/~raven/Sysadmin/ASR.Quotes.html

From mwh at python.net  Wed Dec  3 05:54:59 2003
From: mwh at python.net (Michael Hudson)
Date: Wed Dec  3 05:55:02 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <3FCD1A48.5000202@livinglogic.de> (
	=?iso-8859-1?q?Walter_D=F6rwald's_message_of?= "Wed,
	03 Dec 2003 00:03:36 +0100")
References: <3FCCDCFA.6070800@livinglogic.de>
	<16332.56789.356676.91289@grendel.fdrake.net>
	<3FCCF2F7.7090805@livinglogic.de>
	<16332.62351.960759.135367@grendel.fdrake.net>
	<3FCD1A48.5000202@livinglogic.de>
Message-ID: <2mu14ixjb0.fsf@starship.python.net>

Walter D?rwald <walter@livinglogic.de> writes:

> The arguments for the PrettyPrinter constructor could
> be exposed in pprint() and pformat():
> """
> import pprint
> print.pprint(foo, indent=3, width=40)
> """

+1e6!  I've wanted this for so long I forgot that I could do something
+about it :-)

Cheers,
mwh

-- 
  at any rate, I'm satisfied that not only do they know which end of
  the pointy thing to hold, but where to poke it for maximum effect.
                                  -- Eric The Read, asr, on google.com

From mwh at python.net  Wed Dec  3 05:56:35 2003
From: mwh at python.net (Michael Hudson)
Date: Wed Dec  3 05:56:38 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312022332.hB2NWHc05139@oma.cosc.canterbury.ac.nz> (Greg
	Ewing's message of "Wed, 03 Dec 2003 12:32:17 +1300 (NZDT)")
References: <200312022332.hB2NWHc05139@oma.cosc.canterbury.ac.nz>
Message-ID: <2mptf6xj8c.fsf@starship.python.net>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> Andrew Koenig <ark-mlist@att.net>
>
>> I think that int should be the base class, because I can imagine long
>> supporting operations that int does not support, but not vice versa.
>
> Perhaps 'int' should be an abstract class, with concrete
> subclasses 'long' and 'short'.

That's quite a nice idea -- a bit like an Objective C class cluster.

Cheers,
mwh

-- 
  ZAPHOD:  You know what I'm thinking?
    FORD:  No.
  ZAPHOD:  Neither do I.  Frightening isn't it?
                   -- The Hitch-Hikers Guide to the Galaxy, Episode 11

From ncoghlan at iinet.net.au  Wed Dec  3 06:18:32 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Dec  3 06:18:38 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312022028.hB2KSC902115@c-24-5-183-134.client.comcast.net>
References: <024201c3b910$d3e855a0$6402a8c0@arkdesktop>
	<200312022028.hB2KSC902115@c-24-5-183-134.client.comcast.net>
Message-ID: <3FCDC688.4080001@iinet.net.au>

Guido van Rossum wrote:
> I don't particularly like either approach, because the 'long' type is
> an implementation detail over which the user has no control (this is a
> change of heart since Python's original design!).  I guess that means
> I have to work harder and make the single int type support both
> representations.  I'm sure it can be done.

Isn't the 'long' format the more general representation, with the 'int' 
format then being a performance hack to take advantage of the speed of C 
integer arithmetic?

I'm just wondering if a different way of thinking about it might help 
with figuring out how to handle a combined implementation.

Cheers,
Nick.



From ncoghlan at iinet.net.au  Wed Dec  3 06:48:49 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Dec  3 06:48:59 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <001001c3b988$8b1054e0$e841fea9@oemcomputer>
References: <001001c3b988$8b1054e0$e841fea9@oemcomputer>
Message-ID: <3FCDCDA1.2040402@iinet.net.au>

Raymond Hettinger wrote:
> Carried to the limit, the idea turns into something that is
> either sublime or severely bonkers.  The good points are that
> Guido gets his dotted access and I get to trade in the two ugly
> names and for a single beautiful "extract".  And there's no performance
> cost, the inner loop is the same.  The downside is I still don't
> know how to explain it (AFAICT, super() is the closest thing to it):

What you end up with is still a unary operator, so it could still live 
in the operator module, to.

And I think what you posted would work for strings as dictionary keys, 
too - answering another of the objections to the original operator.extract

Which leaves figuring out a concise explanation for what the hell it 
does (without using lambda in the examples, since I assume part of the 
aim here is to avoid explaining lambda to people who don't need it). . .

"operator.extract provides an interim target for an attribute or item 
access where the real target of the access is to be determined later. 
The access is made normally (i.e. dotted notation or indexing), with 
'operator.extract' substituted where the target would normally be 
written. The actual access is carried out by calling the result returned 
by the expression operator.extract is part of with the real target as 
the first argument.

E.g.
   y = operator.extract.foo
is equivalent to
   def y(x):
     return x.foo

   y = operator.extract[2]
is equivalent to
   def y(x):
     return x[2]
"

If you meant a concise explanation of _how_ it does what it does, then 
I'm not sure that can be done :)

And I still don't know if I should be admiring this or running away 
screaming!

Cheers,
Nick.



From ajsiegel at optonline.net  Tue Dec  2 23:21:42 2003
From: ajsiegel at optonline.net (Arthur)
Date: Wed Dec  3 07:01:49 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Libary
Message-ID: <0HPB006FGILSWO@mta2.srv.hcvlny.cv.net>

No mention of the copy module.  Is this open for discussion?

Raymond had written:

>My first draft included:
>    copy, glob, shelve, pickle, os, re, math/cmath, urllib, smtplib

>Guido's thoughts:
>- copy tends to be overused by beginners

While Guido offers no evidence, I have no evidence to dispute him.  Let's
accept it as true.

[Snip]

>While I think of copy() and deepcopy() as builtins that got tucked away 
>in module, Guido is right about their rarity in well-crafted code.

Accepting Guido is right and you are right in acknowledging that, why is it
that you, nonetheless, think of copy()/deepcopy() as displaced built-ins. 

I would argue because you acknowledge they are conceptually core - though
(we are accepting) not functionally core.  

OTOH, functionally, haven't some built-ins been growing copy methods? Could
be wrong about that - seem to get that idea somewhere.

Is it unreasonable to argue that hiding - isn't that is what is being
advocated, in essence - something that is arguably conceptually important,
cannot properly be considered to be a service to beginners?

Is it totally out-of-left field to argue that confronting "copy" forces a
confrontation with the fundamentals of assignment - perhaps the most subtle
concept faced in approaching Python.  Particularly difficult because the
beginner does not know to expect subtleties.  I would argue that these
subtleties are *always* surprising - particularly and importantly because of
an important missing clue.  That clue, specifically, the existence of copy. 

I had always assumed that the absence of copy()/deepcopy() from built-ins
was for 'big boy' reasons.  And accepted it as that - feeling, in general,
that the big boy issues should override the 'good for beginners' issue.  And
only take up the cause of copy() to the extent that - on better evidence -
it appears to be being handled as it is for paternalistic reasons. I am
closer to being a beginner than most.  And nonetheless have in all debates
concluded with some confidence that what makes sense for Python, independent
of any focus on the needs of beginners, is in fact what makes sense for
beginners. 

I believe that - at core - that is a position respectful of Python and its
BDFL, though it seems to put me consistently in conflict with it and him.

Art   


From tzot at sil-tec.gr  Wed Dec  3 07:17:44 2003
From: tzot at sil-tec.gr (Christos Georgiou)
Date: Wed Dec  3 07:18:07 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <ptjrsvscj5g911umb2hksog02odeq0fsb4@4ax.com>

I don't know if this will be useful to anyone, but what the heck...

This post does not propose a complete substitute for lambda, but covers
most of its use cases using current python capabilities and syntax.
It's a very similar idea to what Michael Hudson already has mentioned.
And it doesn't apply to lambdas with more than one argument.

(There is also a module of mine, at
http://www.sil-tec.gr/~tzot/predicates.py that implements this idea in a
somehow harsh way to implement all and any predicates on iterables
--check at the end of the module for example usage.)

The whole idea is a C-implemented (after all, extract et al are about
speed) class, say Evaluator or Eval for short, which has an
implementation for all __*__ special methods; calling (directly or
indirectly) any special method of an instance of Eval returns a callable
that does exactly that operation to its argument.  Precedence etc are
taken care of by python.

So, this way,
>>> myf= lambda x: x**2 + 1
is equivalent to
>>> myf= Eval()**2 + 1

attrgetter("attribute_name") becomes Eval().attribute_name

and indexgetter(index) becomes Eval()[index]

However, I have no obvious way to replace stuff as

lambda x: x[:2] + x[-2:]

or

lambda x: divmod(x, 6)

although in my mentioned module above I have a special .do() method for
the latter case, but I presume most of you would find that ugly.

So, this idea covers the attrgetter and indexgetter use cases and then
some more, but it is not a complete lambda replacement.  Just my two
eurocents :-|
-- 
Christos Georgiou, Customer Support Engineer
Silicon Technologies S.A.
41-45 Kifissias Ave, Maroussi 151 23 Greece
Tel +30 (21) 06152600 (ext. 605) Fax +30 (21) 06198140
"Dave always exaggerated." --HAL

From mwh at python.net  Wed Dec  3 07:40:05 2003
From: mwh at python.net (Michael Hudson)
Date: Wed Dec  3 07:40:32 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Libary
In-Reply-To: <0HPB006FGILSWO@mta2.srv.hcvlny.cv.net> (Arthur's message of
	"Tue, 02 Dec 2003 23:21:42 -0500")
References: <0HPB006FGILSWO@mta2.srv.hcvlny.cv.net>
Message-ID: <2mllpuxefu.fsf@starship.python.net>

I don't want to be rude -- is English your first language? -- but I
found this article to be written in particularly baroque prose.  I'm
not quite sure what your central point is.

Arthur <ajsiegel@optonline.net> writes:

> No mention of the copy module.  Is this open for discussion?

Well, I dunno.  If it was my decision, there's no way it would be
mentioned in the builtins.

> OTOH, functionally, haven't some built-ins been growing copy methods? Could
> be wrong about that - seem to get that idea somewhere.

No changes here in a long while.

> Is it unreasonable to argue that hiding - isn't that is what is being
> advocated, in essence - something that is arguably conceptually important,
> cannot properly be considered to be a service to beginners?

I don't think it's unreasonable at all.  To me the copy module is
something you only need when you really need it, then you go looking
for it.

> Is it totally out-of-left field to argue that confronting "copy" forces a
> confrontation with the fundamentals of assignment - perhaps the most subtle
> concept faced in approaching Python.

Certainly, the failure to understand assignment and erroneous beliefs
that the copy module is necessary in a given situation go hand in hand.

Assignment in Python *isn't* particularly subtle, though.  My
impression is that it's people from e.g. C who get tripped up here,
not total newbies.

> Particularly difficult because the beginner does not know to expect
> subtleties.  I would argue that these subtleties are *always*
> surprising - particularly and importantly because of an important
> missing clue.  That clue, specifically, the existence of copy.
>
> I had always assumed that the absence of copy()/deepcopy() from built-ins
> was for 'big boy' reasons.

Another reason might just be that copy() is implemented in Python...

> And accepted it as that - feeling, in general, that the big boy
> issues should override the 'good for beginners' issue.  And only
> take up the cause of copy() to the extent that - on better evidence
> - it appears to be being handled as it is for paternalistic
> reasons. I am closer to being a beginner than most.

Have you ever used the copy module?  I am *not* a beginner, and have
used it *once* (and I can't remember what for, either).

Cheers,
mwh

-- 
  if-you-need-your-own-xxx.py-you-know-where-to-shove-it<wink>-ly 
  y'rs - tim
              -- Tim Peters dishes out versioning advice on python-dev

From guido at python.org  Wed Dec  3 08:50:55 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 08:51:05 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Libary
In-Reply-To: Your message of "Wed, 03 Dec 2003 01:56:20 EST."
	<001701c3b96a$8f1cb100$e841fea9@oemcomputer> 
References: <001701c3b96a$8f1cb100$e841fea9@oemcomputer> 
Message-ID: <200312031350.hB3DotY03405@c-24-5-183-134.client.comcast.net>

> Thank you everyone for the ideas on what to include and exclude from
> the new tutorial section.
> 
> Attached is a revised draft.  Comments, suggestions, nitpicks,
> complaints, and accolades are welcome.

This version looks great!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec  3 09:09:58 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 09:10:12 2003
Subject: [Python-Dev] unittest output
In-Reply-To: Your message of "Wed, 03 Dec 2003 05:02:03 EST."
	<000f01c3b984$80bb8360$e841fea9@oemcomputer> 
References: <000f01c3b984$80bb8360$e841fea9@oemcomputer> 
Message-ID: <200312031409.hB3E9wt03471@c-24-5-183-134.client.comcast.net>

> > > When a test fails, it is accompanied by tracebacks that include
> > > all the processing within the unittest module itself.  This makes
> > > it darned difficult to see at a glance what went wrong.  If like
> > > me, you use unittest as part of the development process, then you
> > > run it hundreds of times each week and this eyesore gets to be a
> > > real PITA.
> > >
> > > I don't recall when or where, but someone has proposed a fix.
> > > But, like many great ideas, it got stymied because someone,
> > > somewhere has a use for the rest of the traceback.   Also, a
> > > developer raised the (IMO red herring) issue -- what if there
> > > is a bug in the unittest module, how would you track it.
> > 
> > http://www.python.org/sf/722638
> 
> Arghh.  I had forgotten that Steve Purcell weighed in on this.
> It's his module.  So unless he can be persuaded, the point is moot :-(

Not so fast.

The patch claims to only suppress the traceback if the test *failed*
(e.g. called self.fail(), or one of the assertXXX variations failed).
If the test raised an unexpected exception (unittest calls this an
*error*) the traceback is printed normally.  This seems right to me.

A refinement could be to make the output *look* like a (short)
traceback as printed by the traceback module even in the failure case;
this would address Steve's issue tht IDEs find lines in the code via
the traceback.  And there should be a command line switch and an
environment variable to show the full traceback in all cases.

I'd also plead for a switch to let unittest *not* catch exceptions at
all -- there are situations where you really want to invoke a debugger.

The IDLE issue is separate; I agree that IDLE should treat SystemExit
differently (simply go back to the >>> prompt without printing a
traceback, printing the status only if it is true).  You should add
this to the idlefork SF tracker so Kurt can address it.

--Guido van Rossum (home page: http://www.python.org/~guido/)


From guido at python.org  Wed Dec  3 09:16:14 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 09:16:25 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Wed, 03 Dec 2003 10:56:35 GMT."
	<2mptf6xj8c.fsf@starship.python.net> 
References: <200312022332.hB2NWHc05139@oma.cosc.canterbury.ac.nz>  
	<2mptf6xj8c.fsf@starship.python.net> 
Message-ID: <200312031416.hB3EGEn03502@c-24-5-183-134.client.comcast.net>

> > Andrew Koenig <ark-mlist@att.net>
> >> I think that int should be the base class, because I can imagine long
> >> supporting operations that int does not support, but not vice versa.

> Greg Ewing <greg@cosc.canterbury.ac.nz> writes:
> > Perhaps 'int' should be an abstract class, with concrete
> > subclasses 'long' and 'short'.

MWH:
> That's quite a nice idea -- a bit like an Objective C class cluster.

I've been pondering this one too.  The only downside I can think of is
that code that inherits from class int to add some method (e.g. a
class hexint whose repr() and str() call hex() instead) will no longer
inherit any implementation, and thus won't be very useful.  Inheriting
from short or long doesn't quite solve the problem either.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec  3 09:17:21 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 09:17:32 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Wed, 03 Dec 2003 21:18:32 +1000."
	<3FCDC688.4080001@iinet.net.au> 
References: <024201c3b910$d3e855a0$6402a8c0@arkdesktop>
	<200312022028.hB2KSC902115@c-24-5-183-134.client.comcast.net> 
	<3FCDC688.4080001@iinet.net.au> 
Message-ID: <200312031417.hB3EHLi03515@c-24-5-183-134.client.comcast.net>

> Guido van Rossum wrote:
> > I don't particularly like either approach, because the 'long' type is
> > an implementation detail over which the user has no control (this is a
> > change of heart since Python's original design!).  I guess that means
> > I have to work harder and make the single int type support both
> > representations.  I'm sure it can be done.

Nick Coghlan:
> Isn't the 'long' format the more general representation, with the 'int' 
> format then being a performance hack to take advantage of the speed of C 
> integer arithmetic?
> 
> I'm just wondering if a different way of thinking about it might help 
> with figuring out how to handle a combined implementation.

I proposed that too, but Andrew Koenig didn't like it.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From duncan at rcp.co.uk  Wed Dec  3 09:25:25 2003
From: duncan at rcp.co.uk (Duncan Booth)
Date: Wed Dec  3 09:25:35 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <001001c3b988$8b1054e0$e841fea9@oemcomputer>
	<3FCDCDA1.2040402@iinet.net.au>
Message-ID: <n2m-g.Xns9446924AE93A6duncanrcpcouk@127.0.0.1>

Nick Coghlan <ncoghlan@iinet.net.au> wrote in
news:3FCDCDA1.2040402@iinet.net.au: 

> Raymond Hettinger wrote:
>> Carried to the limit, the idea turns into something that is
>> either sublime or severely bonkers.  The good points are that
>> Guido gets his dotted access and I get to trade in the two ugly
>> names and for a single beautiful "extract".  And there's no
>> performance cost, the inner loop is the same.  The downside is I
>> still don't know how to explain it (AFAICT, super() is the closest
>> thing to it): 
> 
> What you end up with is still a unary operator, so it could still live
> in the operator module, to.
> 
> And I think what you posted would work for strings as dictionary keys,
> too - answering another of the objections to the original
> operator.extract 
> 
You could even consider reinstating the ability to chain extract 
operations. (Although this may be even harder to explain).
The version below allows attribute and item access to be chained and mixed 
freely:


class ExtractorClass(object):
    def __init__(self, fn=None, arg=None, parent=None):
        if fn is None:
            extract = []
        else:
            extract = [(fn,arg)]
        if parent is not None:
            extract = parent._accessors + extract
        self._accessors = extract
            
    def __getattribute__(self, attr):
        if attr == '_accessors':
            return object.__getattribute__(self, attr)
        return ExtractorClass(getattr, attr, self)
    
    def __getitem__(self, key):
        return ExtractorClass(operator.getitem, key, self)
    
    def __call__(self, obj):
        for fn, arg in self._accessors:
            obj=fn(obj, arg)
        return obj
    
extract = ExtractorClass()

class A:
    pass
a = A()
a.score = 10
b = A()
a.b = b
b.score = 42
getscore = extract.score      # Houston, we have dotted access
print getscore(a)

getsubscore = extract.b.score # Chaining
print getsubscore(a)

b = [10, 20, 30, 40]

getsecond = extract[1]        # and, we have the bracketed lookups 
print getsecond(b)

a.b = b
getbsecond = extract.b[1]   # Chain a mix of attributes and indexes.
print getbsecond(a)



-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

From theller at python.net  Wed Dec  3 09:40:42 2003
From: theller at python.net (Thomas Heller)
Date: Wed Dec  3 09:40:55 2003
Subject: [Python-Dev] unittest output
In-Reply-To: <200312031409.hB3E9wt03471@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Wed, 03 Dec 2003 06:09:58 -0800")
References: <000f01c3b984$80bb8360$e841fea9@oemcomputer>
	<200312031409.hB3E9wt03471@c-24-5-183-134.client.comcast.net>
Message-ID: <3cc22cd1.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

>> > > When a test fails, it is accompanied by tracebacks that include
>> > > all the processing within the unittest module itself.  This makes
>> > > it darned difficult to see at a glance what went wrong.  If like
>> > > me, you use unittest as part of the development process, then you
>> > > run it hundreds of times each week and this eyesore gets to be a
>> > > real PITA.
>> > >
>> > > I don't recall when or where, but someone has proposed a fix.
>> > > But, like many great ideas, it got stymied because someone,
>> > > somewhere has a use for the rest of the traceback.   Also, a
>> > > developer raised the (IMO red herring) issue -- what if there
>> > > is a bug in the unittest module, how would you track it.
>> > 
>> > http://www.python.org/sf/722638
>> 
>> Arghh.  I had forgotten that Steve Purcell weighed in on this.
>> It's his module.  So unless he can be persuaded, the point is moot :-(
>
> Not so fast.
>
> The patch claims to only suppress the traceback if the test *failed*
> (e.g. called self.fail(), or one of the assertXXX variations failed).
> If the test raised an unexpected exception (unittest calls this an
> *error*) the traceback is printed normally.  This seems right to me.
>
> A refinement could be to make the output *look* like a (short)
> traceback as printed by the traceback module even in the failure case;
> this would address Steve's issue tht IDEs find lines in the code via
> the traceback.  And there should be a command line switch and an
> environment variable to show the full traceback in all cases.

The patch does this, also.  My original usecase was that I wanted Xemacs
to automatically go to the line of the failed test, and not inside the
unittest source line where the exception was raised.

And the patch *only* suppresses the deepest frame (that inside the
unittest source code), if the traceback is n levels deep, n-1 levels
are printed.

At least this was my intention, if the patch has a chance to make it in,
I'll try it again, and bring it up to date if needed.

(Another question is this: is raising an exception the right thing to do
when a test fails? I'm not so sure, although changing this would be a
much bigger change...)

Thomas


From guido at python.org  Wed Dec  3 10:00:04 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 10:00:16 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Wed, 03 Dec 2003 13:16:30 +1300."
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz> 
References: <200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.net>

(This thread has nothing to do with the groupby iterator any more, but
I'm loathe to change the subject since so many messages are already
under this one.)

(I've read quite a few messages posted after Greg's post, but Greg
still summarizes the issue best for me, *and* it has an alternative
idea that needs a response.)

[Greg Ewing]
> We seem to be talking about two different things in this thread,
> speed and readability. The nice thing about the "attrgetter.x"
> etc. idea is that it has the potential to provide both at once.
> 
> The nasty thing about it is that it smells a bit too much like a
> clever trick. It's really abusing the syntax to make it mean
> something different from what it usually means.

It is also somewhat weak in that it only addresses lambdas with one
argument, and only allows a single reference to that argument in the
resulting expression, and can't really be made to handle method calls
without more gross notational hacks -- even though it *can* be made to
handle arbitrary binary and unary operators.

Yet, it captures 90% of the use cases quite well.  I also wonder if
the simple trick of requiring to call a "constructor" on each use
might not make it more palatable.  I.e., instead of writing

  map(Voodoo.address[0], database)

you'd write

  map(Voodoo().address[0], database)

where you can replace Voodoo with a name of your choice, perhaps
operator.extract -- although I think this is too different to belong
in the operator module.  Nick Goghlan showed that a pretty readable
brief explanation *can* be written.

On the other hand...

> I think I like the idea of optimising lambda, but that doesn't do
> anything for the readability.

It's also been shown by now to be a bad idea -- the semantic
differences are too subtle (e.g. keyword args).

> So, how about a nicer syntax for lambda?  Maybe along the lines of
> 
>    x -> x.something
> 
> A bonus of introducing a new lambda syntax is that it would provide
> the opportunity to give it early-binding semantics for free
> variables, like generator expressions.

This is what everyone seems to expect and want of lambda anyway...

> The old lambda would have to be kept around for a while for programs
> relying on the old semantics, but it could be deprecated, and
> removed in 3.0.

I'm not sure that the -> notation is more understandable than lambda;
it would surely confuse C/C++ programmers who are new to Python.

Scary thought: how about simply introducing early-binding semantics
for lambda in 3.0?

Another radical idea would be to use an anonymous-block notation like
Smalltalk and Ruby.  We could use some kind of funky brackets like
[|...|].  A lambda would require an argument notation too.  I believe
Ruby uses [|x| x+1] where we would write lambda x: x+1, maybe we could
use [|x: x+1|].  (I like structure with an explicit close more than
open ones like lambda.)

Yet another far-out thought: I'd hoped to have gotten rid of most use
cases for lambda with list comprehensions, recently generalized into
generator expressions.  But we keep inventing things (like
list.sort(key=), and now groupby(key=)) that aren't expressible using
generator expressions.  Perhaps we should try harder to find a
generalization that covers these cases too, or to define APIs that
*can* be used with generator expressions?

For groupby, the best I can think of would be to change its API to
take an iterable of (key, value) pairs, so you could write:

  groupby((x.key, x) for x in sequence)

instead of

  groupby(sequence, lambda x: x.key)

but that doesn't work for list.sort(), where the sequence already
exists, and the whole point is to avoid having to make the explicit
decorate-sort-undecorate step.  (Well, the groupby does the decorate
part sort-of explicit and avoids the undecorate, so it gets there at
least halfway.)

I guess the most radical idea would be to have the scope of a
generator expression extend to other arguments of the same call, so
you could write

  groupby(x for x in sequence, x.key)

but that looks too subtle, not to mention ambiguous, and perhaps
unimplementable -- what if it was instead

  groupby(x.value for x in sequence, x.key)

???

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec  3 10:05:16 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 10:05:27 2003
Subject: [Python-Dev] unittest output
In-Reply-To: Your message of "Wed, 03 Dec 2003 15:40:42 +0100."
	<3cc22cd1.fsf@python.net> 
References: <000f01c3b984$80bb8360$e841fea9@oemcomputer>
	<200312031409.hB3E9wt03471@c-24-5-183-134.client.comcast.net> 
	<3cc22cd1.fsf@python.net> 
Message-ID: <200312031505.hB3F5Gj03643@c-24-5-183-134.client.comcast.net>

> http://www.python.org/sf/722638 

> > A refinement could be to make the output *look* like a (short)
> > traceback as printed by the traceback module even in the failure case;
> > this would address Steve's issue tht IDEs find lines in the code via
> > the traceback.  And there should be a command line switch and an
> > environment variable to show the full traceback in all cases.
> 
> The patch does this, also.  My original usecase was that I wanted Xemacs
> to automatically go to the line of the failed test, and not inside the
> unittest source line where the exception was raised.

Right.  I have the same use case.

> And the patch *only* suppresses the deepest frame (that inside the
> unittest source code), if the traceback is n levels deep, n-1 levels
> are printed.

Ah.  I didn't look at the patch, but in your description you also
suppress the line

Traceback (most recent call last):

*and* you only show a one-item traceback, so that wasn't very clear.

> At least this was my intention, if the patch has a chance to make it in,
> I'll try it again, and bring it up to date if needed.

Please do -- feel free to add a pointer to this thread to the SF tracker.

> (Another question is this: is raising an exception the right thing to do
> when a test fails? I'm not so sure, although changing this would be a
> much bigger change...)

Well, we're stuck with that because everybody writes tests that
contain many assertEqual() (of failIf() etc.) calls in a row and
expects the control flow to end if one such call fails.  Without
raising an exception the rest of the test function would be executed
regardless, and that would probably raise the probability of
unexpected exceptions dramatically.  So don't bring that issue up in
the same SF item...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Wed Dec  3 10:27:49 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Dec  3 10:28:32 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.net>
Message-ID: <003a01c3b9b2$02c3a7c0$e841fea9@oemcomputer>

> (This thread has nothing to do with the groupby iterator any more, but
> I'm loathe to change the subject since so many messages are already
> under this one.)
> 
> (I've read quite a few messages posted after Greg's post, but Greg
> still summarizes the issue best for me, *and* it has an alternative
> idea that needs a response.)

I'm am still curious to hear your thoughts on the either sublime or
crazy idea for a generalized extractor object:

list.sorted(students, key=extract.score)
list.sorted(animal_weights, key=extract[1])
groupby(students, key=extract.homeroom)
groupby(recordtuples, key=extract[0])



Raymond Hettinger



P.S.  The implementation code is very simple:

class ExtractorClass(object):
    def __getattribute__(self, attr):
        return operator.attrgetter(attr)
    def __getitem__(self, key):
        return operator.itemgetter(key)

extract = ExtractorClass()


From guido at python.org  Wed Dec  3 10:31:33 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 10:31:49 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Wed, 03 Dec 2003 10:27:49 EST."
	<003a01c3b9b2$02c3a7c0$e841fea9@oemcomputer> 
References: <003a01c3b9b2$02c3a7c0$e841fea9@oemcomputer> 
Message-ID: <200312031531.hB3FVXQ03914@c-24-5-183-134.client.comcast.net>

> I'm am still curious to hear your thoughts on the either sublime or
> crazy idea for a generalized extractor object:
> 
> list.sorted(students, key=extract.score)
> list.sorted(animal_weights, key=extract[1])
> groupby(students, key=extract.homeroom)
> groupby(recordtuples, key=extract[0])

This is the same as the "Voodoo" idea; my comments on that are
repeated here:

    """
    It is also somewhat weak in that it only addresses lambdas with one
    argument, and only allows a single reference to that argument in the
    resulting expression, and can't really be made to handle method calls
    without more gross notational hacks -- even though it *can* be made to
    handle arbitrary binary and unary operators.

    Yet, it captures 90% of the use cases quite well.  I also wonder if
    the simple trick of requiring to call a "constructor" on each use
    might not make it more palatable.  I.e., instead of writing

      map(Voodoo.address[0], database)

    you'd write

      map(Voodoo().address[0], database)

    where you can replace Voodoo with a name of your choice, perhaps
    operator.extract -- although I think this is too different to belong
    in the operator module.  Nick Goghlan showed that a pretty readable
    brief explanation *can* be written.
    """

--Guido van Rossum (home page: http://www.python.org/~guido/)

From ark-mlist at att.net  Wed Dec  3 10:41:43 2003
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed Dec  3 10:41:38 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312031417.hB3EHLi03515@c-24-5-183-134.client.comcast.net>
Message-ID: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop>

> > Isn't the 'long' format the more general representation, with the 'int'
> > format then being a performance hack to take advantage of the speed of C
> > integer arithmetic?
> >
> > I'm just wondering if a different way of thinking about it might help
> > with figuring out how to handle a combined implementation.
> 
> I proposed that too, but Andrew Koenig didn't like it.

It's not me, it's Barbara Liskov at MIT.

The general notion is that if Y is derived from X, then you should be able
to use a Y wherever you might otherwise want to use an X.  In other words, Y
should support every operation that X supports, but Y can add operations of
its own.


From guido at python.org  Wed Dec  3 11:10:00 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 11:10:14 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Wed, 03 Dec 2003 10:41:43 EST."
	<011201c3b9b3$f42b7880$6402a8c0@arkdesktop> 
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop> 
Message-ID: <200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>

> It's not me, it's Barbara Liskov at MIT.

:-)

(For non-OO wizards, this is called "Liskov substitutability".)

> The general notion is that if Y is derived from X, then you should
> be able to use a Y wherever you might otherwise want to use an X.
> In other words, Y should support every operation that X supports,
> but Y can add operations of its own.

So the question is, does long have operations that int doesn't have?
And if so, why can't those operations be added to int?  And if there's
a reason, is it good enough?

If the sets of operations are identical, is there a way to break the
tie?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From edcjones at erols.com  Wed Dec  3 11:06:48 2003
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Dec  3 11:10:31 2003
Subject: [Python-Dev] Python suggestions mostly from C99
Message-ID: <3FCE0A18.7080108@erols.com>

Here are a few suggestions for Python:

Extend hex() and %x to work on longs, floats, and strings.

Add the C99 %a and %A formats.

Add a module that wraps the C99 macros and functions that work with 
special floating point values and floating point exceptions.

Since most C compilers are not C99 compliant, some of these ideas might 
need to be written in C rather than just wrapped.



From fincher.8 at osu.edu  Wed Dec  3 12:15:17 2003
From: fincher.8 at osu.edu (Jeremy Fincher)
Date: Wed Dec  3 11:17:23 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop>
	<200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
Message-ID: <200312031215.17933.fincher.8@osu.edu>

On Wednesday 03 December 2003 11:10 am, Guido van Rossum wrote:
> So the question is, does long have operations that int doesn't have?
> And if so, why can't those operations be added to int?  And if there's
> a reason, is it good enough?

Taking into account their difference in representation, a long can support 
1<<32, but an int can't.

I'm not saying that the "proper" (long) behavior can't be unified into a 
single type, just giving an example of an operation long supports but int 
doesn't.

Jeremy

From gball at cfa.harvard.edu  Wed Dec  3 11:18:21 2003
From: gball at cfa.harvard.edu (Greg Ball)
Date: Wed Dec  3 11:18:25 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <Pine.LNX.4.44.0312031055550.1066-100000@tane.cfa.harvard.edu>

Would it work to generalise generator expressions in the following way:

(x[1] for x) == lambda x:x[1]
(x.score for x) == lambda x: x.score
(x+y for x,y) == lambda x,y: x+y
(len(x) for x) == lambda x,len=len: len(x)  # roughly equivalent

i.e. an anonymous block notation, with early binding semantics.  It could
work as a lambda replacement. But no-argument lambdas would require some
extra ugliness.

(1 for)  == lambda:1 #    yuck!
(1 for ()) == lambda:1 # better?


And should this be allowed?

((a,b) for a in l for b,l) == lambda b,l: ((a,b) for a in l)


--
Greg Ball





From tim at zope.com  Wed Dec  3 11:26:08 2003
From: tim at zope.com (Tim Peters)
Date: Wed Dec  3 11:26:35 2003
Subject: [Python-Dev] test_unicode_file failing on trunk, Win98SE
Message-ID: <LNBBLJKPBEHFEDALKOLCGEKOHIAB.tim@zope.com>

C:\Code\python\PCbuild>python ../lib/test/test_unicode_file.py
test_directories (__main__.TestUnicodeFiles) ... ERROR
test_equivalent_files (__main__.TestUnicodeFiles) ... ok
test_single_files (__main__.TestUnicodeFiles) ... ERROR

======================================================================
ERROR: test_directories (__main__.TestUnicodeFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 161, in test_directories
    os.getcwdu)
  File "../lib/test/test_unicode_file.py", line 98, in _do_directory
    os.mkdir(make_name)
OSError: [Errno 2] No such file or directory: '@test-?????.dir'
Don't know about 2.3 maint, & can't test that today:

C:\Code\python\PCbuild>python ../lib/test/test_unicode_file.py
test_directories (__main__.TestUnicodeFiles) ... ERROR
test_equivalent_files (__main__.TestUnicodeFiles) ... ok
test_single_files (__main__.TestUnicodeFiles) ... ERROR

*======================================================================
ERROR: test_single_files (__main__.TestUnicodeFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 142, in test_single_files
    self._test_single(TESTFN_UNICODE)
  File "../lib/test/test_unicode_file.py", line 116, in _test_single
    self._do_single(filename)
  File "../lib/test/test_unicode_file.py", line 32, in _do_single
    os.utime(filename, None)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-7:
ordinal not in range(128)

----------------------------------------------------------------------
Ran 3 tests in 0.170s

FAILED (errors=2)
Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 169, in ?
    test_main()
  File "../lib/test/test_unicode_file.py", line 166, in test_main
    run_suite(suite)
  File "C:\CODE\PYTHON\lib\test\test_support.py", line 267, in run_suite
    raise TestFailed(msg)
test.test_support.TestFailed: errors occurred; run in verbose mode for
details

C:\Code\python\PCbuild>


From pje at telecommunity.com  Wed Dec  3 11:30:35 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  3 11:28:37 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <Pine.LNX.4.44.0312031055550.1066-100000@tane.cfa.harvard.e
 du>
Message-ID: <5.1.0.14.0.20031203112646.03c0a8f0@mail.telecommunity.com>

At 11:18 AM 12/3/03 -0500, Greg Ball wrote:
>Would it work to generalise generator expressions in the following way:
>
>(x[1] for x) == lambda x:x[1]
>(x.score for x) == lambda x: x.score
>(x+y for x,y) == lambda x,y: x+y
>(len(x) for x) == lambda x,len=len: len(x)  # roughly equivalent

-1.  It looks like a generator expression, but the semantics are way too 
different.  Unless you intended that it should be invoked with .next() 
rather than __call__, but then there's no way to pass the arguments.

(FWIW, my first reaction was, "Whoa!  Cool!", until I thought about how 
you'd actually have to use it.)


From pje at telecommunity.com  Wed Dec  3 11:38:51 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  3 11:36:53 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312031215.17933.fincher.8@osu.edu>
References: <200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
	<011201c3b9b3$f42b7880$6402a8c0@arkdesktop>
	<200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
Message-ID: <5.1.0.14.0.20031203113051.03c05e20@mail.telecommunity.com>

At 12:15 PM 12/3/03 -0500, Jeremy Fincher wrote:
>On Wednesday 03 December 2003 11:10 am, Guido van Rossum wrote:
> > So the question is, does long have operations that int doesn't have?
> > And if so, why can't those operations be added to int?  And if there's
> > a reason, is it good enough?
>
>Taking into account their difference in representation, a long can support
>1<<32, but an int can't.

Not so.  Both '1' and '32' can be represented by int, so only the operation 
*result* needs to be a long.

Further, if the idea here is that 'int' will be a subclass of 'long', then 
it's perfectly valid to return an int from any operation declared as 
returning a long.  Further, since it's acceptable to *pass* an int for any 
argument declared 'long', it should suffice to use 'long' for all integer 
inputs and outputs of methods on 'long'.

Anyway, if the idea is that 'long' will be the base class, IMO the name 
'long' is confusing.  It should probably be called 'integer', with the 
subclass being either 'int' or 'short'.


From pje at telecommunity.com  Wed Dec  3 11:44:33 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  3 11:42:36 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Wed, 03 Dec 2003 13:16:30 +1300."
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
Message-ID: <5.1.0.14.0.20031203100859.0247ddd0@mail.telecommunity.com>

At 07:00 AM 12/3/03 -0800, Guido van Rossum wrote:
>It is also somewhat weak in that it only addresses lambdas with one
>argument, and only allows a single reference to that argument in the
>resulting expression, and can't really be made to handle method calls
>without more gross notational hacks -- even though it *can* be made to
>handle arbitrary binary and unary operators.
>
>Yet, it captures 90% of the use cases quite well.  I also wonder if
>the simple trick of requiring to call a "constructor" on each use
>might not make it more palatable.  I.e., instead of writing
>
>   map(Voodoo.address[0], database)
>
>you'd write
>
>   map(Voodoo().address[0], database)
>
>where you can replace Voodoo with a name of your choice, perhaps
>operator.extract -- although I think this is too different to belong
>in the operator module.  Nick Goghlan showed that a pretty readable
>brief explanation *can* be written.

What if it was possible to use, e.g:

arg(0).foo + arg('somekeyword').bar

That is, have the constructor take an argument position number or kwarg 
name (perhaps defaulting to 0 for convenience)?  Then, you could create 
expressions that replaced multi-argument lambdas.  Then, the only things 
missing are ways to construct structure expressions like 
[arg(0).foo,arg(1).bar].  For that, you'd also need an 'args', and to 
invoke other functions you'd need an argapply.  Yuck.

I guess that limiting these "argument expressions" to what one might find 
in an SQL 'select' or 'where' clause (minus nested selects, of course) 
might not be a bad thing.

There might also be a problem with supporting all binary 
operators...  using 'not in' would break, I think, since there isn't a 
__notcontains__ method.  :)   You'd have to use e.g. ~(arg(0) in [1,2,9]) 
instead of 'arg(0) not in [1,2,9]' or 'not arg(0) in [1,2,9]'.

Anyway, the implementation of binary operators would also need to check if 
the 'other' argument was another "argument expression", and if so return a 
slightly different result than it would for a non-argument (i.e. constant) 
expression.

Interestingly, all this is quite similar to Ian Bicking's SQLObject, which 
uses expressions of this sort to represent SQL as Python code.


From tzot at sil-tec.gr  Wed Dec  3 12:04:44 2003
From: tzot at sil-tec.gr (Christos Georgiou)
Date: Wed Dec  3 12:04:52 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <7j5ssvs27cb89qni7eggfo782lo6gdbahj@4ax.com>

Guido van Rossum said:

>> The nasty thing about it is that it smells a bit too much like a
>> clever trick. It's really abusing the syntax to make it mean
>> something different from what it usually means.
>
>It is also somewhat weak in that it only addresses lambdas with one
>argument, and only allows a single reference to that argument in the
>resulting expression, and can't really be made to handle method calls
>without more gross notational hacks -- even though it *can* be made to
>handle arbitrary binary and unary operators.

A minor correction: the Voodoo approach handles fine method calls by
defining a __call__ method in the Voodoo class; it can't handle
*function* calls in an elegant way:

lambda x: x.startswith("text")
This works fine as:
Voodoo().startswith("text")

lambda x: math.sin(x)
This needs gross syntax, the following does not work:
math.sin(Voodoo())
-- 
TZOTZIOY, I speak England very best,
Ils sont fous ces Redmontains! --Harddix

From guido at python.org  Wed Dec  3 12:26:06 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 12:26:17 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Wed, 03 Dec 2003 12:15:17 EST."
	<200312031215.17933.fincher.8@osu.edu> 
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop>
	<200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net> 
	<200312031215.17933.fincher.8@osu.edu> 
Message-ID: <200312031726.hB3HQ6504171@c-24-5-183-134.client.comcast.net>

> > So the question is, does long have operations that int doesn't have?
> > And if so, why can't those operations be added to int?  And if there's
> > a reason, is it good enough?
> 
> Taking into account their difference in representation, a long can support 
> 1<<32, but an int can't.

We're talking about a hypothetical int here where that operation
returns 4294967296L.  (Not so hypothetical, it's implemented in Python
2.4 in CVS.)

> I'm not saying that the "proper" (long) behavior can't be unified into a 
> single type, just giving an example of an operation long supports but int 
> doesn't.

Sorry, your counterexample is rejected. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Dec  3 12:29:46 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  3 12:27:49 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <7j5ssvs27cb89qni7eggfo782lo6gdbahj@4ax.com>
Message-ID: <5.1.0.14.0.20031203121737.04321020@mail.telecommunity.com>

At 07:04 PM 12/3/03 +0200, Christos Georgiou wrote:

>A minor correction: the Voodoo approach handles fine method calls by
>defining a __call__ method in the Voodoo class;

__call__ is needed for the resulting object to be callable.


>it can't handle
>*function* calls in an elegant way:
>
>lambda x: x.startswith("text")
>This works fine as:
>Voodoo().startswith("text")

Voodoo().startswith("text") is equivalent to (lambda x: 
x.startswith)("text"), which is not the same thing.



>lambda x: math.sin(x)
>This needs gross syntax, the following does not work:
>math.sin(Voodoo())

That too, but also method calls.  Basically, once you get past really 
simple expressions, the whole thing collapses under its own weight.

Ironically, it would be easy to do this with generic functions, since one 
would simply define a methods for the case 'math.sin(Voodoo)' that returns 
lambda x: math.sin(x).

(Hm.  This gives me an entirely unrelated idea about how to unify object 
methods and generic functions in Python using "curried 
generics".  Interesting.  And the Voodoo() aka 'arg()' concept could be 
applied to predicate dispatching as well.)


From edloper at gradient.cis.upenn.edu  Wed Dec  3 13:30:25 2003
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Wed Dec  3 12:28:53 2003
Subject: [Python-Dev] Tutorial: Brief Introduction to the Standard Libary
In-Reply-To: <E1ARTPt-0001tE-QE@mail.python.org>
References: <E1ARTPt-0001tE-QE@mail.python.org>
Message-ID: <3FCE2BC1.10304@gradient.cis.upenn.edu>

> Attached is a revised draft.  Comments, suggestions, nitpicks,
> complaints, and accolades are welcome.

General comments:
   - Overall, it looks very good. :)
   - Since your descriptions are so brief, you should probably provide
     links to the reference documentation for each module/function that
     you reference.
   - You might also want to mention pydoc.  Not only is it a very useful
     module for beginners in its own right, but it would give them a way
     to learn more about the other modules that you describe.

> >>>import os
> >>>os.system('copy /data/mydata.fil /backup/mydata.fil')
> 0
> 
> >>>os.getcwd() 
> 'C:\\Python24'
> 
> >>>os.chdir('/server/accesslogs')

You might want to add short "#" comments for these, saying what they do. 
  (Some novices won't recognize what system() is doing; and not everyone 
immediately associates cwd with current working directory.)  Nothing 
fancy, just something like:

   >>> os.getcwd()    # Return the current working directory

> Be sure to use the "import os" style instead of "from os import *".  
> This will keep os.open() from shadowing __builtin__.open()  which 
> operates much differently.

This sentence would be more accessible to beginners if you said "from 
shadowing the builtin open() function, which operates much differently."

>>>>optlist, args = getopt.getopt(sys.argv[1:], 'abc:d:')

It might be worth including an option that doesn't get used, just to 
show that they're optional.

> because they are easier to read and debug.  For more sophisticated applications,

I would replace s/For/But for/

> >>>import urllib

That should be "import urllib2"

> For example, it may be tempting to use the tuple packing and unpacking feature
> instead of the traditional approach to swapping arguments.  The timeit module
> quickly demonstrates that the traditional approach is faster:

I think that you should add a caveat here. Something along the lines of 
"But the tuple method is much easier to read, and not much faster, so 
unless it's in a really tight loop where speed is critical, you should 
use the tuple method."

> In contrast to timeit's fine level of granularity, the profile and pstats modules
> provide tools for identifying time critical sections of larger blocks of code.

I think it's worth noting that profile & pstats are generally much more 
useful for optimizing code than timeit is.  (That fact might not be 
obvious to beginners.)

> Batteries Included
> 
> Python has a "batteries included" philosophy.  The is best seen through the
> sophisticated and robust capabilites of its larger packages:

s/packages:/packages, including:/

Also, I'm not sure if the meaning of "batteries included" will be 
obvious to people who haven't seen the term before.  Perhaps you could 
steal some text from PEP 206; so something like:

   Python has a "batteries included" philosophy -- it includes
   a rich and versatile standard library which is immediately available,
   without making the user download separate packages.  The is best seen
   through the sophisticated and robust capabilites of its larger
   packages, including:

-Edward


From mfb at lotusland.dyndns.org  Wed Dec  3 12:34:33 2003
From: mfb at lotusland.dyndns.org (Matthew F. Barnes)
Date: Wed Dec  3 12:34:50 2003
Subject: [Python-Dev] Python suggestions mostly from C99
In-Reply-To: <3FCE0A18.7080108@erols.com>
References: <3FCE0A18.7080108@erols.com>
Message-ID: <10469.130.76.64.14.1070472873.squirrel@server.lotusland.dyndns.org>

Edward C. Jones said:
> Extend hex() and %x to work on longs, floats, and strings.

Just FYI -
Extending hex() (and oct()) to work with strings has been discussed in the
past:

<http://mail.python.org/pipermail/python-dev/2003-August/037855.html>

Matthew Barnes


From guido at python.org  Wed Dec  3 12:48:05 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 12:48:21 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Wed, 03 Dec 2003 19:04:44 +0200."
	<7j5ssvs27cb89qni7eggfo782lo6gdbahj@4ax.com> 
References: <7j5ssvs27cb89qni7eggfo782lo6gdbahj@4ax.com> 
Message-ID: <200312031748.hB3Hm5504233@c-24-5-183-134.client.comcast.net>

> >It is also somewhat weak in that it only addresses lambdas with one
> >argument, and only allows a single reference to that argument in the
> >resulting expression, and can't really be made to handle method calls
> >without more gross notational hacks -- even though it *can* be made to
> >handle arbitrary binary and unary operators.
> 
> A minor correction: the Voodoo approach handles fine method calls by
> defining a __call__ method in the Voodoo class; [...]
> 
> lambda x: x.startswith("text")
> This works fine as:
> Voodoo().startswith("text")

Hm.  But the whole point of Voodoo is that the resulting object has a
__call__ method that evaluates the expression for a given argument.
How do you do that?  (I suppose you can do it if you don't do
chaining, but I like the chaining.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tzot at sil-tec.gr  Wed Dec  3 13:28:47 2003
From: tzot at sil-tec.gr (Christos Georgiou)
Date: Wed Dec  3 13:28:52 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <ug9ssvsa9lojou3ma7r7ib9p7d3jlv44fo@4ax.com>

Guido van Rossum said:

[Christos]
>> A minor correction: the Voodoo approach handles fine method calls by
>> defining a __call__ method in the Voodoo class; [...]
>> 
>> lambda x: x.startswith("text")
>> This works fine as:
>> Voodoo().startswith("text")
>
>Hm.  But the whole point of Voodoo is that the resulting object has a
>__call__ method that evaluates the expression for a given argument.
>How do you do that?  (I suppose you can do it if you don't do
>chaining, but I like the chaining.)

(replying to Phillip Eby too)

Yes, you are correct (I was thinking in terms of my All and Any classes,
which don't need to be used as callables).

There is a solution involving a Voodoo and an (unexported)
VoodooAttribute class which *can* solve this (Voodoo.__getattr__ returns
an instance of VoodooAttribute (subclass of Voodoo overloading
__call__), whose .__call__ method returns an instance of Voodoo), *but*
then you can't use:

Voodoo().startswith # this is a VoodooAttribute instance

as an argument to groupby, unless you use:

Voodoo(Voodoo().startswith) # this is a Voodoo instance

I can't think at the moment a case in the groupby context where the
latter would be needed (a VoodooAttribute behaves the same for
attributes that are not methods), but I'm sure many others will, and
even if they don't, it sure smells 'breakage'.

Unless the Voodoo class does what it does best (and I'll rename it to
Argument for clarity below), and an instance of it is the only argument
to the *explicit* initialisation of the FuncMaker class (the final
Voodoo :):

lambda x: x.startswith("text")
equals
FuncMaker(Argument().startswith("text"))

Too unpythonic perhaps?

PS attrgetter and itemgetter sound fine btw, I'm just trying to help
this brainstorming for more general solutions --if any.
-- 
TZOTZIOY, I speak England very best,
Ils sont fous ces Redmontains! --Harddix

From pf_moore at yahoo.co.uk  Wed Dec  3 13:31:52 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Wed Dec  3 13:31:41 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
	<200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.net>
Message-ID: <llptkb1j.fsf@yahoo.co.uk>

Guido van Rossum <guido@python.org> writes:

> [Greg Ewing]
>> We seem to be talking about two different things in this thread,
>> speed and readability. The nice thing about the "attrgetter.x"
>> etc. idea is that it has the potential to provide both at once.

Yes. Frankly, readability (anybody's flavour) is easy to handle - all
of the various suggestions can be implemented in user code, without
any real problem. While a standard library solution may be useful for
consistency across applications, it'll only get consistently used if
it gives some other significant benefit (ie, speed).

So speed is the crucial point here, with readability a poor second. 
But speed is very rarely *really* crucial, no matter what people might
think. So we need something that is fast and nice-looking, to avoid
the "attractive nuisance" of something slower, but more "obvious" or
"readable" (ie, lambda).

Personally, I find the name operator.attrgetter (with semantics as
currently implemented) ugly enough that I'll probably never use it (or
only rarely, where speed is crucial). It's definitely not "the one
obvious way" for me. But naming is highly personal, so that's not a
good argument by itself.

>> The nasty thing about it is that it smells a bit too much like a
>> clever trick. It's really abusing the syntax to make it mean
>> something different from what it usually means.

While it still uses the name "attrgetter", I agree. However, with the
name "extract", I like the look of it enough that I'd overlook the
syntax abuse. But Guido's suggestion of requiring a constructor call
would swing me back the other way again (actually, I'd cheat and do
extract = extract() once and be done with it, but I'd feel unclean and
avoid the construct because of this...)

All personal opinion again, of course.

> It is also somewhat weak in that it only addresses lambdas with one
> argument, and only allows a single reference to that argument in the
> resulting expression, and can't really be made to handle method calls
> without more gross notational hacks -- even though it *can* be made to
> handle arbitrary binary and unary operators.
>
> Yet, it captures 90% of the use cases quite well.

I think this is the key thing here. Lambda in its full form is very
flexible and wide ranging - but 90% of the time, this flexibility
isn't needed. And everything pays a cost for that flexibility, even
when it's not used.

Something that did that 90%, fast and efficiently, while leaving
lambda for the few remaining cases where its flexibility is needed
(possibly with an intent to deprecate it in the future) would probably
be the best option.

Paul.
-- 
This signature intentionally left blank


From pf_moore at yahoo.co.uk  Wed Dec  3 13:33:35 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Wed Dec  3 13:40:52 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <200311282146.hASLkr317367@c-24-5-183-134.client.comcast.net>
	<200311302157.49205.aleaxit@yahoo.com>
	<8ylvmcmq.fsf@yahoo.co.uk> <200312021654.34386.fincher.8@osu.edu>
Message-ID: <he0hkayo.fsf@yahoo.co.uk>

Jeremy Fincher <fincher.8@osu.edu> writes:

> On Tuesday 02 December 2003 11:02 am, Paul Moore wrote:
>> Alex Martelli <aleaxit@yahoo.com> writes:
>> > Since these functions or types are going to be in operator, I think
>> > we can afford to "spend" two names to distinguish functionality
>> > (even though attgetter and itemgetter look nowhere as neat as
>> > extract -- I don't have better suggestions offhand).
>>
>> operator.getattr and operator.getitem?
>
> I think the functions they're talking about are actually right-curried 
> versions of those functions, like this:

I know - I was hoping that the fact that these particular versions
were in the operator module might justify using the same name for two
mildly-related things.

But it's probably more confusing than helpful.

Paul
-- 
This signature intentionally left blank


From eppstein at ics.uci.edu  Wed Dec  3 13:50:03 2003
From: eppstein at ics.uci.edu (David Eppstein)
Date: Wed Dec  3 13:49:58 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <7j5ssvs27cb89qni7eggfo782lo6gdbahj@4ax.com>
	<200312031748.hB3Hm5504233@c-24-5-183-134.client.comcast.net>
Message-ID: <eppstein-10D632.10500303122003@sea.gmane.org>

In article 
<200312031748.hB3Hm5504233@c-24-5-183-134.client.comcast.net>,
 Guido van Rossum <guido@python.org> wrote:

> > A minor correction: the Voodoo approach handles fine method calls by
> > defining a __call__ method in the Voodoo class; [...]
> > 
> > lambda x: x.startswith("text")
> > This works fine as:
> > Voodoo().startswith("text")
> 
> Hm.  But the whole point of Voodoo is that the resulting object has a
> __call__ method that evaluates the expression for a given argument.
> How do you do that?  (I suppose you can do it if you don't do
> chaining, but I like the chaining.)

I think you would need two kinds of voodoo.
    extract.attr
could expand to
    lambda x: x.attr
(or something similar via __call__ methods) but
    method.startswith
could expand to
    lambda *args: lambda x: x.startswith(*args)

Then you could do
    sort(stringlist, key=method.count("kibo"))
or whatever.

Obviously this doesn't allow you to easily express e.g. methods of 
attributes, but beyond a certain level of complexity you should be using 
separate defs anyway.

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From pedronis at bluewin.ch  Wed Dec  3 14:14:57 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Wed Dec  3 14:11:52 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Wed, 03 Dec 2003 13:16:30 +1300."
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
Message-ID: <5.2.1.1.0.20031203200451.02860ed0@pop.bluewin.ch>

At 07:00 03.12.2003 -0800, Guido van Rossum wrote:
> > The old lambda would have to be kept around for a while for programs
> > relying on the old semantics, but it could be deprecated, and
> > removed in 3.0.
>
>I'm not sure that the -> notation is more understandable than lambda;
>it would surely confuse C/C++ programmers who are new to Python.
>
>Scary thought: how about simply introducing early-binding semantics
>for lambda in 3.0?

that would confuse schemers <wink>

>Another radical idea would be to use an anonymous-block notation like
>Smalltalk and Ruby.  We could use some kind of funky brackets like
>[|...|].  A lambda would require an argument notation too.  I believe
>Ruby uses [|x| x+1] where we would write lambda x: x+1, maybe we could
>use [|x: x+1|].  (I like structure with an explicit close more than
>open ones like lambda.)

I would expect blocks to be blocks, accepting also statements and sharing 
the scope
with the surrounding code, not having early-binding semantics.

Honestly, given the introduction of generator exprs, a substitute 
expression for lambda
with early-binding semantics makes sense. Personally I can see it also as 
meaningful wrt  the statemenent/expression dichotomy in python, and we 
would have early-binding as a general rule for "expressions".

But again I think that a block-like syntax should be used, if at all, for 
real blocks.

regards. 


From guido at python.org  Wed Dec  3 14:14:40 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 14:14:52 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Wed, 03 Dec 2003 18:31:52 GMT."
	<llptkb1j.fsf@yahoo.co.uk> 
References: <200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
	<200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.net> 
	<llptkb1j.fsf@yahoo.co.uk> 
Message-ID: <200312031914.hB3JEeW04395@c-24-5-183-134.client.comcast.net>

> I think this is the key thing here. Lambda in its full form is very
> flexible and wide ranging - but 90% of the time, this flexibility
> isn't needed. And everything pays a cost for that flexibility, even
> when it's not used.
> 
> Something that did that 90%, fast and efficiently, while leaving
> lambda for the few remaining cases where its flexibility is needed
> (possibly with an intent to deprecate it in the future) would probably
> be the best option.

For the remaining 10%, you could just use 'def' if lambda didn't exist
at all.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec  3 14:19:52 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 14:20:05 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Wed, 03 Dec 2003 20:14:57 +0100."
	<5.2.1.1.0.20031203200451.02860ed0@pop.bluewin.ch> 
References: <Your message of "Wed, 03 Dec 2003 13:16:30 +1300."
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz> 
	<5.2.1.1.0.20031203200451.02860ed0@pop.bluewin.ch> 
Message-ID: <200312031919.hB3JJqg04439@c-24-5-183-134.client.comcast.net>

[Samuele]
> I would expect blocks to be blocks, accepting also statements and
> sharing the scope with the surrounding code, not having
> early-binding semantics.
> 
> Honestly, given the introduction of generator exprs, a substitute
> expression for lambda with early-binding semantics makes
> sense. Personally I can see it also as meaningful wrt the
> statemenent/expression dichotomy in python, and we would have
> early-binding as a general rule for "expressions".
> 
> But again I think that a block-like syntax should be used, if at
> all, for real blocks.

Thanks -- that all makes sense.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Wed Dec  3 14:27:25 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Wed Dec  3 14:25:29 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312031914.hB3JEeW04395@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Wed,
	03 Dec 2003 18:31:52 GMT." <llptkb1j.fsf@yahoo.co.uk>
	<200312030016.hB30GU205334@oma.cosc.canterbury.ac.nz>
	<200312031500.hB3F04b03619@c-24-5-183-134.client.comcast.net>
	<llptkb1j.fsf@yahoo.co.uk>
Message-ID: <5.1.0.14.0.20031203142518.030a3660@mail.telecommunity.com>

At 11:14 AM 12/3/03 -0800, Guido van Rossum wrote:
> > I think this is the key thing here. Lambda in its full form is very
> > flexible and wide ranging - but 90% of the time, this flexibility
> > isn't needed. And everything pays a cost for that flexibility, even
> > when it's not used.
> >
> > Something that did that 90%, fast and efficiently, while leaving
> > lambda for the few remaining cases where its flexibility is needed
> > (possibly with an intent to deprecate it in the future) would probably
> > be the best option.
>
>For the remaining 10%, you could just use 'def' if lambda didn't exist
>at all.

Some use cases could get quite inconvenient unless function decorators were 
also available, but if we're talking about 3.0, then I suppose they would 
be.  :)


From walter at livinglogic.de  Wed Dec  3 14:37:11 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Dec  3 14:37:16 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <2mu14ixjb0.fsf@starship.python.net>
References: <3FCCDCFA.6070800@livinglogic.de>	<16332.56789.356676.91289@grendel.fdrake.net>	<3FCCF2F7.7090805@livinglogic.de>	<16332.62351.960759.135367@grendel.fdrake.net>	<3FCD1A48.5000202@livinglogic.de>
	<2mu14ixjb0.fsf@starship.python.net>
Message-ID: <3FCE3B67.604@livinglogic.de>

Michael Hudson wrote:

> Walter D?rwald <walter@livinglogic.de> writes:
> 
>>The arguments for the PrettyPrinter constructor could
>>be exposed in pprint() and pformat():
>>"""
>>import pprint
>>print.pprint(foo, indent=3, width=40)
>>"""
> 
> +1e6!  I've wanted this for so long I forgot that I could do something
> +about it :-)

OK, then I'll go ahead and implement that.

Bye,
    Walter D?rwald



From walter at livinglogic.de  Wed Dec  3 15:27:46 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Wed Dec  3 15:27:51 2003
Subject: [Python-Dev] pprint and list/tuple/dict subclasses
In-Reply-To: <2mu14ixjb0.fsf@starship.python.net>
References: <3FCCDCFA.6070800@livinglogic.de>	<16332.56789.356676.91289@grendel.fdrake.net>	<3FCCF2F7.7090805@livinglogic.de>	<16332.62351.960759.135367@grendel.fdrake.net>	<3FCD1A48.5000202@livinglogic.de>
	<2mu14ixjb0.fsf@starship.python.net>
Message-ID: <3FCE4742.9040104@livinglogic.de>

Michael Hudson wrote:

> Walter D?rwald <walter@livinglogic.de> writes:
> 
> 
>>The arguments for the PrettyPrinter constructor could
>>be exposed in pprint() and pformat():
>>"""
>>import pprint
>>print.pprint(foo, indent=3, width=40)
>>"""
> 
> 
> +1e6!  I've wanted this for so long I forgot that I could do something
> +about it :-)

Done. (And patch #750542 too)

Bye,
    Walter D?rwald



From ark-mlist at att.net  Wed Dec  3 15:33:23 2003
From: ark-mlist at att.net (Andrew Koenig)
Date: Wed Dec  3 15:33:18 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
Message-ID: <003f01c3b9dc$b2bfe510$6402a8c0@arkdesktop>

> So the question is, does long have operations that int doesn't have?
> And if so, why can't those operations be added to int?  And if there's
> a reason, is it good enough?

If every operation that might conceivably yield an int quietly yields a long
when the result doesn't fit in an int, then it is possible for int and long
to have the same operations.

In which case, why do they have different types?  Just as an implementation
detail?


From guido at python.org  Wed Dec  3 15:42:29 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 15:42:40 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Wed, 03 Dec 2003 15:33:23 EST."
	<003f01c3b9dc$b2bfe510$6402a8c0@arkdesktop> 
References: <003f01c3b9dc$b2bfe510$6402a8c0@arkdesktop> 
Message-ID: <200312032042.hB3KgTu04629@c-24-5-183-134.client.comcast.net>

> > So the question is, does long have operations that int doesn't
> > have?  And if so, why can't those operations be added to int?  And
> > if there's a reason, is it good enough?
> 
> If every operation that might conceivably yield an int quietly
> yields a long when the result doesn't fit in an int, then it is
> possible for int and long to have the same operations.
> 
> In which case, why do they have different types?  Just as an
> implementation detail?

Right.  The implementation uses the type as the vtable of a C++ class,
and the functions in the vtable "know" the representation, which is
very different for short and long ints.  This is all nice and
efficient, except that the identity of the vtable is returned as the
type of the object, so two different vtables means two different
types.  The alternative implementation technique is to have a single
type/vtable, embed a flag in each instance that tells the
implementation which representation is used, and have an "if" at the
top of each implementation function switching between the two.  The
downslide should be obvious: not only an extra test+branch for each
operation, but also extra space in the object (the short
representation has no spare bits).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mhammond at skippinet.com.au  Wed Dec  3 17:22:12 2003
From: mhammond at skippinet.com.au (Mark Hammond)
Date: Wed Dec  3 17:22:28 2003
Subject: [Python-Dev] RE: test_unicode_file failing on trunk, Win98SE
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEKOHIAB.tim@zope.com>
Message-ID: <045401c3b9eb$e6d70450$2c00a8c0@eden>

> C:\Code\python\PCbuild>python ../lib/test/test_unicode_file.py
> test_directories (__main__.TestUnicodeFiles) ... ERROR
> test_equivalent_files (__main__.TestUnicodeFiles) ... ok
> test_single_files (__main__.TestUnicodeFiles) ... ERROR

My apologies - I have fixed it (I hope! No Win98 SE here for me to test on)

Note the error was in the test suite semantics only, not the core code.

> OSError: [Errno 2] No such file or directory: '@test-?????.dir'

Interestingly, this is yet another bug.  The error you *should* have got is
a UnicodeDecodeError.  However, the Windows 'mbcs' encoding currently acts
as if in errors=replace mode.  See 850997.  That is really neither here nor
there tho - all it does is change the error you get when passing a Unicode
filename.

> Don't know about 2.3 maint, & can't test that today:

The fix to posixmodule was checked into both head and 2.3-maint, but the
test suite changes were only made to head - for exactly this reason <wink>.
Thus, 2.3 should be fine.

Please let me know if there are still errors.

Thanks,

Mark.


From greg at cosc.canterbury.ac.nz  Wed Dec  3 18:49:38 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec  3 18:49:46 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <87y8turaep.fsf@egil.codesourcery.com>
Message-ID: <200312032349.hB3Nncq13330@oma.cosc.canterbury.ac.nz>

Zack Weinberg <zack@codesourcery.com>:

> If you're going to mess with lambda at all, please try to come up with
> a way for it not to be limited to a single expression.

That's not the direction I want to go in. Lambdas are best
for extremely small functions. For anything more complicated,
I believe you're better off using a def.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From andrew-pythondev at puzzling.org  Wed Dec  3 19:23:48 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed Dec  3 19:23:55 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <ug9ssvsa9lojou3ma7r7ib9p7d3jlv44fo@4ax.com>
References: <ug9ssvsa9lojou3ma7r7ib9p7d3jlv44fo@4ax.com>
Message-ID: <20031204002348.GA28006@frobozz>

On Wed, Dec 03, 2003 at 08:28:47PM +0200, Christos Georgiou wrote:
> 
> There is a solution involving a Voodoo and an (unexported)
> VoodooAttribute class which *can* solve this (Voodoo.__getattr__ returns
> an instance of VoodooAttribute (subclass of Voodoo overloading
> __call__), whose .__call__ method returns an instance of Voodoo), *but*
> then you can't use:

[...]

This discussion is starting to remind me of something that's been in the
Twisted sandbox for quite a while:
    http://cvs.twistedmatrix.com/cvs/sandbox/cake.py?rev=1.4&content-type=text/vnd.viewcvs-markup

The code isn't exactly clear, but the example in the if __name__ ==
'__main__' block at the bottom should give you some idea of what it can
do... I can't escape the feeling that this discussion is slowly reinventing
cake.py.

I'm not sure I'd really like to read code that uses some of the more
outlandish proposals in this thread... it looks like a good way to write
convoluted code.

-Andrew.


From mike.thompson at day8.com.au  Wed Dec  3 20:46:36 2003
From: mike.thompson at day8.com.au (Mike Thompson)
Date: Wed Dec  3 21:10:37 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop>
	<200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
Message-ID: <bqm3m4$aov$1@sea.gmane.org>


"Guido van Rossum" <guido@python.org> wrote in message
news:200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net...
> > It's not me, it's Barbara Liskov at MIT.
>
> :-)
>
> (For non-OO wizards, this is called "Liskov substitutability".)
>
> > The general notion is that if Y is derived from X, then you should
> > be able to use a Y wherever you might otherwise want to use an X.
> > In other words, Y should support every operation that X supports,
> > but Y can add operations of its own.
>
> So the question is, does long have operations that int doesn't have?
> And if so, why can't those operations be added to int?  And if there's
> a reason, is it good enough?
>
> If the sets of operations are identical, is there a way to break the
> tie?
>

In this case, one could argue that no tie breaker is needed.

If one accepts that int and long have the same operations but offer different
implementations, then the obvious solution is: an abstract base class (Int) and
two sub-classes (Short and Long?).

Of course, this obvious solution has a problem:  a programmer can't easily
sub-class Int.  They must choose one a sub-class from Short or Long (unless
they want to provide a full implementation themselves) and that's probably not
a decision they want to make.

Perhaps, because of this, the GOF "Bridge Pattern" might be suitable here.
(This pattern can be framed as:  "adapt multiple implementations to an
interface using delegation" -- which, after all, is pretty much what the vtable
in previous solution gives you.)

If the existence of Short and Long is an implementation detail best hidden from
the python programmer, then a Bridge Pattern implementation has a lot going for
it.

Use of the Bridge pattern might even allow for three different implementations:
PreDefined, Short, Long  ... where PreDefined is one of the numbers between 0
and 256 which I'm given to understand are pre-allocated by the runtime
environment.

--
Mike




From greg at cosc.canterbury.ac.nz  Wed Dec  3 21:36:54 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec  3 21:37:04 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <5.1.0.14.0.20031203121737.04321020@mail.telecommunity.com>
Message-ID: <200312040236.hB42asw13987@oma.cosc.canterbury.ac.nz>

"Phillip J. Eby" <pje@telecommunity.com>:

> Ironically, it would be easy to do this with generic functions, since one 
> would simply define a methods for the case 'math.sin(Voodoo)' that returns 
> lambda x: math.sin(x).

> (Hm.  This gives me an entirely unrelated idea about how to unify object 
> methods and generic functions in Python using "curried 
> generics".  Interesting.  And the Voodoo() aka 'arg()' concept could be 
> applied to predicate dispatching as well.)

If we go much further along this road, we're going to end
up re-implementing the whole Python interpreter using
threaded code!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From ajsiegel at optonline.net  Wed Dec  3 14:08:13 2003
From: ajsiegel at optonline.net (Arthur)
Date: Wed Dec  3 21:45:16 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard Libary
Message-ID: <0HPC003JRNNFLF@mta7.srv.hcvlny.cv.net>

Michael writes -

>I don't want to be rude -- is English your first language? -- but I
>found this article to be written in particularly baroque prose. 

English first and only.  Same with Python.

But if I have your demographics correct, I am of a different era.

Baroque?  I prefer Dylan, actually,

>I'm not quite sure what your central point is.

I believe you.

>Well, I dunno.  If it was my decision, there's no way it would be
>mentioned in the built-ins

Or in the tutorial accompanying Python?  Which is what is under discussion. 

>No changes here in a long while.

I believe you again.

>I don't think it's unreasonable at all.  To me the copy module is
>something you only need when you really need it, then you go looking
>for it.

If you understand a) when you need it b) it exists.  Tutorial discussion. In
the negative, if you like. It exists, it does this, and if you think you
need it in a particular case are probably wrong, because...would serve my
purpose. Perfectly.

>Certainly, the failure to understand assignment and erroneous beliefs
>that the copy module is necessary in a given situation go hand in hand.

I don't think that view is inconsistent with anything I am asserting.

>Assignment in Python *isn't* particularly subtle, though.  My
>impression is that it's people from e.g. C who get tripped up here,
>not total newbies.

That assertion is, IMO, an overly convenient way to put the matter to bed.
It is an assertion with which I disagree. And I don't know who can claim
particular insight or expertise on this kind of issue, and on what basis.
Which is a problem.

I can promise you that a background slanted by C was not at work in my case.
It would be nice if it were.

>Another reason might just be that copy() is implemented in Python...

I prefer those kinds of reasons, than the newbies ones. Sound weighty and
beyond my ken.  The newbies ones don't. Sorry. 

>Have you ever used the copy module?  I am *not* a beginner, and have
>used it *once* (and I can't remember what for, either).

In my view, there are usually more convenient ways to accomplish what copy
is about, because you don't have to import copy.  And we are just in a
circular argument. And its use is discouraged, as a matter of style. More
circles.  But isn't it a harmless alternative, and a bit more explicit, in a
good deal of circumstances - arguably.

But I truly do understand that I may be missing something fundamental here.
Performance issues? Who knows what else. I'm afraid to find out, actually.


[OT]

"""
EuroPython: What do you like best about Python?

MH: I think, probably, the single biggest plus of Python is that it blurs
the distinction between using and programming a computer.
"""

One of my favorite Python quotes of recent vintage, BTW.

Art


From greg at cosc.canterbury.ac.nz  Wed Dec  3 21:54:21 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec  3 21:54:27 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <bqm3m4$aov$1@sea.gmane.org>
Message-ID: <200312040254.hB42sLI14069@oma.cosc.canterbury.ac.nz>

Mike Thompson <mike.thompson@day8.com.au>:

> Of course, this obvious solution has a problem:  a programmer can't easily
> sub-class Int.

I can't see that subclassing int is a particularly useful thing to do,
even as matters stand today. As soon as you do any operation on your
int subclass, you get a result which is not an instance of your
subclass any more, which makes using it rather fragile.

Does anyone have a real-life use case for subclassing int?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Wed Dec  3 22:16:25 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec  3 22:16:30 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard
	Libary
In-Reply-To: <0HPC003JRNNFLF@mta7.srv.hcvlny.cv.net>
Message-ID: <200312040316.hB43GPM14271@oma.cosc.canterbury.ac.nz>

Arthur <ajsiegel@optonline.net>:

> In my view, there are usually more convenient ways to accomplish what copy
> is about, because you don't have to import copy.  And we are just in a
> circular argument. And its use is discouraged, as a matter of style. More
> circles.  But isn't it a harmless alternative, and a bit more explicit, in a
> good deal of circumstances - arguably.

There isn't any "good deal of circumstances" in the first place.  I'm
not sure why it is, but in my experience, and in other people's too,
it seems, it's very rare that you ever *need* to copy anything in
Python. There's nothing circular here -- it would still be true even
if copy() were a built-in.

And on the rare occasions where you *do* need to copy something, you
need to know a fair bit about the kind of thing you're copying, and
for what purpose you're copying it, so that you know exactly how much
of it to copy. It's a delicate operation that I'd be reluctant to
trust to any general-purpose copying function.

The result of all this is that I don't think I've *ever* used
copy() or deepcopy(). Whenever I've need to copy something I've
always used something type-specific such as list[:] or dict.copy(),
or written my own copying function.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Wed Dec  3 22:22:49 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec  3 22:22:59 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <Pine.LNX.4.44.0312031055550.1066-100000@tane.cfa.harvard.edu>
Message-ID: <200312040322.hB43Mn814298@oma.cosc.canterbury.ac.nz>

Greg Ball <gball@cfa.harvard.edu>:

> (x[1] for x) == lambda x:x[1]
> (x.score for x) == lambda x: x.score
> (x+y for x,y) == lambda x,y: x+y
> (len(x) for x) == lambda x,len=len: len(x)  # roughly equivalent

Interesting idea, but it reads a bit strangely.

Maybe this would work better with a different keyword...

  x[1] given x
  x.score given x
  x+y given x,y
  len(x) given x

  stuff = groupby((x for x in seq), key = (x.score given x))

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From anthony at ekit-inc.com  Wed Dec  3 22:42:05 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Wed Dec  3 22:42:28 2003
Subject: [Python-Dev] minor interruption to service. 
In-Reply-To: Message from Skip Montanaro <skip@pobox.com> of "Tue,
	02 Dec 2003 14:13:34 MDT."
	<16332.62062.305234.903783@montanaro.dyndns.org> 
Message-ID: <200312040342.hB43g6Co009702@maxim.off.ekorp.com>


>>> Skip Montanaro wrote
>     Anthony> In the meantime, if someone wants to take on the "upgrade to
>     Anthony> autoconf 2.59" task, I'd appreciate it very much.
> 
> Do you mean 2.58?  I looked here:
> 
>     http://ftp.gnu.org/gnu/autoconf/
> 
> and 2.58 appears to be the latest release.

There's a 2.59 release announced on the autoconf list 
http://mail.gnu.org/archive/html/autoconf-maintainers/2003-11/msg00001.html 
but it's never appeared on the GNU ftp servers. Based on this bit of
uncertainty, as well as a sheer lack of round tuits on my part, I've
held off on the autoconf upgrade. 2.3.4 can have it.

Anthony

From ajsiegel at optonline.net  Wed Dec  3 23:05:55 2003
From: ajsiegel at optonline.net (Arthur)
Date: Wed Dec  3 23:02:11 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard Libary
Message-ID: <0HPC00A6FR7INO@mta2.srv.hcvlny.cv.net>

Greg writes:

>The result of all this is that I don't think I've *ever* used
>copy() or deepcopy(). Whenever I've need to copy something I've
>always used something type-specific such as list[:] or dict.copy(),
>or written my own copying function.

What is fundamentally wrong with copy(list) as an alternative to list[:]?  

Guess I am feeling invincible.  Worst case, I can be told that I need to
read a tutorial ;),

Art


From alexis.layton at post.harvard.edu  Wed Dec  3 23:05:41 2003
From: alexis.layton at post.harvard.edu (Alexis Layton)
Date: Wed Dec  3 23:05:44 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <20199DBD-260F-11D8-A36A-000A957D895E@post.harvard.edu>

Greetings illustrious Python developers!

I've been reading the python-dev archives for years now, but the
current discussion of lambda alternatives has finally got me to post.

It seems there is a tradeoff between more general (fully functional,
if you will, pardon the groans) lambda-type constructs, and the
simpler "90%" constructs that have been being batted around here.

One simple style is to "infer" the lambda using positional
arguments, for example:

	(&1 + &2.foo())

(this would use "&" as a unary operator which I believe is available?)
Or one could use "$", which begins to look a little like shell
programming:

     ($1['fred'].x)

(Or \ if you want to risk being Haskelly....)

This style of construct implies the lambda or function without really
saying it, which may be too magical for Python.

The other main problem with constructs like this are that some number
of current uses of lambda need to ignore arguments; this happens
particularly for GUI callbacks, I think.  Also, the syntax doesn't 
really
support no-argument lambdas.

On the other side is the spelled-out form.  Perhaps def can really
be re-used:

	def(arglist: expr)
	
as in

	def(a, b: a + b.foo())  #equivalent to first example
	
	def(a: a['fred'].x) #equivalent to second example

of course, this is very close to lambda in syntax, although it does have
the advantage of a close paren; it can clearly handle all forms of 
arguments
(keyword, default values (although why would you need...), *, **, etc.)

The hitch there is that the syntax looks different from regular def.
That may be ok, considering the beasts are different....

I'd personally hate to see something equivalent to lambda completely
disappear from the language.

----
Alexis Layton
alexis dot layton at post dot harvard dot edu


From tdelaney at avaya.com  Wed Dec  3 23:32:25 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec  3 23:32:31 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard
	Libary
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEF5FC06@au3010avexu1.global.avaya.com>

> From: Arthur
> 
> What is fundamentally wrong with copy(list) as an alternative 
> to list[:]?  

Well, it's slower ... it involves a Python function call and python code, whereas list[:] is all C.

Of course, I didn't say it's a *lot* slower ... the actual copying is likely to take longer.

However, I have to agree that I've almost never used copy or deepcopy.

Tim Delaney

From guido at python.org  Wed Dec  3 23:50:04 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec  3 23:50:16 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard
	Libary
In-Reply-To: Your message of "Thu, 04 Dec 2003 15:32:25 +1100."
	<338366A6D2E2CA4C9DAEAE652E12A1DEF5FC06@au3010avexu1.global.avaya.com> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF5FC06@au3010avexu1.global.avaya.com>
Message-ID: <200312040450.hB44o4x05333@c-24-5-183-134.client.comcast.net>

Can we stop this thread on copy()?  It's one of Arthur's pet subjects
and you're not going to convince him, so it's pointless to argue.

Arthur, why don't you blog on this so the world at large can read your
views, rather than boring the developers here?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec  4 00:05:18 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec  4 00:05:34 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Thu, 04 Dec 2003 12:46:36 +1100."
	<bqm3m4$aov$1@sea.gmane.org> 
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop>
	<200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net> 
	<bqm3m4$aov$1@sea.gmane.org> 
Message-ID: <200312040505.hB455IJ05377@c-24-5-183-134.client.comcast.net>

> If one accepts that int and long have the same operations but offer
> different implementations, then the obvious solution is: an abstract
> base class (Int) and two sub-classes (Short and Long?).

Right, this seems to be one of the preferred solutions.

> Of course, this obvious solution has a problem: a programmer can't
> easily sub-class Int.  They must choose one a sub-class from Short
> or Long (unless they want to provide a full implementation
> themselves) and that's probably not a decision they want to make.

(It has been pointed out that subclassing int isn't very useful, so
maybe this is a moot point.  Does anybody have a real use case?)

> Perhaps, because of this, the GOF "Bridge Pattern" might be suitable
> here.  (This pattern can be framed as: "adapt multiple
> implementations to an interface using delegation" -- which, after
> all, is pretty much what the vtable in previous solution gives you.)
> 
> If the existence of Short and Long is an implementation detail best
> hidden from the python programmer, then a Bridge Pattern
> implementation has a lot going for it.

Hmm...  I'm not very familiar with the Bridge pattern (and my GoF book
is on one of 65 boxes still in the garage waiting until we move into a
larger house :-).  Can you give a little more detail about how this
would be done?  Is it something that a user wishing to subclass Int
should do, or something that the Int class (or its subclasses) should
provide?

> Use of the Bridge pattern might even allow for three different
> implementations: PreDefined, Short, Long ... where PreDefined is one
> of the numbers between 0 and 256 which I'm given to understand are
> pre-allocated by the runtime environment.

I don't think so -- the beauty is that (most of) the implementation
doesn't know that certain int values have eternal life; their
representation is exactly the same as that of regular ints.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Thu Dec  4 00:26:07 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Dec  4 00:26:11 2003
Subject: [Python-Dev] RE: test_unicode_file failing on trunk, Win98SE
In-Reply-To: <045401c3b9eb$e6d70450$2c00a8c0@eden>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEODHIAB.tim.one@comcast.net>

[Mark Hammond]
> My apologies - I have fixed it (I hope! No Win98 SE here for me to
> test on)

Not yet, but we may be getting closer <wink>.

First problem:  test_support.py did

               TESTFN_UNICODE_UNENCODABLE = None

on Win98SE, but test_unicode_filename.py tries to import

               TESTFN_UNICODE_UNENCODEABLE

That's a clear spelling mistake in the former, so I'll fix that and check it
in.

Second problem:  Then the test gets one failure:

======================================================================
ERROR: test_single_files (__main__.TestUnicodeFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 142, in test_single_files
    self._test_single(TESTFN_UNICODE)
  File "../lib/test/test_unicode_file.py", line 116, in _test_single
    self._do_single(filename)
  File "../lib/test/test_unicode_file.py", line 32, in _do_single
    os.utime(filename, None)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-7:
ordinal not in range(128)


Note that the test doesn't skip itself on 98SE, because the

try:
    TESTFN_ENCODED = TESTFN_UNICODE.encode(TESTFN_ENCODING)
except (UnicodeError, TypeError):
    # Either the file system encoding is None, or the file name
    # cannot be encoded in the file system encoding.
    raise TestSkipped("No Unicode filesystem semantics on this platform.")

doesn't trigger:

>>> import sys
>>> sys.getfilesystemencoding()
'mbcs'
>>> u'@test-\xe0\xf2'.encode('mbcs')
'@test-\xe0\xf2'
>>>


>> Don't know about 2.3 maint, & can't test that today:

> The fix to posixmodule was checked into both head and 2.3-maint, but
> the test suite changes were only made to head - for exactly this
> reason <wink>. Thus, 2.3 should be fine.

Yes, 2.3 maint *is* fine on 98SE.  All the above was about the trunk.


From mike.thompson at day8.com.au  Thu Dec  4 05:53:55 2003
From: mike.thompson at day8.com.au (Mike Thompson)
Date: Thu Dec  4 05:54:06 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop><200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
	<bqm3m4$aov$1@sea.gmane.org>
	<200312040505.hB455IJ05377@c-24-5-183-134.client.comcast.net>
Message-ID: <bqn3o7$s2j$1@sea.gmane.org>

[Mike Thompson]
> > Of course, this obvious solution has a problem: a programmer can't
> > easily sub-class Int.  They must choose one a sub-class from Short
> > or Long (unless they want to provide a full implementation
> > themselves) and that's probably not a decision they want to make.
>

[Guido van Rossum]
> (It has been pointed out that subclassing int isn't very useful, so
> maybe this is a moot point.  Does anybody have a real use case?)

I don't.   It just rubs my intuition up the wrong way for it not to be
subclassable, but that'll be because I'm not a fully reformed OO Bigot  ;-)


[Mike Thompson]
> > Perhaps, because of this, the GOF "Bridge Pattern" might be suitable
> > here.  (This pattern can be framed as: "adapt multiple
> > implementations to an interface using delegation" -- which, after
> > all, is pretty much what the vtable in previous solution gives you.)
> >
> > If the existence of Short and Long is an implementation detail best
> > hidden from the python programmer, then a Bridge Pattern
> > implementation has a lot going for it.
>


[Guido van Rossum]
> Hmm...  I'm not very familiar with the Bridge pattern (and my GoF book
> is on one of 65 boxes still in the garage waiting until we move into a
> larger house :-).  Can you give a little more detail about how this
> would be done?

To me, Bridge Pattern is most useful when either:
     1.   There is a need to sub-class the abstraction (Int) other than for
            implementational purposes.

      2.   The implementation of the abstraction (Int) needs to change at
runtime.

So, given you doubt the need for 1. and that 2. was never needed because of
Int-immutability, I should withdraw my suggestion in favour of the more simple
abstract-Int-base-with-Long-And-Short-Iimplementation-in-sub-classes approach
previously mentioned.

BTW, a summary of most patterns can be found on-line at
http://patterndigest.com/

[Guido van Rossum]
> Is it something that a user wishing to subclass Int
> should do, or something that the Int class (or its subclasses) should
> provide?

Its something that the writer of the Int class would use iff they wanted to
ensure that programmers were oblivious to the  two or three possible
implementations of Int AND one of the two conditions I mentioned above (needed
subclassability or runtime-change in implementation) held true.

[Mike Thompson]
> > Use of the Bridge pattern might even allow for three different
> > implementations: PreDefined, Short, Long ... where PreDefined is one
> > of the numbers between 0 and 256 which I'm given to understand are
> > pre-allocated by the runtime environment.
>


[Guido van Rossum]
> I don't think so -- the beauty is that (most of) the implementation
> doesn't know that certain int values have eternal life; their
> representation is exactly the same as that of regular ints.
>

That beauty could remain.

At the risk of lapsing completely into pattern vocabulary ...  an
AbstractFactory would create Ints and bind them to a subsequently hidden
implementation.  When the int involved was tiny (-1 to 256?) this
AbstractFactory would use one of the pre-allocated, shared (see FlyWeight)
implementations. Outside of this range distinct Short, Long or DamnHuge
implementations would be created.

--
Mike





From pedronis at bluewin.ch  Thu Dec  4 08:09:43 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Thu Dec  4 08:06:42 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <200312040322.hB43Mn814298@oma.cosc.canterbury.ac.nz>
References: <Pine.LNX.4.44.0312031055550.1066-100000@tane.cfa.harvard.edu>
Message-ID: <5.2.1.1.0.20031204140010.0288fff8@pop.bluewin.ch>

At 16:22 04.12.2003 +1300, Greg Ewing wrote:
>Greg Ball <gball@cfa.harvard.edu>:
>
> > (x[1] for x) == lambda x:x[1]
> > (x.score for x) == lambda x: x.score
> > (x+y for x,y) == lambda x,y: x+y
> > (len(x) for x) == lambda x,len=len: len(x)  # roughly equivalent
>
>Interesting idea, but it reads a bit strangely.
>
>Maybe this would work better with a different keyword...
>
>   x[1] given x
>   x.score given x
>   x+y given x,y
>   len(x) given x
>
>   stuff = groupby((x for x in seq), key = (x.score given x))

on the plus side it nicely parallels generator expressions and the keyword 
makes lookup in the docs easy. On the minus side is maybe verbose.

On the other hand non-keyword notations risk to be too cryptic: some 
candidates:

stuff = groupby((x for x in seq), key = (x.score\x))
# this would require the lexer to distinguish \ as line continuation vs 
this usage,
# it would probably still be read as "x.score given x"

stuff = groupby((x for x in seq), key = (x-->x.score)) # --> would be a 
single token






From anthony at interlink.com.au  Thu Dec  4 08:41:21 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Dec  4 08:41:36 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: Message from Anthony Baxter <anthony@interlink.com.au> 
	of "Tue, 02 Dec 2003 01:22:36 +1100."
Message-ID: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>

In the absence of any screams of "no, dear god, please don't", I 
am declaring the start of the 2.3.3 release process. I'll start 
cutting the rc1 tomorrow morning (in around 10 hours time). 

So, from now, until we get 2.3.3 final out the door (in about
2 weeks), please do NOT check in on the release23-maint branch
unless your name is Anthony, Thomas, or Fred. If you see something
that absolutely MUST be fixed, please check with me first.

Remember - suprises on the release23-maint branch make for an
unhappy release manager, and after the last couple of weeks, I'd
_really_ like a nice, simple, smooth release.

Thanks for your co-operation,
Anthony

From fdrake at acm.org  Thu Dec  4 08:54:26 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec  4 08:54:32 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
References: <anthony@interlink.com.au>
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
Message-ID: <16335.15506.191582.612851@grendel.fdrake.net>


Anthony Baxter writes:
 > So, from now, until we get 2.3.3 final out the door (in about
 > 2 weeks), please do NOT check in on the release23-maint branch
 > unless your name is Anthony, Thomas, or Fred. If you see something
 > that absolutely MUST be fixed, please check with me first.

The release23-maint branch's "logging" package still uses apply(),
which generates a PendingDeprecationWarning.  I'd like to fix that (by
removing the use of apply()).

If that's ok, I can fix that now on the release23-maint branch and the
trunk this morning.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From mcherm at mcherm.com  Thu Dec  4 08:56:17 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Dec  4 08:56:23 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
Message-ID: <1070546177.3fcf3d01e6f52@mcherm.com>

[Guido writes:]
> The implementation uses the type as the vtable of a C++ class,
> and the functions in the vtable "know" the representation, which is
> very different for short and long ints.  This is all nice and
> efficient, except that the identity of the vtable is returned as the
> type of the object, so two different vtables means two different
> types.
     ...
[and clipping overaggressively:]
> The alternative ... embed a flag ... downslide should be obvious 
> ... extra test+branch ... also extra space ... no spare bits.

I have an idea, but don't know enough about the type mechanisms to
know whether this is possible. So I'll throw it out there and let
you shoot it down.

To my mind, the *ideal* is that there's a single type, probably
called "int" or "integer" which can take on an arbitrary size and
precision BUT which, if it happens to be small enough to fit in
a C int will take up less memory and run faster. The fact that
it's stored differently should be an implementation detail not
visible to the user.

But which user? I would say that someone using the C api's to
write extension modules probably DOES care about the particular
representation of ints vs longs, and it's not unreasonable to
expect them to deal with it. But for someone programming from
Python it should "just work".

If you agree with this premise, then it suggests a different
approach. Use different implementations in C, but have type(x)
in Python lie. type(x) would always return the type object which
is now known as "long". That type object would be smart enough
to realize that any time the object it is dealing with had a
value in the range [-MAXINT-1, MAXINT] that it was ACTUALLY
using the layout now known as "int". This would require some
clever tricks in the implementation of the "long" type object,
but that's not where we care about performance. Of course, it's
possible that there are no tricks clever enough to handle this
-- I certainly don't know how to do it, but I wouldn't be TOO
surprised if someone else here did.

-- Michael Chermside

PS: If this approach were used, the sensible type names would
be something like "shortint" (visible only from C) and "int"
(today's long).


From mcherm at mcherm.com  Thu Dec  4 09:08:29 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Dec  4 09:08:33 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
Message-ID: <1070546909.3fcf3fdd143e5@mcherm.com>

Guido writes:
> Hmm...  I'm not very familiar with the Bridge pattern (and my GoF book
> is on one of 65 boxes still in the garage waiting until we move into a
> larger house :-).  Can you give a little more detail about how this
> would be done?

If I understand it properly (and I DID check my copy of GOF which isn't
packed <wink>), it may be better recognized by its alternative name:
"Handle/Body". The basic idea is to add a level of indirection... the
GOF name the "front" object the "Abstraction" and the "back" object
the "Implementor".

And it seems like a poor choice for implementing ints, which by
definition should be designed for the minimum possible overhead, unless
that "front" object can be made VERY lightweight. (Hmm... to avoid
cache effects we might want to store the "abstraction" adjacent in
memory to the "implementor". And if we did THAT we might even be able
to dispense with even a pointer in the "abstraction". But now I've
talked myself out of the bridge pattern into something more like just
having two separate types... really 3 now, since there's a third
layout needed to handle user-designed int subclasses. I still don't
like it.)

-- Michael Chermside



From ark-mlist at att.net  Thu Dec  4 09:12:17 2003
From: ark-mlist at att.net (Andrew Koenig)
Date: Thu Dec  4 09:12:09 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312040505.hB455IJ05377@c-24-5-183-134.client.comcast.net>
Message-ID: <002b01c3ba70$9fce5970$6402a8c0@arkdesktop>

> (It has been pointed out that subclassing int isn't very useful, so
> maybe this is a moot point.  Does anybody have a real use case?)

isinstance(x, int)

That doesn't require an inheritance relationship between the integral types,
but it does require that long inherit from int.   So int could be a base
class with long and short inheriting from it, or long could inherit from
int.


From ark-mlist at att.net  Thu Dec  4 09:36:40 2003
From: ark-mlist at att.net (Andrew Koenig)
Date: Thu Dec  4 09:36:35 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <bqn3o7$s2j$1@sea.gmane.org>
Message-ID: <003301c3ba74$0849c1d0$6402a8c0@arkdesktop>

> At the risk of lapsing completely into pattern vocabulary ...  an
> AbstractFactory would create Ints and bind them to a subsequently hidden
> implementation.  When the int involved was tiny (-1 to 256?) this
> AbstractFactory would use one of the pre-allocated, shared (see FlyWeight)
> implementations. Outside of this range distinct Short, Long or DamnHuge
> implementations would be created.

Or, since we're talking implementation, we could use the following hack,
which I borrowed from Standard ML of New Jersey:

There is just one type, namely int.

An int is a 31-bit integer, INCLUDING sign.  One extra bit indicates whether
the integer is really a number or, alternatively, a pointer to the rest of
the representation.

Now, you may object that implementing this strategy in C will require lots
of shifting and masking.  I would have thought so, too.  However, every C
implementation of which I am aware puts all but the most trivial data
structures on boundaries of two or more bytes.  This fact frees the
LOW-order bit of the integer to indicate whether it is a pointer.

So here's the strategy:  If the low-order bit of an integer is *off*, it's
really a pointer to the rest of the implementation.  If the low-order bit is
*on*, then it represents an integral value that can be obtained by doing a
one-bit arithmetic right shift.

Yes, it's sleazy.  But I imagine it would be much faster than using
inheritance.


From fincher.8 at osu.edu  Thu Dec  4 10:38:25 2003
From: fincher.8 at osu.edu (Jeremy Fincher)
Date: Thu Dec  4 09:41:07 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <16335.15506.191582.612851@grendel.fdrake.net>
References: <anthony@interlink.com.au>
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<16335.15506.191582.612851@grendel.fdrake.net>
Message-ID: <200312041038.26937.fincher.8@osu.edu>

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

On Thursday 04 December 2003 08:54 am, Fred L. Drake, Jr. wrote:
> Anthony Baxter writes:
>  > So, from now, until we get 2.3.3 final out the door (in about
>  > 2 weeks), please do NOT check in on the release23-maint branch
>  > unless your name is Anthony, Thomas, or Fred. If you see something
>  > that absolutely MUST be fixed, please check with me first.
>
> The release23-maint branch's "logging" package still uses apply(),
> which generates a PendingDeprecationWarning.  I'd like to fix that (by
> removing the use of apply()).

I thought the logging module was supposed to remain 1.5.2-compatible.  At 
least, that's the reason I saw the last time someone wanted to remove those 
apply calls.

Jeremy
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.2.3 (GNU/Linux)

iD8DBQE/z1TxqkDiu+Bs+JIRAuAUAJ4601VMuG59mew2nKJvtH6w1w0xewCfY2ly
KAy/iM77SCqEawJODJAfCZ8=
=C9V5
-----END PGP SIGNATURE-----

From fdrake at acm.org  Thu Dec  4 09:51:46 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec  4 09:52:03 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <200312041038.26937.fincher.8@osu.edu>
References: <anthony@interlink.com.au>
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<16335.15506.191582.612851@grendel.fdrake.net>
	<200312041038.26937.fincher.8@osu.edu>
Message-ID: <16335.18946.557564.95591@grendel.fdrake.net>


Jeremy Fincher writes:
 > I thought the logging module was supposed to remain 1.5.2-compatible.  At 
 > least, that's the reason I saw the last time someone wanted to remove those 
 > apply calls.

That sounds vaguely familiar.  If we really want the copy distributed
with Python to be compatible with Python's that old, we can use a
different approach.  Unfortunately, we can't simply add something like:

if sys.version_info >= (2, 3):
    def apply(func, args, kw=None):
        if kw is None:
            return func(*args)
        else:
            return func(*args, **kw)

This isn't permissible since it would cause a SyntaxError for Python
1.5.2; we'd have to enlist the "exec" statement to make this work.

Do we still really want to support Python 1.5.2?  That just seems
quite extreme to me; even PyXML is no longer maintained for Python
1.5.2.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From mwh at python.net  Thu Dec  4 09:52:24 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec  4 09:53:04 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
Message-ID: <78CB3A69-2669-11D8-9361-0003931DF95C@python.net>

It seems to me that any time there are more than two CObjects around we 
have a good chance of causing mischief:

 >>> import cStringIO, _curses
 >>> cStringIO.cStringIO_CAPI = _curses._C_API
 >>> import cPickle
 >>> cPickle.dumps(range(10))
Illegal instruction

(if you're trying this at home, starting python with -E might be 
necessary to make sure cPickle hasn't been imported already).

Question 1: do we care?  I think we do: "no core dumps" is a pretty 
strong Pythonic principle, even though this is certainly a "doctor it 
hurts when I do this" situation.  OTOH, Armin's gc.get_referrers hack 
was deemed allowable.

It seems to me that it would be more sensible to have a dict mapping 
names (maybe just module names...) to cobjects in the interpreter 
state.  I guess there might be a way of getting ahold of the dict with 
the gc module (though I can't think of one off hand).  This wouldn't be 
a difficult change, and if the CObjects are left in the module dicts, 
it shouldn't even do that much damage to binary compatibility.

Thoughts?

Cheers,
mwh
(who also notices that the cobject section in Doc/ext/ could do with 
being updated to use PyCObject_Import...)


From ncoghlan at iinet.net.au  Thu Dec  4 09:50:40 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Dec  4 09:55:38 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312031726.hB3HQ6504171@c-24-5-183-134.client.comcast.net>
References: <011201c3b9b3$f42b7880$6402a8c0@arkdesktop>	<200312031610.hB3GA1204029@c-24-5-183-134.client.comcast.net>
	<200312031215.17933.fincher.8@osu.edu>
	<200312031726.hB3HQ6504171@c-24-5-183-134.client.comcast.net>
Message-ID: <3FCF49C0.4010702@iinet.net.au>

[Guido]
 >> I guess that means
 >> I have to work harder and make the single int type support both
 >> representations.  I'm sure it can be done.

Guido van Rossum wrote:
>>>So the question is, does long have operations that int doesn't have?
>>>And if so, why can't those operations be added to int?  And if there's
>>>a reason, is it good enough?
>>
>>Taking into account their difference in representation, a long can support 
>>1<<32, but an int can't.
> 
> We're talking about a hypothetical int here where that operation
> returns 4294967296L.  (Not so hypothetical, it's implemented in Python
> 2.4 in CVS.)

The 'performance hack' point of view I was trying to suggest was along 
the lines of:

"Python integers are capable of storing values of arbitrary magnitude, 
subject only to the memory capacity of the machine. As a matter of 
performance, Python will use native C integers (and native arithmetic) 
when the stored value is small enough to fit."

That is, I was agreeing with Guido's first point above.


From barry at python.org  Thu Dec  4 10:02:54 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec  4 10:03:01 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was
	Re: 2.3.3 cycle)
In-Reply-To: <16335.18946.557564.95591@grendel.fdrake.net>
References: <anthony@interlink.com.au>
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<16335.15506.191582.612851@grendel.fdrake.net>
	<200312041038.26937.fincher.8@osu.edu>
	<16335.18946.557564.95591@grendel.fdrake.net>
Message-ID: <1070550162.27289.5.camel@anthem>

On Thu, 2003-12-04 at 09:51, Fred L. Drake, Jr. wrote:
> Jeremy Fincher writes:
>  > I thought the logging module was supposed to remain 1.5.2-compatible.

So says PEP 291.

> That sounds vaguely familiar.  If we really want the copy distributed
> with Python to be compatible with Python's that old, we can use a
> different approach.  Unfortunately, we can't simply add something like:
> 
> if sys.version_info >= (2, 3):
>     def apply(func, args, kw=None):
>         if kw is None:
>             return func(*args)
>         else:
>             return func(*args, **kw)
> 
> This isn't permissible since it would cause a SyntaxError for Python
> 1.5.2; we'd have to enlist the "exec" statement to make this work.

The email package has a fairly elaborate mechanism to support
compatibility with older Pythons.  It segregates syntactically
incompatible stuff into separate modules _compat21.py and _compat22.py
and then imports the appropriate one based on the Python version.  Yes,
it's a hack.

> Do we still really want to support Python 1.5.2?  That just seems
> quite extreme to me; even PyXML is no longer maintained for Python
> 1.5.2.

I think I'd like to elaborate PEP 291 a bit.  It seems perfectly
reasonable to say that Python 2.3's email package must remain compatible
with Python 2.1, but that Python 2.4's email package needs no such
backwards compatibility.  Now, the email package has its own version
number so in that sense, I could say that email 2.x is Python 2.1
compatible but email 3.x requires Python 2.3 or 2.4.  I'm not sure how
well that works with the other packages listed in PEP 291, but given
that Python has multiple branches, I don't think we have to require the
code on the trunk to keep backwards compatibility with ancient Pythons.

-Barry



From barry at python.org  Thu Dec  4 10:14:50 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec  4 10:14:55 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was
	Re: 2.3.3 cycle)
In-Reply-To: <1070550162.27289.5.camel@anthem>
References: <anthony@interlink.com.au>
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<16335.15506.191582.612851@grendel.fdrake.net>
	<200312041038.26937.fincher.8@osu.edu>
	<16335.18946.557564.95591@grendel.fdrake.net>
	<1070550162.27289.5.camel@anthem>
Message-ID: <1070550889.27289.8.camel@anthem>

On Thu, 2003-12-04 at 10:02, Barry Warsaw wrote:

> I think I'd like to elaborate PEP 291 a bit.

Which I've just done.  It's a little awkward to get all the information
in there, but see if you like my 5 minute update.

http://www.python.org/peps/pep-0291.html

-Barry



From pje at telecommunity.com  Thu Dec  4 10:38:21 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec  4 10:36:23 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <003301c3ba74$0849c1d0$6402a8c0@arkdesktop>
References: <bqn3o7$s2j$1@sea.gmane.org>
Message-ID: <5.1.0.14.0.20031204103428.034866f0@mail.telecommunity.com>

At 09:36 AM 12/4/03 -0500, Andrew Koenig wrote:
>So here's the strategy:  If the low-order bit of an integer is *off*, it's
>really a pointer to the rest of the implementation.  If the low-order bit is
>*on*, then it represents an integral value that can be obtained by doing a
>one-bit arithmetic right shift.
>
>Yes, it's sleazy.  But I imagine it would be much faster than using
>inheritance.

I imagine it wouldn't, because it'd add an extra test to not only every 
Py_INCREF and Py_DECREF, but every PyObject_something call.


From mwh at python.net  Thu Dec  4 10:45:11 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec  4 10:45:17 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <5.1.0.14.0.20031204103428.034866f0@mail.telecommunity.com>
	(Phillip J. Eby's message of "Thu, 04 Dec 2003 10:38:21 -0500")
References: <bqn3o7$s2j$1@sea.gmane.org>
	<5.1.0.14.0.20031204103428.034866f0@mail.telecommunity.com>
Message-ID: <2moeuowprs.fsf@starship.python.net>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 09:36 AM 12/4/03 -0500, Andrew Koenig wrote:
>>So here's the strategy:  If the low-order bit of an integer is *off*, it's
>>really a pointer to the rest of the implementation.  If the low-order bit is
>>*on*, then it represents an integral value that can be obtained by doing a
>>one-bit arithmetic right shift.
>>
>>Yes, it's sleazy.  But I imagine it would be much faster than using
>>inheritance.
>
> I imagine it wouldn't, because it'd add an extra test to not only
> every Py_INCREF and Py_DECREF, but every PyObject_something call.

It wouldn't have to be that bad if you put the pointer/int thingy in
the ob_ival slot.

Cheers,
mwh

-- 
  The only problem with Microsoft is they just have no taste.
              -- Steve Jobs, (From _Triumph of the Nerds_ PBS special)
                                and quoted by Aahz on comp.lang.python

From fdrake at acm.org  Thu Dec  4 11:30:19 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec  4 11:30:30 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <003301c3ba74$0849c1d0$6402a8c0@arkdesktop>
References: <bqn3o7$s2j$1@sea.gmane.org>
	<003301c3ba74$0849c1d0$6402a8c0@arkdesktop>
Message-ID: <16335.24859.801823.731366@grendel.fdrake.net>


Andrew Koenig writes:
 > An int is a 31-bit integer, INCLUDING sign.  One extra bit indicates whether
 > the integer is really a number or, alternatively, a pointer to the rest of
 > the representation.
 > 
 > Now, you may object that implementing this strategy in C will require lots
 > of shifting and masking.  I would have thought so, too.  However, every C
 > implementation of which I am aware puts all but the most trivial data
 > structures on boundaries of two or more bytes.  This fact frees the
 > LOW-order bit of the integer to indicate whether it is a pointer.

I recently mentioned this approach to coaxing an extra bit from an
existing structure to Tim, and he pointed out (rightly!) that this
only works for machines with byte-addressable memories.  On
word-addressable memories, there's no chance to coax the extra bit
from a pointer at all, and Python still runs on at least one such
platform (some Cray thing, IIRC).

Otherwise I like the approach.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From tim.one at comcast.net  Thu Dec  4 11:44:09 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Dec  4 11:44:11 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <2moeuowprs.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEALHJAB.tim.one@comcast.net>

[Andrew Koenig]
>>> So here's the strategy:  If the low-order bit of an integer is
>>> *off*, it's really a pointer to the rest of the implementation.  If
>>> the low-order bit is *on*, then it represents an integral value
>>> that can be obtained by doing a one-bit arithmetic right shift.
>>>
>>> Yes, it's sleazy.  But I imagine it would be much faster than using
>>> inheritance.

[Phillip J. Eby]
>> I imagine it wouldn't, because it'd add an extra test to not only
>> every Py_INCREF and Py_DECREF, but every PyObject_something call.

[Michael Hudson]
> It wouldn't have to be that bad if you put the pointer/int thingy in
> the ob_ival slot.

Not all HW Python runs on is byte-addressed, so the base idea that "the last
bit" of a pointer-to-non-trivial-structure is always 0 doesn't get off the
ground.  C doesn't have an arithmetic right-shift operator either, but that
one is easier to worm around (pyport.h already has a
Py_ARITHMETIC_RIGHT_SHIFT macro to "do the right thing").

OTOH, the only current Python platform I know of that's word-addressed is
the Cray T3E, which also happens to be the only one I know of where C's
signed ">>" zero-fills (and is also the only one I know of that has no
16-bit integral type -- the T3E porters had an interesting <wink> time).


From kbk at shore.net  Thu Dec  4 12:35:05 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Dec  4 12:35:14 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com> (Anthony
	Baxter's message of "Fri, 05 Dec 2003 00:41:21 +1100")
References: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
Message-ID: <m3d6b41o6u.fsf@float.localdomain>

Anthony Baxter <anthony@interlink.com.au> writes:

> So, from now, until we get 2.3.3 final out the door (in about
> 2 weeks), please do NOT check in on the release23-maint branch
> unless your name is Anthony, Thomas, or Fred. If you see something
> that absolutely MUST be fixed, please check with me first.

idlelib:

idlever.py  needs to be updated to 1.0.2

The release date needs to be set in NEWS.txt

I can do this if you like, or you can.  Let me know.

-- 
KBK

From gjc at inescporto.pt  Thu Dec  4 12:51:06 2003
From: gjc at inescporto.pt (Gustavo J. A. M. Carneiro)
Date: Thu Dec  4 12:51:48 2003
Subject: [Python-Dev] LC_NUMERIC patch updated
Message-ID: <1070560266.12807.25.camel@spectrum.inescn.pt>

	It is SF patch [ 774665 ] making Python LC_NUMERIC agnostic.

	With this new patch I have also removed the code hacks that keep
LC_NUMERIC always set to 'C'.  I hope there aren't serious objections,
because this is actually the main point of adding locale-independent
floating point conversion routines.

	Anything else I can do?

	Regards.

-- 
Gustavo Jo?o Alves Marques Carneiro
<gjc@inescporto.pt> <gustavo@users.sourceforge.net>



From gisle at ActiveState.com  Thu Dec  4 13:40:16 2003
From: gisle at ActiveState.com (Gisle Aas)
Date: Thu Dec  4 13:40:29 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <003301c3ba74$0849c1d0$6402a8c0@arkdesktop>
References: <003301c3ba74$0849c1d0$6402a8c0@arkdesktop>
Message-ID: <lr8ylsifzj.fsf@caliper.activestate.com>

"Andrew Koenig" <ark-mlist@att.net> writes:

> An int is a 31-bit integer, INCLUDING sign.  One extra bit indicates whether
> the integer is really a number or, alternatively, a pointer to the rest of
> the representation.

This is exactly how it's done in Ruby as well.  Search for FIXNUM in
http://www.ruby-lang.org/cgi-bin/cvsweb.cgi/ruby/ruby.h?rev=1.95&content-type=text/x-cvsweb-markup

Regards,
Gisle

From Juli at joelaw.cc  Thu Dec  4 14:33:23 2003
From: Juli at joelaw.cc (Juli Longacre)
Date: Thu Dec  4 14:33:34 2003
Subject: [Python-Dev] PEP draft - Guidelines to processing patches
Message-ID: <008e01c3ba9d$7c1d9090$07fea8c0@joesonydesktop>

I am trying to find a Michael Bartl, son of Theresa Irvin and Richard
Bajenske for a legal matter.  If you are that person, please contact me
immediately.  Thank you.


Juli Longacre
Paralegal to Joe B. Stephens
The Stephens Law Firm
The Lyric Center
440 Louisiana, Suite 2000
Houston, Texas  77002
713-224-0000
713-224-0055 fax
juli@joelaw.cc
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031204/9dbea920/attachment.html
From theller at python.net  Thu Dec  4 15:23:51 2003
From: theller at python.net (Thomas Heller)
Date: Thu Dec  4 15:24:13 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <m3d6b41o6u.fsf@float.localdomain> (Kurt B. Kaiser's message of
	"Thu, 04 Dec 2003 12:35:05 -0500")
References: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<m3d6b41o6u.fsf@float.localdomain>
Message-ID: <brqoz608.fsf@python.net>

kbk@shore.net (Kurt B. Kaiser) writes:

> Anthony Baxter <anthony@interlink.com.au> writes:
>
>> So, from now, until we get 2.3.3 final out the door (in about
>> 2 weeks), please do NOT check in on the release23-maint branch
>> unless your name is Anthony, Thomas, or Fred. If you see something
>> that absolutely MUST be fixed, please check with me first.
>
> idlelib:
>
> idlever.py  needs to be updated to 1.0.2
>
> The release date needs to be set in NEWS.txt
>
> I can do this if you like, or you can.  Let me know.

I'll do it.

Can you update PEP 101 and 102 with this information, please?

Thanks, Thomas


From guido at python.org  Thu Dec  4 15:35:17 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec  4 15:35:28 2003
Subject: [Python-Dev] Re: Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Thu, 04 Dec 2003 09:36:40 EST."
	<003301c3ba74$0849c1d0$6402a8c0@arkdesktop> 
References: <003301c3ba74$0849c1d0$6402a8c0@arkdesktop> 
Message-ID: <200312042035.hB4KZHn06585@c-24-5-183-134.client.comcast.net>

> Or, since we're talking implementation, we could use the following hack,
> which I borrowed from Standard ML of New Jersey:
[...]

This is a well-known scheme.  We used it in ABC 20 years ago.  My
experience with it was negative though: there were too many core dumps
because some code was dereferencing a pointer without checking the low
bit.

So I'd rather not try this.  I also somehow doubt that the speed gains
are that significant.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec  4 15:40:08 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec  4 15:40:17 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Thu, 04 Dec 2003 05:56:17 PST."
	<1070546177.3fcf3d01e6f52@mcherm.com> 
References: <1070546177.3fcf3d01e6f52@mcherm.com> 
Message-ID: <200312042040.hB4Ke8D06619@c-24-5-183-134.client.comcast.net>

> If you agree with this premise, then it suggests a different
> approach. Use different implementations in C, but have type(x) in
> Python lie.  type(x) would always return the type object which is
> now known as "long".

If this can be made to work, I like it.  __class__ should also be
hacked, and isinstance(); and who knows how many other places, but
probably not too many.

> That type object would be smart enough to realize that any time the
> object it is dealing with had a value in the range [-MAXINT-1,
> MAXINT] that it was ACTUALLY using the layout now known as "int".

This part doesn't make sense -- the value cannot be known unless the
layout is known.  But it shouldn't matter -- the long methods should
never be passed a short representation, because at the C level the
short implementation would be invoked automatically through the vtable
or type.

Of course, the long code should be careful to downcase any *results*
in the short range to the short representation before returning it.

> This would require some clever tricks in the implementation of the
> "long" type object, but that's not where we care about performance.
> Of course, it's possible that there are no tricks clever enough to
> handle this -- I certainly don't know how to do it, but I wouldn't
> be TOO surprised if someone else here did.

Someone ought to try to implement this on a branch or as a patch, to
see how feasible it is.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From python-kbutler at sabaydi.com  Thu Dec  4 16:27:31 2003
From: python-kbutler at sabaydi.com (Kevin J. Butler)
Date: Thu Dec  4 16:27:50 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard
	Library
In-Reply-To: <E1ARm1B-0002ru-PA@mail.python.org>
References: <E1ARm1B-0002ru-PA@mail.python.org>
Message-ID: <3FCFA6C3.90509@sabaydi.com>

From: Arthur <ajsiegel@optonline.net>

>What is fundamentally wrong with copy(list) as an alternative to list[:]?  
>  
>
Well, copy(list) doesn't contain an emoticon of two beady little eyes 
peeking out of a box, for one.  ;-)

The opposite question, "What is fundamentally wrong with list[:]..." has 
an easy answer:

It takes experience or explanation of slicing to know what it does.  
copy(list) is easy even for novices to understand.

When I know my code will be read by people unfamiliar with Python, I 
tend to annotate use of
slices something like:

    list[:] # copy the list

Python usually makes it easier to avoid sacrificing clarity on the altar 
of micro-optimization, but since the slice is the idiomatically correct 
way to spell copy(list) in Python, that's the way to write it.

kb


From kbk at shore.net  Thu Dec  4 16:52:32 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Dec  4 16:52:35 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <brqoz608.fsf@python.net> (Thomas Heller's message of "Thu, 04
	Dec 2003 21:23:51 +0100")
References: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<m3d6b41o6u.fsf@float.localdomain> <brqoz608.fsf@python.net>
Message-ID: <m3wu9cz1wf.fsf@float.localdomain>

Thomas Heller <theller@python.net> writes:

> Can you update PEP 101 and 102 with this information, please?

Yes, I'll update them to cover the IDLE details.

-- 
KBK

From martin at v.loewis.de  Thu Dec  4 14:37:07 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec  4 17:06:48 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <78CB3A69-2669-11D8-9361-0003931DF95C@python.net>
References: <78CB3A69-2669-11D8-9361-0003931DF95C@python.net>
Message-ID: <m3ptf4idcs.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> It seems to me that it would be more sensible to have a dict mapping
> names (maybe just module names...) to cobjects in the interpreter
> state.  I guess there might be a way of getting ahold of the dict with
> the gc module (though I can't think of one off hand).  This wouldn't
> be a difficult change, and if the CObjects are left in the module
> dicts, it shouldn't even do that much damage to binary compatibility.
> 
> Thoughts?

Good idea. Alternatively, we could make "APIs" a feature of modules:
PyModule_GetAPI, PyModule_SetAPI. We would then define

typedef struct {
	PyObject_HEAD
	PyObject *md_dict;
        PyObject *md_api;
} PyModuleObject;

We could restrict md_api to CObjects, which, means we would not need
to change module_traverse.

Regards,
Martin

From pje at telecommunity.com  Thu Dec  4 18:27:28 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec  4 18:30:17 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the
	Standard Library
In-Reply-To: <3FCFA6C3.90509@sabaydi.com>
References: <E1ARm1B-0002ru-PA@mail.python.org>
	<E1ARm1B-0002ru-PA@mail.python.org>
Message-ID: <5.1.1.6.0.20031204182545.02ae79c0@telecommunity.com>

At 02:27 PM 12/4/03 -0700, Kevin J. Butler wrote:

>The opposite question, "What is fundamentally wrong with list[:]..." has 
>an easy answer:
>
>It takes experience or explanation of slicing to know what it does.
>copy(list) is easy even for novices to understand.
>
>When I know my code will be read by people unfamiliar with Python, I tend 
>to annotate use of
>slices something like:
>
>    list[:] # copy the list
>
>Python usually makes it easier to avoid sacrificing clarity on the altar 
>of micro-optimization, but since the slice is the idiomatically correct 
>way to spell copy(list) in Python, that's the way to write it.

Well, dictionaries have a .copy() method; lists could also grow one.


From jjl at pobox.com  Thu Dec  4 18:36:05 2003
From: jjl at pobox.com (John J Lee)
Date: Thu Dec  4 18:36:12 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <002b01c3ba70$9fce5970$6402a8c0@arkdesktop>
References: <002b01c3ba70$9fce5970$6402a8c0@arkdesktop>
Message-ID: <Pine.LNX.4.58.0312042333300.437@alice>

On Thu, 4 Dec 2003, Andrew Koenig wrote:

> > (It has been pointed out that subclassing int isn't very useful, so
> > maybe this is a moot point.  Does anybody have a real use case?)
>
> isinstance(x, int)

If that's really the only use case, is the following not sufficient?

isinstance(x, types.IntegralTypes)


(where IntegralTypes == (int, long), analogously to types.StringTypes)


John

From greg at cosc.canterbury.ac.nz  Thu Dec  4 18:44:03 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec  4 18:44:11 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: <20199DBD-260F-11D8-A36A-000A957D895E@post.harvard.edu>
Message-ID: <200312042344.hB4Ni3K22308@oma.cosc.canterbury.ac.nz>

Alexis Layton <alexis.layton@post.harvard.edu>:

> One simple style is to "infer" the lambda using positional
> arguments, for example:
> 
> 	(&1 + &2.foo())
> 
> This style of construct implies the lambda or function without really
> saying it, which may be too magical for Python.

I think it's ambiguous, too, since it's not clear how far
the lambda extends. Is

  ((&1 + &2), (&1 - &2))

one lambda returning a tuple, or a tuple containing
two lambdas?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From nas-python at python.ca  Thu Dec  4 19:02:17 2003
From: nas-python at python.ca (Neil Schemenauer)
Date: Thu Dec  4 18:55:25 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312040254.hB42sLI14069@oma.cosc.canterbury.ac.nz>
References: <bqm3m4$aov$1@sea.gmane.org>
	<200312040254.hB42sLI14069@oma.cosc.canterbury.ac.nz>
Message-ID: <20031205000217.GA19503@mems-exchange.org>

On Thu, Dec 04, 2003 at 03:54:21PM +1300, Greg Ewing wrote:
> Does anyone have a real-life use case for subclassing int?

Not me. Dylan does not allow it's concrete numeric classes to be
subclassed.  That allows efficient method dispatch given restrictive
enough declarations.  Here is the standard numberic classes:

    class number(object):
        # Open Abstract Class

    class complex(number):
        # Sealed Abstract Class

    class real(complex):
        # Sealed Abstract Class

    class float(real):
        # Sealed Abstract Class

    class single_float(float)
        # Sealed Class

    class double_float(float)
        # Sealed Class

    class rational(real):
        # Sealed Abstract Class

    class integer(rational):
        # Sealed Class

complex is sealed which basically means that it cannot be subclassed
by users.  That allows Sealing is described here:
http://www.gwydiondylan.org/drm/drm_70.htm.

The language reference does not describe a "long" integer type but
the implementation is free to provide one.

  Neil

From ajsiegel at optonline.net  Thu Dec  4 06:59:45 2003
From: ajsiegel at optonline.net (Arthur)
Date: Thu Dec  4 18:56:10 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard Libary
Message-ID: <0HPD007N0D5XMT@mta4.srv.hcvlny.cv.net>


Guido writes:

>Can we stop this thread on copy()?  It's one of Arthur's pet subjects
>and you're not going to convince him, so it's pointless to argue.

Convince me of what?

If I have done nothing more than demonstrate that someone - from a substrata
of the audience Python talks about reaching, and who has been using it for
over 4 years - is confused and ignorant in this area....  Seems to me I have
made my point.

So will go away.

Art


From greg at cosc.canterbury.ac.nz  Thu Dec  4 19:47:02 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec  4 19:47:09 2003
Subject: [Python-Dev] Re: Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <002b01c3ba70$9fce5970$6402a8c0@arkdesktop>
Message-ID: <200312050047.hB50l2K22493@oma.cosc.canterbury.ac.nz>

Andrew Koenig <ark-mlist@att.net>:

> > (It has been pointed out that subclassing int isn't very useful, so
> > maybe this is a moot point.  Does anybody have a real use case?)
> 
> isinstance(x, int)
> 
> That doesn't require an inheritance relationship between the integral types,
> but it does require that long inherit from int.

Having int be an abstract class doesn't prevent it from being
subclassed - it just means the subclass needs to to more work
in order to behave like other concrete int subclasses.

Along with short and long, there could be a UserInt subclass
which delegates all the operations to an implementation object,
for people who really want an int subclass for some reason.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From edloper at gradient.cis.upenn.edu  Thu Dec  4 20:52:35 2003
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Thu Dec  4 19:51:11 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <E1AS39Q-0002mi-Q5@mail.python.org>
References: <E1AS39Q-0002mi-Q5@mail.python.org>
Message-ID: <3FCFE4E3.4000602@gradient.cis.upenn.edu>

>>If you agree with this premise, then it suggests a different
>>approach. Use different implementations in C, but have type(x) in
>>Python lie.  type(x) would always return the type object which is
>>now known as "long".
> 
> If this can be made to work, I like it.  __class__ should also be
> hacked, and isinstance(); and who knows how many other places, but
> probably not too many.

If we go this route, then how general should the mechanism for "merging" 
types in Python-space (but keeping them separate in c-space) be?  I.e., 
should this be an integer-specific change, or a general mechanism that 
happens to be used by int?  It seems like there might be other use cases 
for similar "merges" (string/unicode comes to mind).

-Edward



From greg at cosc.canterbury.ac.nz  Thu Dec  4 19:56:43 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec  4 19:56:50 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <78CB3A69-2669-11D8-9361-0003931DF95C@python.net>
Message-ID: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> It seems to me that any time there are more than two CObjects around we 
> have a good chance of causing mischief:

Indeed, CObjects seem fundamentally dangerous to me, unless
the modules which create and use them are extremely careful
to make sure they can tell whether they've got the right
kind of CObject.

And I suspect that many modules which make use of CObjects
aren't that careful. CStringIO obviously isn't...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From kbk at shore.net  Thu Dec  4 19:57:11 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Thu Dec  4 19:57:16 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <brqoz608.fsf@python.net> (Thomas Heller's message of "Thu, 04
	Dec 2003 21:23:51 +0100")
References: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<m3d6b41o6u.fsf@float.localdomain> <brqoz608.fsf@python.net>
Message-ID: <m3d6b4ytco.fsf@float.localdomain>

I notice there are a number of version updates you guys are making
which aren't in the PEPs.  Do you want me to work up a patch?

Second, the PEPs talk about "rc" release id's but you are all using
"c".  Do you want that included in the patch?

-- 
KBK

From greg at cosc.canterbury.ac.nz  Thu Dec  4 20:09:17 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec  4 20:09:23 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312042040.hB4Ke8D06619@c-24-5-183-134.client.comcast.net>
Message-ID: <200312050109.hB519Hw22644@oma.cosc.canterbury.ac.nz>

Guido van Rossum <guido@python.org>:

> > If you agree with this premise, then it suggests a different
> > approach. Use different implementations in C, but have type(x) in
> > Python lie.  type(x) would always return the type object which is
> > now known as "long".
> 
> If this can be made to work, I like it.  __class__ should also be
> hacked, and isinstance(); and who knows how many other places, but
> probably not too many.

What would happen under this scheme if someone did try
to subclass int, or long? What would isinstance() make of
such subclasses?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From fdrake at acm.org  Thu Dec  4 22:32:42 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec  4 22:32:56 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <m3d6b4ytco.fsf@float.localdomain>
References: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<m3d6b41o6u.fsf@float.localdomain> <brqoz608.fsf@python.net>
	<m3d6b4ytco.fsf@float.localdomain>
Message-ID: <16335.64602.857668.389843@grendel.fdrake.net>


Kurt B. Kaiser writes:
 > I notice there are a number of version updates you guys are making
 > which aren't in the PEPs.  Do you want me to work up a patch?

Which changes in particular?  I think the version updates in Doc/
aren't in the PEP anymore because they're no longer needed on the
trunk.  I don't know about the rest of the changes.

I'd be more interested in problems making a trunk release based on the
PEPs than a branch release, but it's probably worth noting the
differences in a separate section.

 > Second, the PEPs talk about "rc" release id's but you are all using
 > "c".  Do you want that included in the patch?

This would be good.  The "c" is for "candidate", which is intended to
sort before "final".


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From fumanchu at amor.org  Fri Dec  5 00:18:08 2003
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Dec  5 00:23:20 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561B11@opus.amorhq.net>

How did this thread start again? ;)

Guido van Rossum, a long time ago, wrote:
> In the shower (really!) I was thinking about the old problem of going
> through a list of items that are supposed to be grouped by some key,
> and doing something extra at the end of each group...
> So I realized this is easy to do with a generator, assuming we can
> handle keeping a list of all items in a group.
> for group in groupby(key, sequence):

Greg Ewing's proposal of a "given" keyword (x.score given x) got me
thinking. I figured I would play around a bit and try to come up with
the most readable version of the original "groupby" idea (for which I
could imagine *some* implementation):

    for group in sequence.groups(using item.score - item.penalty):
        ...do stuff with group

Having written this down, it seems to me the most readable so far. The
keyword "using" creates a new scope, within which "item" is bound to the
arg (or *args?) passed in. I don't know about you all, but the thing I
like least about lambda is having to mention 'x' twice:

    lambda x: x.score

Why have the programmer bind a custom name to an object we're going to
then use 'anonymously' anyway? I understand its historical necessity,
but it's always struck me as more complex than the concept being
implemented. Ideally, we should be able to reference the passed-in
objects without having to invent names for them.

Now, consider multi-arg lambdas such as:

    sequence.sort(lambda x, y: cmp(x[0], y[0]))

In these cases, we wish to apply the same operation to each item (that
is, we calculate x[0] and y[0]). If we bind "item" to each argument *in
turn*, we save a lot of syntax. The above might then be written as:
    sequence.sort(using cmp(item[0])) # Hard to implement.

or:
    sequence.sort(cmp(using item[0])) # Easier but ugly. Meh.

or:
    sequence.sort(cmp using item[0])  # Oooh. Nice. :)

or:
    # might we assume cmp(), since sort does...?
    sequence.sort(using item[0])

I like #3, since cmp is explicit but doesn't use cmp(), which looks too
much like a call. Given (cmp using item[0]), the "using block" would
look at the arguments supplied by sort(), call __getitem__[0] for each,
and pass those values in order into cmp, returning the result.

The "item" keyword functions similarly to Guido's Voodoo.foo() proposal,
now that I think about it. There's no reason it couldn't grow some early
binding, either, as suggested, although multiple operations would become
unwieldy. How would you early-bind this?

    sequence.groups(using divmod(item, 4)[1])

...except perhaps by using multiply-nested scopes to bind the "1" and
then the "4"?

Hmm. It would have to do some fancy dancing to get everything in the
right order. Too much like reinventing Python to think about at the
moment. :) The point is, passing the "item" instance through such a
scheme should be the easy part.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From aahz at pythoncraft.com  Fri Dec  5 00:57:33 2003
From: aahz at pythoncraft.com (Aahz)
Date: Fri Dec  5 00:57:35 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <1070546177.3fcf3d01e6f52@mcherm.com>
References: <1070546177.3fcf3d01e6f52@mcherm.com>
Message-ID: <20031205055733.GA12283@panix.com>

On Thu, Dec 04, 2003, Michael Chermside wrote:
>
> To my mind, the *ideal* is that there's a single type, probably
> called "int" or "integer" which can take on an arbitrary size and
> precision BUT which, if it happens to be small enough to fit in
> a C int will take up less memory and run faster. The fact that
> it's stored differently should be an implementation detail not
> visible to the user.

Hmmmm...  How important is the "less memory" angle versus the "faster"
angle?  Why not just add a few bytes to the long type to store an int?
If you're using the int, the pointer is NULL, and you're saving that
memory, at least.  The only harm is some extra memory in the type
object.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From tim at multitalents.net  Fri Dec  5 00:57:47 2003
From: tim at multitalents.net (Tim Rice)
Date: Fri Dec  5 00:58:04 2003
Subject: [Python-Dev] release23-maint tests
Message-ID: <Pine.UW2.4.58.0312042132580.9807@ou8.int.multitalents.net>


I didn't see a catagory for tests on source forge so I'm reporting here.

I pulled the release23-maint branch this morning and built on
UnixWare 7.1.1.

-----------------
test_dl skipped -- Could not open any shared libraries
It does have shared libraries, just doesn't have /usr/lib/libc.so.
Adding /usr/lib/libc.so.1 to the list will fix it.

-----------------
test_largefile skipped -- filesystem does not have largefile support
UnixWare 7.0.0 and later does have largefile suport.
The config.log shows
configure:9741: checking whether to enable large file support
configure:9751: result: yes
12G of free space to play with.

Any tips on how to track this down would be apreciated.

-----------------
test test_resource produced unexpected output:
**********************************************************************
*** mismatch between line 2 of expected output and line 2 of actual output:
- True
+ False
**********************************************************************
Tips on tracking this down?

test test_tarfile crashed -- exceptions.IOError: [Errno 30] Read-only file system: '/opt/src/utils/python/python-2.3/src/Lib/test/testtar.tar.gz'
Some of us with multiple platforms build from a single read only
source tree.

-----------------
test_tempfile
Exception exceptions.AttributeError: "mkstemped instance has no attribute 'fd'" in <bound method mkstemped.__del__ of <test.test_tempfile.mkstemped instance at 0x8acf06c>> ignored
test test_tempfile failed -- Traceback (most recent call last):
  File "/opt/src/utils/python/python-2.3/src/Lib/test/test_tempfile.py", line 246, in test_basic_many
  File "/opt/src/utils/python/python-2.3/src/Lib/test/test_tempfile.py", line 229, in do_create
  File "/opt/src/utils/python/python-2.3/src/Lib/test/test_tempfile.py", line 45, in failOnException
  File "/opt/src/utils/python/python-2.3/src/Lib/unittest.py", line 270, in fail
AssertionError: _mkstemp_inner raised exceptions.OSError: [Errno 24] Too many open files: '/tmp/aakOvEqF'

limits.h shows 60 for OPEN_MAX.
Lowering TEST_FILES to 55 in test_tempfile.py makes it pass.
Maybe you just want to add openunix8 and unixware7 to
if sys.platform == 'mac':



-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net



From Jack.Jansen at cwi.nl  Fri Dec  5 05:13:35 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Dec  5 05:13:38 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <3FCFE4E3.4000602@gradient.cis.upenn.edu>
References: <E1AS39Q-0002mi-Q5@mail.python.org>
	<3FCFE4E3.4000602@gradient.cis.upenn.edu>
Message-ID: <AFE9DFAD-270B-11D8-989C-000A27B19B96@cwi.nl>


On 5-dec-03, at 2:52, Edward Loper wrote:

>>> If you agree with this premise, then it suggests a different
>>> approach. Use different implementations in C, but have type(x) in
>>> Python lie.  type(x) would always return the type object which is
>>> now known as "long".
>> If this can be made to work, I like it.  __class__ should also be
>> hacked, and isinstance(); and who knows how many other places, but
>> probably not too many.
>
> If we go this route, then how general should the mechanism for 
> "merging" types in Python-space (but keeping them separate in c-space) 
> be?  I.e., should this be an integer-specific change, or a general 
> mechanism that happens to be used by int?  It seems like there might 
> be other use cases for similar "merges" (string/unicode comes to 
> mind).

I think this is a good idea, I may have some uses for it too. But, more 
importantly, making this a general mechanism will force us to formulate 
the exact set of restrictions we place on two such types.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From mwh at python.net  Fri Dec  5 06:04:53 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec  5 06:04:56 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <m3ptf4idcs.fsf@mira.informatik.hu-berlin.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "04 Dec 2003 20:37:07
	+0100")
References: <78CB3A69-2669-11D8-9361-0003931DF95C@python.net>
	<m3ptf4idcs.fsf@mira.informatik.hu-berlin.de>
Message-ID: <2mk75bwmne.fsf@starship.python.net>

martin@v.loewis.de (Martin v. L?wis) writes:

> Michael Hudson <mwh@python.net> writes:
>
>> It seems to me that it would be more sensible to have a dict mapping
>> names (maybe just module names...) to cobjects in the interpreter
>> state.  I guess there might be a way of getting ahold of the dict with
>> the gc module (though I can't think of one off hand).  This wouldn't
>> be a difficult change, and if the CObjects are left in the module
>> dicts, it shouldn't even do that much damage to binary compatibility.
>> 
>> Thoughts?
>
> Good idea. Alternatively, we could make "APIs" a feature of modules:
> PyModule_GetAPI, PyModule_SetAPI. We would then define
>
> typedef struct {
> 	PyObject_HEAD
> 	PyObject *md_dict;
>         PyObject *md_api;
> } PyModuleObject;
>
> We could restrict md_api to CObjects, which, means we would not need
> to change module_traverse.

I think you can still break this, though:

>>> import _curses
>>> import sys
>>> sys.modules['cStringIO'] = _curses

(doesn't break today because cStringIO and _curses use different
attribute names for the CObject).

Cheers,
mwh

-- 
31. Simplicity does not precede complexity, but follows it.
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From mwh at python.net  Fri Dec  5 06:06:53 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec  5 06:07:02 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz> (Greg
	Ewing's message of "Fri, 05 Dec 2003 13:56:43 +1300 (NZDT)")
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>
Message-ID: <2mfzfzwmk2.fsf@starship.python.net>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> Michael Hudson <mwh@python.net>:
>
>> It seems to me that any time there are more than two CObjects around we 
>> have a good chance of causing mischief:
>
> Indeed, CObjects seem fundamentally dangerous to me, unless
> the modules which create and use them are extremely careful
> to make sure they can tell whether they've got the right
> kind of CObject.

Well, the right *CObject* -- it's CObject identity that matters.

> And I suspect that many modules which make use of CObjects
> aren't that careful. CStringIO obviously isn't...

As things stand, I don't think cPickle *can* be careful enough.
Perhaps we could just add a "name" field to CObjects and have module
init functions check that.

Cheers,
mwh

-- 
 (Of course SML does have its weaknesses, but by comparison, a
  discussion of C++'s strengths and flaws always sounds like an
  argument about whether one should face north or east when one
  is sacrificing one's goat to the rain god.)         -- Thant Tessman

From mwh at python.net  Fri Dec  5 06:11:00 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec  5 06:11:03 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312042040.hB4Ke8D06619@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Thu, 04 Dec 2003 12:40:08 -0800")
References: <1070546177.3fcf3d01e6f52@mcherm.com>
	<200312042040.hB4Ke8D06619@c-24-5-183-134.client.comcast.net>
Message-ID: <2mbrqnwmd7.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> If you agree with this premise, then it suggests a different
>> approach. Use different implementations in C, but have type(x) in
>> Python lie.  type(x) would always return the type object which is
>> now known as "long".
>
> If this can be made to work, I like it.

I think Armin is ill at the moment, but you're all singing his tune
:-) (He has some very similar sounding ideas for PyPy).

Cheers,
mwh

-- 
  If Unicode is a horde of zombies with flaming dung sticks, 
  the hideous intricacies of JIS, Chinese Big-5, Chinese 
  Traditional, KOI-8, et cetera are at least an army of ogres 
  with salt and flensing knives.        -- Eric S. Raymond, python-dev

From ajsiegel at optonline.net  Fri Dec  5 08:28:06 2003
From: ajsiegel at optonline.net (Arthur)
Date: Fri Dec  5 08:24:22 2003
Subject: [Python-Dev] re: Tutorial: Brief Introduction to the Standard
Message-ID: <0HPF00AVIBWJ3Q@mta5.srv.hcvlny.cv.net>

>The opposite question, "What is fundamentally wrong with list[:]..." has 
>an easy answer:

>It takes experience or explanation of slicing to know what it does.  
>copy(list) is easy even for novices to understand.

I agree.  But think the problem is even deeper than that.  Because it all
impacts, in the end, understanding what = does.

As Guido says, a favorite subject of mine.

Art


From Jack.Jansen at cwi.nl  Fri Dec  5 08:52:38 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Dec  5 08:52:36 2003
Subject: [Python-Dev] 2.3 maint: test_poll fails on OSX
Message-ID: <49E8102C-272A-11D8-989C-000A27B19B96@cwi.nl>

With the current state of the 2.3 maintenance branch test_poll fails:

   File "/Volumes/Vuur/Users/jack/src/python-23/Lib/test/test_poll.py", 
line 171, in ?
     test_poll1()
   File "/Volumes/Vuur/Users/jack/src/python-23/Lib/test/test_poll.py", 
line 65, in test_poll1
     poll_unit_tests()
   File "/Volumes/Vuur/Users/jack/src/python-23/Lib/test/test_poll.py", 
line 77, in poll_unit_tests
     r = p.poll()
error: (9, 'Bad file descriptor')

This also looks legitimate to me: the test goes to great lengths to 
make sure it's passing an invalid file descriptor to poll(). But I 
assume there's something deeper going on that I don't understand...
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From anthony at python.org  Fri Dec  5 09:40:53 2003
From: anthony at python.org (Anthony Baxter)
Date: Fri Dec  5 09:41:21 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1)
Message-ID: <200312051440.hB5Eer1P009275@maxim.off.ekorp.com>


On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.3.3 (release candidate 1).

Python 2.3.3 is a bug-fix release of Python 2.3. A couple of serious
bugs related to weakrefs and the cyclic garbage collector have been 
fixed, along with a number of bugs in the standard library. See the 
release notes on the web page for more details.

For more information on Python 2.3.3c1, including download links for
various platforms, release notes, and known issues, please see

    http://www.python.org/2.3.3

Highlights of this new release include:

  - A couple of serious bugs in the interactions of weakrefs and 
    cyclic GC have been squashed.
  - At shutdown, the second call to the cyclic garbage collector has
    been removed. This caused more problems than it solved.
  - The xml.parsers.expat module now provides Expat 1.95.7.
  - urllib2's HTTP Digest Auth support works again.
  - See http://www.python.org/2.3.3/NEWS.html for other bugfixes.

Highlights of the previous major Python release (2.3) are available     
from the Python 2.3 page, at                                            

    http://www.python.org/2.3/highlights.html

A final version of Python 2.3.3 should follow in a couple of weeks,
just in time for your Christmas stockings.

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python 2.3.x Release Manager
(on behalf of the entire python-dev team)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 226 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20031206/5d033837/attachment.bin
From guido at python.org  Fri Dec  5 09:41:20 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 09:41:25 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: Your message of "Fri, 05 Dec 2003 11:06:53 GMT."
	<2mfzfzwmk2.fsf@starship.python.net> 
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>  
	<2mfzfzwmk2.fsf@starship.python.net> 
Message-ID: <200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>

> > Indeed, CObjects seem fundamentally dangerous to me, unless
> > the modules which create and use them are extremely careful
> > to make sure they can tell whether they've got the right
> > kind of CObject.
>> 
> Well, the right *CObject* -- it's CObject identity that matters.

I seem to recall that the CObject design contains a feature to avoid
using the wrong CObject, but it's not used by any of the CObject
implementations in the standard library, and the CObject docs don't
describe how it's meant to use (they were clearly reverse-engineered
from the code).

Several early checkin messages of cobject.c mention Jim Fulton as the
author, so maybe we could ask him.  (I'm cc'ing Jim in case he
remembers the true intention for the description; he might also know
whether Zope 2 still uses it; I can only check Zope 3, and it
doesn't.)

But even without Jim's confirmation, ISTM from memory and looking at
the source that the 'desc' field (which stands for description) was
intended as a clue that could be checked by the user of a C Object to
verify it had gotten the right one: in a shared header (needed anyway
to describe the layout of the struct pointer contained in the C
Object) you could define a magic word or a magic string value that
would be stored as the description, and the users would check that
value before believing the struct pointer.

Unfortunately the same CObject version that introduced the description
field also introduced the convenience function PyCObject_Import(),
which doesn't return the description, and returns the void pointer
contained in the CObject without returning the CObject itself, leaving
its caller without a clue about the description...

Anyway, I think it can be fixed by starting to use the description
field and adding a new convenience C API, perhaps named
PyCObject_ImportEx(), whch takes a description and checks it.  We
should try to standardize on what kind of thing to use as the
description -- if we use a magic word, PyCObject_ImportEx() should do
a simple compare, but if we use a string, it should do a strcmp().  Or
the API could simply return the description into an output parameter,
making the caller responsible for doing the checking.

And the docs should be updated to explain the description better
(currently at one point the description is called "extra callback data
for the destructor function" which seems a rather odd feature).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Dec  5 10:09:21 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 10:09:27 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Fri, 05 Dec 2003 00:57:33 EST."
	<20031205055733.GA12283@panix.com> 
References: <1070546177.3fcf3d01e6f52@mcherm.com>  
	<20031205055733.GA12283@panix.com> 
Message-ID: <200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net>

> > To my mind, the *ideal* is that there's a single type, probably
> > called "int" or "integer" which can take on an arbitrary size and
> > precision BUT which, if it happens to be small enough to fit in
> > a C int will take up less memory and run faster. The fact that
> > it's stored differently should be an implementation detail not
> > visible to the user.
> 
> Hmmmm...  How important is the "less memory" angle versus the "faster"
> angle?  Why not just add a few bytes to the long type to store an int?
> If you're using the int, the pointer is NULL, and you're saving that
> memory, at least.  The only harm is some extra memory in the type
> object.

I'd say that the "less memory" angle is pretty important.  The int
implementation has been using every trick in the book to save memory
from very early days on: it has the most sophisticated special-purpose
allocator, *and* it uses a cache for popular values, *and* you really
can't squeeze any more bits out of the structure.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Fri Dec  5 10:12:45 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec  5 10:12:49 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Fri, 05 Dec 2003 06:41:20 -0800")
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>
	<2mfzfzwmk2.fsf@starship.python.net>
	<200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
Message-ID: <2m7k1bwb6a.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> > Indeed, CObjects seem fundamentally dangerous to me, unless
>> > the modules which create and use them are extremely careful
>> > to make sure they can tell whether they've got the right
>> > kind of CObject.
>>> 
>> Well, the right *CObject* -- it's CObject identity that matters.
>
> I seem to recall that the CObject design contains a feature to avoid
> using the wrong CObject, but it's not used by any of the CObject
> implementations in the standard library, and the CObject docs don't
> describe how it's meant to use (they were clearly reverse-engineered
> from the code).
[...]
> And the docs should be updated to explain the description better
> (currently at one point the description is called "extra callback data
> for the destructor function" which seems a rather odd feature).

It had occurred to me that you could use this field for this purpose,
but it hadn't occurred to me that this might have been the *intent* of
this field.

Why a void* then?  That's particularly unassertive.  char* would be
better, surely...

Cheers,
mwh

-- 
  I also fondly recall Paris because that's where I learned to
  debug Zetalisp while drunk.                          -- Olin Shivers

From theller at python.net  Fri Dec  5 10:17:50 2003
From: theller at python.net (Thomas Heller)
Date: Fri Dec  5 10:17:59 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Fri, 05 Dec 2003 07:09:21 -0800")
References: <1070546177.3fcf3d01e6f52@mcherm.com>
	<20031205055733.GA12283@panix.com>
	<200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net>
Message-ID: <brqnuwdd.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

>> Hmmmm...  How important is the "less memory" angle versus the "faster"
>> angle?  Why not just add a few bytes to the long type to store an int?
>> If you're using the int, the pointer is NULL, and you're saving that
>> memory, at least.  The only harm is some extra memory in the type
>> object.
>
> I'd say that the "less memory" angle is pretty important.  The int
> implementation has been using every trick in the book to save memory
> from very early days on: it has the most sophisticated special-purpose
> allocator, *and* it uses a cache for popular values, *and* you really
> can't squeeze any more bits out of the structure.

IIUC, doing range(10000000) in a program somewhere allocates 10 million
integers, and these are *never* freed, they live in the cache forever.
Correct?

I do not have an actual use case for this, but it seems somewhat
strange.
And the point comes up from time to time on c.l.p.

Thomas


From anthony at ekit-inc.com  Fri Dec  5 10:21:26 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Fri Dec  5 10:22:23 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs 
In-Reply-To: Message from Thomas Heller <theller@python.net> 
	of "Fri, 05 Dec 2003 16:17:50 BST." <brqnuwdd.fsf@python.net> 
Message-ID: <200312051521.hB5FLRmr009991@maxim.off.ekorp.com>


>>> Thomas Heller wrote
> Guido van Rossum <guido@python.org> writes:
> IIUC, doing range(10000000) in a program somewhere allocates 10 million
> integers, and these are *never* freed, they live in the cache forever.
> Correct?


My understanding was that it was only ints between -5 and +100 that were
cached forever.



From guido at python.org  Fri Dec  5 10:22:42 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 10:23:18 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Thu, 04 Dec 2003 21:18:08 PST."
	<DE1CF2B4FEC4A342BF62B6B2B334601E561B11@opus.amorhq.net> 
References: <DE1CF2B4FEC4A342BF62B6B2B334601E561B11@opus.amorhq.net> 
Message-ID: <200312051522.hB5FMgQ08216@c-24-5-183-134.client.comcast.net>

> Greg Ewing's proposal of a "given" keyword (x.score given x) got me
> thinking. I figured I would play around a bit and try to come up with
> the most readable version of the original "groupby" idea (for which I
> could imagine *some* implementation):
> 
>     for group in sequence.groups(using item.score - item.penalty):
>         ...do stuff with group
> 
> Having written this down, it seems to me the most readable so far. The
> keyword "using" creates a new scope, within which "item" is bound to the
> arg (or *args?) passed in. I don't know about you all, but the thing I
> like least about lambda is having to mention 'x' twice:
> 
>     lambda x: x.score
> 
> Why have the programmer bind a custom name to an object we're going to
> then use 'anonymously' anyway? I understand its historical necessity,
> but it's always struck me as more complex than the concept being
> implemented. Ideally, we should be able to reference the passed-in
> objects without having to invent names for them.

Huh?  How can you reference something without having a name for it?
Are you proposing to add pronouns to Python?

> Now, consider multi-arg lambdas such as:
> 
>     sequence.sort(lambda x, y: cmp(x[0], y[0]))
> 
> In these cases, we wish to apply the same operation to each item (that
> is, we calculate x[0] and y[0]). If we bind "item" to each argument *in
> turn*, we save a lot of syntax. The above might then be written as:
>     sequence.sort(using cmp(item[0])) # Hard to implement.
> 
> or:
>     sequence.sort(cmp(using item[0])) # Easier but ugly. Meh.
> 
> or:
>     sequence.sort(cmp using item[0])  # Oooh. Nice. :)
> 
> or:
>     # might we assume cmp(), since sort does...?
>     sequence.sort(using item[0])
> 
> I like #3, since cmp is explicit but doesn't use cmp(), which looks too
> much like a call. Given (cmp using item[0]), the "using block" would
> look at the arguments supplied by sort(), call __getitem__[0] for each,
> and pass those values in order into cmp, returning the result.

There are lots of situations where the comparison lambda is just a bit
more complex than this, for example:

  lambda x, y: cmp(x[0], y[0]) or cmp(x[1], y[1])

And how would you spell lambda x, y: x+y?  "+ using item"???  That
becomes a syntactical nightmare.  (Or what about lambda x, y: 2*(x+y)?)

I also think you are cheating by using sort() as the example -- other
examples of multi-argument lambdas aren't necessarily so uniform in
the arguments.

> The "item" keyword functions similarly to Guido's Voodoo.foo() proposal,
> now that I think about it. There's no reason it couldn't grow some early
> binding, either, as suggested, although multiple operations would become
> unwieldy. How would you early-bind this?
> 
>     sequence.groups(using divmod(item, 4)[1])
> 
> ...except perhaps by using multiply-nested scopes to bind the "1" and
> then the "4"?

I see all sorts of problems with this, but early-binding "1" and "4"
aren't amongst them -- early binding only applies to free variables,
not to constants.

> Hmm. It would have to do some fancy dancing to get everything in the
> right order. Too much like reinventing Python to think about at the
> moment. :) The point is, passing the "item" instance through such a
> scheme should be the easy part.

I've read this whole post twice, and I still don't understand what
you're really proposing (or how it could ever work given the rest of
Python), so I think it's probably not a good idea from a readability
perspective...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From anthony at interlink.com.au  Fri Dec  5 10:23:19 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Dec  5 10:23:30 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle) 
In-Reply-To: Message from Anthony Baxter <anthony@interlink.com.au> of "Fri,
	05 Dec 2003 00:41:21 +1100."
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com> 
Message-ID: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>


After a short chat with Thomas, we'd like to pencil in the 19th for 
the final release (so two weeks from the RC1). Does this cause anyone
pain? 

ta,
Anthony


From guido at python.org  Fri Dec  5 10:26:24 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 10:26:32 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: Your message of "Fri, 05 Dec 2003 15:12:45 GMT."
	<2m7k1bwb6a.fsf@starship.python.net> 
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>
	<2mfzfzwmk2.fsf@starship.python.net>
	<200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net> 
	<2m7k1bwb6a.fsf@starship.python.net> 
Message-ID: <200312051526.hB5FQO908272@c-24-5-183-134.client.comcast.net>

> It had occurred to me that you could use this field for this purpose,
> but it hadn't occurred to me that this might have been the *intent* of
> this field.
> 
> Why a void* then?  That's particularly unassertive.  char* would be
> better, surely...

Jim only knows...  Probably a case of mistaken generality. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Dec  5 10:29:42 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 10:29:48 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Fri, 05 Dec 2003 16:17:50 +0100."
	<brqnuwdd.fsf@python.net> 
References: <1070546177.3fcf3d01e6f52@mcherm.com>
	<20031205055733.GA12283@panix.com>
	<200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net> 
	<brqnuwdd.fsf@python.net> 
Message-ID: <200312051529.hB5FTgq08289@c-24-5-183-134.client.comcast.net>

> IIUC, doing range(10000000) in a program somewhere allocates 10 million
> integers, and these are *never* freed, they live in the cache forever.
> Correct?

Wrong.  The memory allocated can never be used for other objects than
ints, but it will be reused for other ints.  IOW if you do

  for i in range(100):
      range(i*1000000, (i+1)*1000000)

you only allocate space for 1000000 ints, not for 100 times 1000000
ints.  (The range(100) OTOH are all in a cache that make the objects
indestructible, but this only applies to values in [-5, 100).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake at acm.org  Fri Dec  5 10:30:15 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec  5 10:30:43 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle) 
In-Reply-To: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
References: <anthony@interlink.com.au>
	<200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
	<200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
Message-ID: <16336.42119.508562.747938@grendel.fdrake.net>


Anthony Baxter writes:
 > After a short chat with Thomas, we'd like to pencil in the 19th for 
 > the final release (so two weeks from the RC1). Does this cause anyone
 > pain? 

Someone, somewhere, of course.  Everything does!  +1


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From recht at netbsd.org  Fri Dec  5 10:35:18 2003
From: recht at netbsd.org (Marc Recht)
Date: Fri Dec  5 10:35:25 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was
	Re:	2.3.3 cycle) 
In-Reply-To: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
Message-ID: <16900000.1070638518@leeloo.intern.geht.de>

> After a short chat with Thomas, we'd like to pencil in the 19th for
> the final release (so two weeks from the RC1). Does this cause anyone
> pain?

Any change to get the patches #854796 and/or #806800 in ?

Cheers,
Marc
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 186 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20031205/bf385417/attachment.bin
From theller at python.net  Fri Dec  5 10:37:10 2003
From: theller at python.net (Thomas Heller)
Date: Fri Dec  5 10:37:24 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312051529.hB5FTgq08289@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Fri, 05 Dec 2003 07:29:42 -0800")
References: <1070546177.3fcf3d01e6f52@mcherm.com>
	<20031205055733.GA12283@panix.com>
	<200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net>
	<brqnuwdd.fsf@python.net>
	<200312051529.hB5FTgq08289@c-24-5-183-134.client.comcast.net>
Message-ID: <4qwfuvh5.fsf@python.net>

Guido van Rossum <guido@python.org> writes:

>> IIUC, doing range(10000000) in a program somewhere allocates 10 million
>> integers, and these are *never* freed, they live in the cache forever.
>> Correct?
>
> Wrong.  The memory allocated can never be used for other objects than
> ints, but it will be reused for other ints.  IOW if you do
>
>   for i in range(100):
>       range(i*1000000, (i+1)*1000000)
>
> you only allocate space for 1000000 ints, not for 100 times 1000000
> ints.  (The range(100) OTOH are all in a cache that make the objects
> indestructible, but this only applies to values in [-5, 100).

Ok, then it makes sense.  Thanks for the clarification.

Thomas


From tim.one at comcast.net  Fri Dec  5 10:50:58 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec  5 10:50:59 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs 
In-Reply-To: <200312051521.hB5FLRmr009991@maxim.off.ekorp.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGEHJAB.tim.one@comcast.net>

[maybe Thomas Heller]
>> IIUC, doing range(10000000) in a program somewhere allocates 10
>> million integers, and these are *never* freed, they live in the
>> cache forever. Correct?

[Anthony Baxter]
> My understanding was that it was only ints between -5 and +100 that
> were cached forever.

Tiny ints are *shared*:  no matter how many computations return, and data
structures hold, the integer 3, they all point to the same integer object.

Space for all ints (tiny or not) is taken from a dedicated freelist, used
for ints and nothing but ints.  Whenever an int is deallocated, its memory
is added to this dedicated freelist.  Unlike most dedicated freelists in
Python, the int freelist is unbounded in size.  So when you do
range(10000000), and that list goes away, most of the space for the int
objects it held ends up in millions of int objects pushed onto the int
freelist.  They all stay there until the program shuts down (although they
get reused to satisfy requests for new int objects).

floats have a very similar (unbounded and immortal) dedicated freelist, so,
e.g., [abs(0.5) for i in xrange(10000000)] reserves space for 10 million
float objects forever after (although [0.5 for i in xrange(10000000)] only
creates 1 float object -- as is obvious to the most casual observer <wink>).


From fdrake at acm.org  Fri Dec  5 11:11:00 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec  5 11:11:42 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: <16900000.1070638518@leeloo.intern.geht.de>
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
Message-ID: <16336.44564.604153.644072@grendel.fdrake.net>


Marc Recht writes:
 > Any change to get the patches #854796 and/or #806800 in ?

It's hard to make changes once the release candidate is out.  ;-(

I still think the use of apply() in the logging module needs to be
fixed.  Here's how I think it should be done, retaining source
compatibility for Python back to 1.5.2:

Add a new module, logging._apply, that defines apply() in terms of the
*args, **kw syntax.  The modules that use apply, "logging" and
"logging.config", would each contain these two lines near the top:

if sys.version_info >= (2, 3):
    from logging._apply import apply

That would allow a compatible version of apply() to be used for newer
Pythons without triggering the deprecation warning.  It would also
minimize the changes to the package.

Regarding the compatibility requirements in PEP 291, I think we need
to decide what in CVS those apply to:  only the trunk, or also to the
maintenance branches.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From guido at python.org  Fri Dec  5 11:15:57 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 11:16:18 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: Your message of "Fri, 05 Dec 2003 11:11:00 EST."
	<16336.44564.604153.644072@grendel.fdrake.net> 
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de> 
	<16336.44564.604153.644072@grendel.fdrake.net> 
Message-ID: <200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>

> I still think the use of apply() in the logging module needs to be
> fixed.  Here's how I think it should be done, retaining source
> compatibility for Python back to 1.5.2:
> 
> Add a new module, logging._apply, that defines apply() in terms of the
> *args, **kw syntax.  The modules that use apply, "logging" and
> "logging.config", would each contain these two lines near the top:
> 
> if sys.version_info >= (2, 3):
>     from logging._apply import apply
> 
> That would allow a compatible version of apply() to be used for newer
> Pythons without triggering the deprecation warning.  It would also
> minimize the changes to the package.

Unfortunately, this would make apply even slower than the system
version with the silent deprecation warning. :-(

I'd rather drop 1.5.2 compatibility for the logging module *as it is
in CVS*.  1.5.2 users can continue to use the last version that was
distributed by its author independently.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake at acm.org  Fri Dec  5 11:22:06 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec  5 11:22:46 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: <200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
	<16336.44564.604153.644072@grendel.fdrake.net>
	<200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
Message-ID: <16336.45230.749132.305648@grendel.fdrake.net>


Guido van Rossum writes:
 > I'd rather drop 1.5.2 compatibility for the logging module *as it is
 > in CVS*.  1.5.2 users can continue to use the last version that was
 > distributed by its author independently.

So would I, and this was what I originally proposed (removing the use
of apply() altogether).  There was some objection to this, mostly
based on PEP 291.  It wasn't clear how many people actually cared
about this case, and I don't recall seeing Vinay chime in.

On the other hand, I find I'm not really a fan of deprecating apply()
in the first place, so I can think of other solutions.  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From barry at python.org  Fri Dec  5 11:46:17 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Dec  5 11:46:22 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: <200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
	<16336.44564.604153.644072@grendel.fdrake.net>
	<200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
Message-ID: <1070642777.1377.22.camel@anthem>

On Fri, 2003-12-05 at 11:15, Guido van Rossum wrote:

> I'd rather drop 1.5.2 compatibility for the logging module *as it is
> in CVS*.  1.5.2 users can continue to use the last version that was
> distributed by its author independently.

+1

I'd suggest updating PEP 291 in a way similar to what I did yesterday
for the email package.

-Barry



From guido at python.org  Fri Dec  5 11:54:09 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 11:54:14 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: Your message of "Fri, 05 Dec 2003 11:22:06 EST."
	<16336.45230.749132.305648@grendel.fdrake.net> 
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
	<16336.44564.604153.644072@grendel.fdrake.net>
	<200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net> 
	<16336.45230.749132.305648@grendel.fdrake.net> 
Message-ID: <200312051654.hB5Gs9e08643@c-24-5-183-134.client.comcast.net>

> On the other hand, I find I'm not really a fan of deprecating apply()
> in the first place, so I can think of other solutions.  ;-)

We could deprecate it in the docs but remove the silent warning which
slows it down.  Then we could leave it in the logging module until
3.0.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake at acm.org  Fri Dec  5 11:58:25 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec  5 11:58:36 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: <200312051654.hB5Gs9e08643@c-24-5-183-134.client.comcast.net>
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
	<16336.44564.604153.644072@grendel.fdrake.net>
	<200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
	<16336.45230.749132.305648@grendel.fdrake.net>
	<200312051654.hB5Gs9e08643@c-24-5-183-134.client.comcast.net>
Message-ID: <16336.47409.107855.751131@grendel.fdrake.net>


Guido van Rossum writes:
 > We could deprecate it in the docs but remove the silent warning which
 > slows it down.  Then we could leave it in the logging module until
 > 3.0.

+42

Does this warrant a c2 release?  I'm happy to make the changes right
now.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From tim.one at comcast.net  Fri Dec  5 12:12:38 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec  5 12:12:41 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs 
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEGEHJAB.tim.one@comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEGMHJAB.tim.one@comcast.net>

[Tim]
> ...
> So when you do range(10000000), and that list goes away, most of the
> space for the int objects it held ends up in millions of int objects
> pushed onto the int freelist.  They all stay there until the program
> shuts down (although they get reused to satisfy requests for new int
> objects).

BTW, using pymalloc instead wouldn't eliminate that allocated memory is
never returned to the OS.  pymalloc grabs large "arenas" from malloc(), and
never free()s them.  The fundamental difference is that pymalloc is
type-agnostic:  space gotten from pymalloc for, say, a dict, can later be
reused for, say, a string.  pymalloc's freelists are segregated by raw
memory size rather than by type, and a given chunk of pymalloc memory may
even move from one size list to another (any number of times).


From guido at python.org  Fri Dec  5 12:19:11 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 12:20:19 2003
Subject: [Python-Dev] source-level backward compatibility requirements
In-Reply-To: Your message of "Fri, 05 Dec 2003 11:58:25 EST."
	<16336.47409.107855.751131@grendel.fdrake.net> 
References: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
	<16336.44564.604153.644072@grendel.fdrake.net>
	<200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
	<16336.45230.749132.305648@grendel.fdrake.net>
	<200312051654.hB5Gs9e08643@c-24-5-183-134.client.comcast.net> 
	<16336.47409.107855.751131@grendel.fdrake.net> 
Message-ID: <200312051719.hB5HJBp08708@c-24-5-183-134.client.comcast.net>

> Guido van Rossum writes:
>  > We could deprecate it in the docs but remove the silent warning which
>  > slows it down.  Then we could leave it in the logging module until
>  > 3.0.
> 
> +42
> 
> Does this warrant a c2 release?  I'm happy to make the changes right
> now.
> 
>   -Fred

Tht's a question for Anthony.  I suppose there will be a c2 release
anyway.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Dec  5 12:26:09 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 12:26:14 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Fri, 05 Dec 2003 12:12:38 EST."
	<LNBBLJKPBEHFEDALKOLCEEGMHJAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCEEGMHJAB.tim.one@comcast.net> 
Message-ID: <200312051726.hB5HQ9Y08752@c-24-5-183-134.client.comcast.net>

> BTW, using pymalloc instead wouldn't eliminate that allocated memory is
> never returned to the OS.  pymalloc grabs large "arenas" from malloc(), and
> never free()s them.  The fundamental difference is that pymalloc is
> type-agnostic:  space gotten from pymalloc for, say, a dict, can later be
> reused for, say, a string.  pymalloc's freelists are segregated by raw
> memory size rather than by type, and a given chunk of pymalloc memory may
> even move from one size list to another (any number of times).

But there aren't a lot of other objects that fit in the size of an
int, right?  I know of no other types that are <= 12 bytes on a 32-bit
machine, and assuming pymalloc rounds up, only one that is <= 16
(float).  There may be some obscure ones (super may be only 16 bytes
too) but they'll never need that many instances to fill the void left
behind by range(10000000).

(I'm assuming that pymalloc doesn't join adjacent small blocks to form
a larger block -- that way lies a full malloc reimplementation, and
what's the point of that...)

--Guido van Rossum (home page: http://www.python.org/~guido/)


From aahz at pythoncraft.com  Fri Dec  5 12:30:00 2003
From: aahz at pythoncraft.com (Aahz)
Date: Fri Dec  5 12:30:03 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net>
References: <1070546177.3fcf3d01e6f52@mcherm.com>
	<20031205055733.GA12283@panix.com>
	<200312051509.hB5F9Lr08164@c-24-5-183-134.client.comcast.net>
Message-ID: <20031205172959.GA1278@panix.com>

On Fri, Dec 05, 2003, Guido van Rossum wrote:
>Aahz:
>> 
>> Hmmmm...  How important is the "less memory" angle versus the "faster"
>> angle?  Why not just add a few bytes to the long type to store an int?
>> If you're using the int, the pointer is NULL, and you're saving that
>> memory, at least.  The only harm is some extra memory in the type
>> object.
> 
> I'd say that the "less memory" angle is pretty important.  The int
> implementation has been using every trick in the book to save memory
> from very early days on: it has the most sophisticated special-purpose
> allocator, *and* it uses a cache for popular values, *and* you really
> can't squeeze any more bits out of the structure.

Right, that's what's historically true.  Question is, by the time 3.0
comes out, *should* it still be true?  As pointed out downthread, we've
already got a problem with the int free list; earlier discussions on
python-dev have talked about ways of dealing with that (the simplest of
which will be to make range() an iterator, since that's the most common
problem source).  Why not use some of that memory savings to make the
int object easier to handle?  "Simple is better than complex"; I suspect
that we might have a win-win situation.

Perhaps we could push some of the memory-saving techniques into the long
object?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From fumanchu at amor.org  Fri Dec  5 12:33:08 2003
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Dec  5 12:38:31 2003
Subject: [Python-Dev] Re: "groupby" iterator
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E42D2F9@opus.amorhq.net>

I wrote:
> > Why have the programmer bind a custom name to an object 
> we're going to
> > then use 'anonymously' anyway? I understand its historical 
> necessity,
> > but it's always struck me as more complex than the concept being
> > implemented. Ideally, we should be able to reference the passed-in
> > objects without having to invent names for them.

And Guido van Rossum answered:
> Huh?  How can you reference something without having a name for it?
> Are you proposing to add pronouns to Python?

That's one way of looking at it. In a sense, names are already personal
pronouns with mutable referents. I'm thinking about a language (and
maybe Python wouldn't be a good candidate) where it also has standard
impersonal pronouns like "it" and "they" in English. But in an
already-restricted context, that is, lambda.

> There are lots of situations where the comparison lambda is just a bit
> more complex than this, for example:
> 
>   lambda x, y: cmp(x[0], y[0]) or cmp(x[1], y[1])
> 
> And how would you spell lambda x, y: x+y?  "+ using item"???  That
> becomes a syntactical nightmare.  (Or what about lambda x, y: 
> 2*(x+y)?)

Good questions.

> I've read this whole post twice, and I still don't understand what
> you're really proposing (or how it could ever work given the rest of
> Python), so I think it's probably not a good idea from a readability
> perspective...

That's fine. I'm really just musing; I'll refrain from that in the
future on this list unless I have code. ;) I work far too intuitively
sometimes. Maybe in the Spring, when I'll have time to dig into
internals and form a sample implementation. Thanks for your comments.

I *do* think this thread needs to get back to, "are we going to have
'groupby' in 2.4 or later?"


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

From guido at python.org  Fri Dec  5 12:45:31 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 12:46:07 2003
Subject: [Python-Dev] Re: "groupby" iterator
In-Reply-To: Your message of "Fri, 05 Dec 2003 09:33:08 PST."
	<DE1CF2B4FEC4A342BF62B6B2B334601E42D2F9@opus.amorhq.net> 
References: <DE1CF2B4FEC4A342BF62B6B2B334601E42D2F9@opus.amorhq.net> 
Message-ID: <200312051745.hB5HjVG08871@c-24-5-183-134.client.comcast.net>

> I *do* think this thread needs to get back to, "are we going to have
> 'groupby' in 2.4 or later?"

AFAIC that's decided.  We have received a high-quality implementation;
Raymond just has to find the time to check it in.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Fri Dec  5 13:05:39 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec  5 13:05:41 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <200312051726.hB5HQ9Y08752@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEHFHJAB.tim.one@comcast.net>

[Tim]
>> BTW, using pymalloc instead wouldn't eliminate that allocated memory
>> is never returned to the OS.  pymalloc grabs large "arenas" from
>> malloc(), and never free()s them.  The fundamental difference is
>> that pymalloc is type-agnostic:  space gotten from pymalloc for,
>> say, a dict, can later be reused for, say, a string.  pymalloc's
>> freelists are segregated by raw memory size rather than by type, and
>> a given chunk of pymalloc memory may even move from one size list to
>> another (any number of times).

[Guido]
> But there aren't a lot of other objects that fit in the size of an
> int, right?

That's true, but doesn't matter <wink>.

> I know of no other types that are <= 12 bytes on a 32-bit machine, and
> assuming pymalloc rounds up, only one that is <= 16 (float).  There
> may be some obscure ones (super may be only 16 bytes too) but they'll
> never need that many instances to fill the void left behind by range
> (10000000).
>
> (I'm assuming that pymalloc doesn't join adjacent small blocks to form
> a larger block

That's the part that's missing -- it does, but cleverly and efficiently.

> -- that way lies a full malloc reimplementation, and what's the
> point of that...)

Large "arenas" are cut into many equal-sized "pools", and pools are in turn
cut into equal-size objects.  All objects within a given pool have the same
size.  When all the objects in a given pool are freed, the entire pool is
available for reuse by objects of any size.  There's a little pressure
favoring reusing an entirely-free pool for objects of the same size as it
was last used for, because that saves some overhead (for example, some
size-dependent info is computed and stored in the pool header, and that part
can be skipped if the pool is used for objects of the same size as last
time).  pymalloc will change the size associated with an entirely-free pool
before it will ask for another arena, though.

So, e.g., if ints used pymalloc, range(10000000) would carve out enough
pools to hold 10 million ints, but when range(10000000) went away almost all
those pools would become entirely free, and so available for reuse by
objects of any size.  The coalescing here is *not* done on an
object-by-object basis.  Instead the pool header keeps a count of the number
of slots in that pool currently allocated.  When an object in the pool is
freed, that count is decremented.  If the count becomes 0 then, the entire
pool can be recycled without further ado, in constant time.  If the count
does not become 0 then, the newly-freed object is put at the head of a
singly-linked list of free objects within the pool, which is also a
constant-time operation (IOW, the objects within a pool act a lot like the
dedicated int freelist; if the pool is recycled to hold objects of a
different size, the freelist links within it are simply ignored and
overwritten).

A more-general malloc has to muck around asking whether the objects on
either side of a newly-freed object are also free, so that it can coalesce
them.  pymalloc never does that.  OTOH, pymalloc is emphatically not a
general-purpose allocator:  it doesn't even try to deal with requests for
more than about 256 bytes (those get passed on to the system malloc).  It
relies (for performance) on that the vast bulk of memory requests Python
makes are for small chunks of memory.  Vladimir did a great job of designing
this!  Ignoring cache effects, the dedicated int freelist is faster.  I'm
not sure it's always faster if cache effects are considered too (it's
complex, and pymalloc is very cache-conscious).


From python at rcn.com  Fri Dec  5 14:58:22 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec  5 14:58:41 2003
Subject: [Python-Dev] Re: "groupby" iterator
References: <DE1CF2B4FEC4A342BF62B6B2B334601E42D2F9@opus.amorhq.net>
	<200312051745.hB5HjVG08871@c-24-5-183-134.client.comcast.net>
Message-ID: <007801c3bb6a$2c9510c0$e841fea9@oemcomputer>

> > I *do* think this thread needs to get back to, "are we going to have
> > 'groupby' in 2.4 or later?"
> 
> AFAIC that's decided.  We have received a high-quality implementation;
> Raymond just has to find the time to check it in.

Expect it to go in this weekend :-)


Raymond

From python at rcn.com  Fri Dec  5 15:00:24 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec  5 15:00:28 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.292.10.3, 2.292.10.4
References: <E1ASJq2-0006IY-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <008a01c3bb6a$6bee1b40$e841fea9@oemcomputer>

FYI, I have a few more doc fixes for Py2.3.3.
Anthony asked me to wait until Sunday to put them in.


Raymond


From fdrake at acm.org  Fri Dec  5 15:02:54 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec  5 15:03:27 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.292.10.3, 2.292.10.4
In-Reply-To: <008a01c3bb6a$6bee1b40$e841fea9@oemcomputer>
References: <E1ASJq2-0006IY-00@sc8-pr-cvs1.sourceforge.net>
	<008a01c3bb6a$6bee1b40$e841fea9@oemcomputer>
Message-ID: <16336.58478.552434.65123@grendel.fdrake.net>


Raymond Hettinger writes:
 > FYI, I have a few more doc fixes for Py2.3.3.
 > Anthony asked me to wait until Sunday to put them in.

Any particular reason for the delay?


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From mcherm at mcherm.com  Fri Dec  5 15:14:38 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Dec  5 15:14:43 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
Message-ID: <1070655278.3fd0e72e6c665@mcherm.com>

I proposed:
> If you agree with this premise, then it suggests a different
> approach. Use different implementations in C, but have type(x) in
> Python lie.  type(x) would always return the type object which is
> now known as "long".

Greg Ewing writes:
> What would happen under this scheme if someone did try
> to subclass int, or long? What would isinstance() make of
> such subclasses?

Well, you could subclass long (probably renamed, but I'll use today's
names to avoid confusion) just fine. But you couldn't subclass int
because it wouldn't even be visible to a Python program. (Of course
we can't stop you from trying to subclass it in C, but then you get
what you deserve.)

Edward Loper writes:
> If we go this route, then how general should the mechanism for "merging" 
> types in Python-space (but keeping them separate in c-space) be?  I.e., 
> should this be an integer-specific change, or a general mechanism that 
> happens to be used by int?  It seems like there might be other use cases 
> for similar "merges" (string/unicode comes to mind).

I could be wrong, but I can't think of any other cases where this would
be useful. Your example of string/unicode is definitely NOT such a case:
unicode is a useful type, and string is ALSO a useful type, one with
slightly different behavior (eg: one is useful for binary data, the
other is handy for storing arabic text). You wouldn't want to go with a
solution where one of those types didn't even seem to exist from Python.

Really, the int/long thing is pretty unique, and it's because ints are
a bit special. With almost any other data type, we could just create
separate types which inherit from each other, but with ints we want them
to appear seamlessly to be the same thing as longs, yet we are willing
to grant them almost NO memory overhead. The only other type I can
think of which we want to be so small is float, and if we decided to
introduce a arbitrary precision binary based floating point type to
complement Decimal and Rational, then I think we'd need our heads
examined.

-- Michael Chermside


From tim.one at comcast.net  Fri Dec  5 15:16:05 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec  5 15:16:16 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was
	Re:2.3.3 cycle) 
In-Reply-To: <200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEHPHJAB.tim.one@comcast.net>

[Anthony Baxter]
> After a short chat with Thomas, we'd like to pencil in the 19th for
> the final release (so two weeks from the RC1). Does this cause anyone
> pain?

Lots of pain, but it's a *good* kind of pain.  I'll take more pain like
that, if you have it.  Thanks for your work on this!


From theller at python.net  Fri Dec  5 15:48:56 2003
From: theller at python.net (Thomas Heller)
Date: Fri Dec  5 15:49:05 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1)
In-Reply-To: <200312051440.hB5Eer1P009275@maxim.off.ekorp.com> (Anthony
	Baxter's message of "Sat, 06 Dec 2003 01:40:53 +1100")
References: <200312051440.hB5Eer1P009275@maxim.off.ekorp.com>
Message-ID: <4qwft2h3.fsf@python.net>

While building and trying out the windows installer yesterday, I had
some crashes in pydocgui.pyw (the module documentation server), both on
win98SE and on XP Pro.  The crash ocurred when it was closed, and it seems
only when the test suite was also installed (usually it also finds
something in the test suite).  Win98 was unhelpful as usual in only
showing the top of the machine stack, winXP displayed a message box
showing an access violation in TclNotifier.

I were not able to debug this, because I couldn't get Tkinter to work in
the version running from the build dir, and I had no more time - I
assume there are some env vars to be set so that it finds the tcl and tk
dlls.

I could not reproduce the problem with the final built which is no out
in the cold, so I assume it was a temporary problem.  But it would be
nice if other people could try this out as well before this comes back
later on me.

Thanks,

Thomas


From Jack.Jansen at cwi.nl  Fri Dec  5 18:17:15 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Dec  5 18:17:21 2003
Subject: [Python-Dev] Constants from C headers and int/long unification
Message-ID: <29CBBE16-2779-11D8-BAB7-000A27B19B96@cwi.nl>

The discussions on unification of int and long have now gone on for so 
many months that I no longer know how to handle them for the following 
scenario, please enlighten me.

I have C header files which I parse and to which I can do some simple 
processing before outputting them as Python code. An example C file 
will contain

enum {
    kConstantOne = 0xffffffff,
    kConstantTwo = -1,
};

Regexps will grab the two constants and output them to Python. The 
Python programmer will pass these constants to an API routine, at which 
point I need a PyArg_Parse format char that will convert both of these 
values to the 32 bit pattern 0xffffffff.

Question 1: Do I need to add anything to the constants before 
outputting them as Python code?
Question 2: Is there a format char that will convert both of the values 
produced by the answer to question 1 to the 32 bit pattern 0xffffffff?
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From guido at python.org  Fri Dec  5 18:24:05 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec  5 18:24:13 2003
Subject: [Python-Dev] Constants from C headers and int/long unification
In-Reply-To: Your message of "Sat, 06 Dec 2003 00:17:15 +0100."
	<29CBBE16-2779-11D8-BAB7-000A27B19B96@cwi.nl> 
References: <29CBBE16-2779-11D8-BAB7-000A27B19B96@cwi.nl> 
Message-ID: <200312052324.hB5NO5o09561@c-24-5-183-134.client.comcast.net>

> I have C header files which I parse and to which I can do some simple 
> processing before outputting them as Python code. An example C file 
> will contain
> 
> enum {
>     kConstantOne = 0xffffffff,
>     kConstantTwo = -1,
> };
> 
> Regexps will grab the two constants and output them to Python. The 
> Python programmer will pass these constants to an API routine, at which 
> point I need a PyArg_Parse format char that will convert both of these 
> values to the 32 bit pattern 0xffffffff.
> 
> Question 1: Do I need to add anything to the constants before 
> outputting them as Python code?

No -- kConstantTwo automatically become a long with value 2**32-1 in
Python 2.3 and up.

> Question 2: Is there a format char that will convert both of the values 
> produced by the answer to question 1 to the 32 bit pattern 0xffffffff?

If you mean a PyArg_Parse format char, yes: use 'k'.

If you meant a Python % format char, you'd have to write "%x" % (x&0xffffffff).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From anthony at ekit-inc.com  Fri Dec  5 22:39:39 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Fri Dec  5 22:39:59 2003
Subject: [Python-Dev] source-level backward compatibility requirements 
In-Reply-To: Message from Guido van Rossum <guido@python.org> of "Fri,
	05 Dec 2003 09:19:11 -0800."
	<200312051719.hB5HJBp08708@c-24-5-183-134.client.comcast.net> 
Message-ID: <200312060339.hB63dd8F022401@maxim.off.ekorp.com>

>>> Guido van Rossum wrote
> > Does this warrant a c2 release?  I'm happy to make the changes right
> > now.
> Tht's a question for Anthony.  I suppose there will be a c2 release
> anyway.

I hadn't planned a c2 release - the c2 release would be only for 
brown-paper-bag kinda bugs. I'm not sure that merely removing a 
warning for apply() really merits a whole nother release candidate,
but I'm willing to be convinced otherwise.

From anthony at ekit-inc.com  Fri Dec  5 22:41:54 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Fri Dec  5 22:42:07 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.292.10.3, 2.292.10.4 
In-Reply-To: Message from "Fred L. Drake, Jr." <fdrake@acm.org> of "Fri,
	05 Dec 2003 15:02:54 CDT."
	<16336.58478.552434.65123@grendel.fdrake.net> 
Message-ID: <200312060341.hB63fsJO022461@maxim.off.ekorp.com>

>>> "Fred L. Drake, Jr." wrote
> 
> Raymond Hettinger writes:
>  > FYI, I have a few more doc fixes for Py2.3.3.
>  > Anthony asked me to wait until Sunday to put them in.
> 
> Any particular reason for the delay?

In case there was a packaging bug in the rc1 release, and I wanted
to cut a new one. 



From tim.one at comcast.net  Sat Dec  6 00:00:08 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec  6 00:00:13 2003
Subject: [Python-Dev] source-level backward compatibility requirements 
In-Reply-To: <200312060339.hB63dd8F022401@maxim.off.ekorp.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEJNHJAB.tim.one@comcast.net>

[Anthony Baxter]
> I hadn't planned a c2 release - the c2 release would be only for
> brown-paper-bag kinda bugs. I'm not sure that merely removing a
> warning for apply() really merits a whole nother release candidate,
> but I'm willing to be convinced otherwise.

I won't be trying to convince you:  I reviewed Fred's checkin, and it's not
going to break anything.  Of course, if it does, Fred will be in really big
trouble, not me <wink>.


From skip at pobox.com  Sat Dec  6 12:48:19 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Dec  6 12:48:18 2003
Subject: [Python-Dev] test_unicode_file failing on Mac OS X
Message-ID: <16338.5731.362994.497561@montanaro.dyndns.org>


Two of the test_unicode_file began failing on my Mac today (fresh cvs up, OS
X 10.2.8, vanilla unix-style build):

    ======================================================================
    FAIL: test_directories (__main__.TestUnicodeFiles)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "../Lib/test/test_unicode_file.py", line 155, in test_directories
        self._do_directory(TESTFN_ENCODED+ext, TESTFN_ENCODED+ext, os.getcwd)
      File "../Lib/test/test_unicode_file.py", line 103, in _do_directory
        make_name)
    AssertionError: '@test-a\xcc\x80o\xcc\x80.dir' != '@test-\xc3\xa0\xc3\xb2.dir'

    ======================================================================
    FAIL: test_single_files (__main__.TestUnicodeFiles)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "../Lib/test/test_unicode_file.py", line 141, in test_single_files
        self._test_single(TESTFN_ENCODED)
      File "../Lib/test/test_unicode_file.py", line 116, in _test_single
        self._do_single(filename)
      File "../Lib/test/test_unicode_file.py", line 41, in _do_single
        self.failUnless(base in os.listdir(path))
    AssertionError

Skip

From raymond.hettinger at verizon.net  Sat Dec  6 15:54:21 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat Dec  6 15:54:27 2003
Subject: [Python-Dev] list.sorted()
Message-ID: <005201c3bc3b$20069320$e841fea9@oemcomputer>

When working on itertools.groupby(), the tutorial updates, and
exercising the newer features, I've made frequent use of list.sorted().

The good news is that after using it many times, I no longer hate the
name ;-)  Also, it fits well inside list comps (and is expected to do
the same with gen exps) plus it works well with itertools.  In fact, it
is a delight to use.  One surprise use case was how nicely it works with
unittest.TestCase.assertEqual() when verifying that two collections are
equal with respect to permutation.

The bad news is that the list dot prefix adds no value, feels awkward,
and interferes with the eye's ability to parse expressions containing
list.sorted().  

Part of the reason for making it a classmethod was that it *is*
effectively an alternative constructor for lists.  OTOH, that could be
said for any function that returns a list.

The two disadvantages of having it as a classmethod are the mandatory
list dot prefix and that it is confusing when called with an existing
list:  

  [f(elem, arg) for elem in [3,2,1].sorted(anotherseq)]

It's up to someone else to propose the obvious solutions -- I've already
gone over my quota and am running low on asbestos ;-)

Whatever the answer, it's important that the result look clean when used
in gen exps or as an argument to functions expecting a sequence.


Raymond Hettinger


From python at rcn.com  Sat Dec  6 17:32:35 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sat Dec  6 17:32:43 2003
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Doc/lib
	libitertools.tex, 1.23, 1.24
In-Reply-To: <E1ASkvp-00064m-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <005d01c3bc48$d8d4d580$e841fea9@oemcomputer>


Thank you.


Raymond


From bac at OCF.Berkeley.EDU  Sat Dec  6 17:32:37 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Sat Dec  6 17:32:49 2003
Subject: [Python-Dev] python-dev Summary for 2003-11-16 through 2003-11-30
	[rough draft]
Message-ID: <3FD25905.4010003@ocf.berkeley.edu>

OK, you guys know the drill.

I plan on sending this out no later than the end of next week.

-----------------------------


python-dev Summary for 2003-11-16 through 2003-11-30
++++++++++++++++++++++++++++++++++++++++++++++++++++
This is a summary of traffic on the `python-dev mailing list`_ from 
November 16, 2003 through November 30, 2003.  It is intended to inform 
the wider Python community of on-going developments on the list.  To 
comment on anything mentioned here, just post to `comp.lang.python`_ (or 
email python-list@python.org which is a gateway to the newsgroup) with a 
subject line mentioning what you are discussing. All python-dev members 
are interested in seeing ideas discussed by the community, so don't 
hesitate to take a stance on something.  And if all of this really 
interests you then get involved and join `python-dev`_!

This is the thirtieth summary written by Brett Cannon (you like me!  You 
really like me!  =).

To contact me, please send email to brett at python.org ; I do not have 
the time to keep up on comp.lang.python and thus do not always catch 
follow-ups posted there.

All summaries are archived at http://www.python.org/dev/summary/ .

Please note that this summary is written using reStructuredText_ which 
can be found at http://docutils.sf.net/rst.html .  Any unfamiliar 
punctuation is probably markup for reST_ (otherwise it is probably 
regular expression syntax or a typo =); you can safely ignore it, 
although I suggest learning reST; it's simple and is accepted for `PEP 
markup`_ and gives some perks for the HTML output.  Also, because of the 
wonders of programs that like to reformat text, I cannot guarantee you 
will be able to run the text version of this summary through Docutils_ 
as-is unless it is from the original text file.

.. _PEP Markup: http://www.python.org/peps/pep-0012.html

The in-development version of the documentation for Python can be found 
at http://www.python.org/dev/doc/devel/ and should be used when looking 
up any documentation on something mentioned here.  PEPs (Python 
Enhancement Proposals) are located at http://www.python.org/peps/ .  To 
view files in the Python CVS online, go to 
http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/ .  Reported bugs 
and suggested patches can be found at the SourceForge_ project page.

.. _python-dev: http://www.python.org/dev/
.. _SourceForge: http://sourceforge.net/tracker/?group_id=5470
.. _python-dev mailing list: 
http://mail.python.org/mailman/listinfo/python-dev
.. _comp.lang.python: http://groups.google.com/groups?q=comp.lang.python
.. _Docutils: http://docutils.sf.net/
.. _reST:
.. _reStructuredText: http://docutils.sf.net/rst.html

.. contents::

.. _last summary: 
http://www.python.org/dev/summary/2003-10-16_2003-11-15.html


=====================
Summary Announcements
=====================
Wow, I actually managed to get a summary out less than a week when its 
coverage dates ended.  Perks of procrastinating for studying for finals.  =)

First, an errata on the last summary.  I said generator expressions were 
Peter Norvig's idea.  Turns out it was Raymond Hettinger, the man who 
has so many new ideas that his flame retardant suit has his initials 
embroidered on it, who came up with the original idea in `PEP 289`_.

PyCon_ registration has gone live!  Go to the page to register.  And if 
you don't we will have a possessed Barry Warsaw come after you (to get 
the reference, go to http://www.pycon.org/ and take a look at the banner 
graphic; Barry is the second from the right)!  =)

The system for accepting proposals for PyCon is still being worked on. 
It should be up in the very near future.  Since it was not up in time 
for the original deadline, the new proposal deadline has been extended.

This summary is brought to you by the `iTunes Music Store`_ and the 
following songs from the `Verve Remixed 2 - Exclusive EP 
<http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewAlbum?playlistId=2611976>`__ 
album (which are linked to the iTMS and thus require iTunes to be 
installed):
- `Manteca, by Dizzy Gillespie & Funky Lowlives 
<http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewAlbum?playlistId=2611976&selectedItemId=2556313>`__
- `Sinnerman, by Felix da Housecat & Nina Simone 
<http://phobos.apple.com/WebObjects/MZStore.woa/wa/viewAlbum?playlistId=2611976&selectedItemId=2556338>`__

.. _PEP 289: http://www.python.org/peps/pep-0289.html
.. _PyCon: http://www.pycon.org/
.. _iTunes Music Store: http://www.apple.com/itunes/store/


=========
Summaries
=========
--------------------------------------------------
Aggregate refcounting is not always a hard science
--------------------------------------------------
Christian Tismer discovered that the aggregate refcount stored in 
_Py_RefTotal (only available in a Py_REF_DEBUG build which is part of a 
debug build; see Misc/SpecialBuilds.txt for all the special builds that 
are available) can be wrong when code starts to do funky things in 
__del__ methods and manipulating the refcount of objects directly.  The 
offending code has been fixed in the 2.3.x branch and up.


Contributing threads:
   - `_Py_RefTotal wrong in Py 2.2.3 
<http://mail.python.org/pipermail/python-dev/2003-November/040269.html>`__


------------------------------------------------
Bulk up reversed() with __reversed__ and __len__
------------------------------------------------
When the new 'reversed' built-in was being discussed, the idea of having 
a __reversed__ magic method for it was proposed.  But because it might 
get misused by applying 'reversed' to iterators that were infinite 
(think generators and itertools iterators), Guido had it removed.

But Raymond Hettinger, who came up with 'reversed' in the first place, 
came up with a modified idea; only have 'reversed' use __reversed__ if 
both that magic method and __len__ are defined.  That way 'reversed' 
will only try to reverse its argument if it has a known bounded length 
(as of right now it only works if the object has __getitem__ and __len__ 
defined).

As of right now it has not been pronounced upon.

Contributing threads:
   - ` __reversed__ protocol 
<http://mail.python.org/pipermail/python-dev/2003-November/040308.html>`__


------------------------------------------------------
Change in object.__setattr__ semantics from 2.2 to 2.3
------------------------------------------------------
A change in how object.__setattr__ works went unmentioned in the 'What's 
New...' doc for 2.3 by accident.  In case you have not run into this, 
you cannot use object.__setattr__ if a derived type defines __setattr__ 
itself.  It is not an issue if you just use the built-in setattr.

Contributing threads:
   - `2.2=>2.3 object.__setattr__(cls,attr,value) 
<http://mail.python.org/pipermail/python-dev/2003-November/040323.html>`__


-----------------------------
Say goodbye to MacOS9 support
-----------------------------
Starting with 2.4, Python will not support MacOS9.  Jack Jansen has 
ripped out the support so that all Mac-related efforts can focus on OS X 
work.

This does not affect any previous versions of Python.

Contributing threads:
   - `Ripping out Macintosh support 
<http://mail.python.org/pipermail/python-dev/2003-November/040336.html>`__


------------------
2.3.3 in the works
------------------
2.3.3c1 has been released as of this writing.  You can find it at 
http://www.python.org/2.3.3/ .  As usual, please download it and run the 
test suite along with your own code.

Contributing threads:
   - `Time for 2.3.3? 
<http://mail.python.org/pipermail/python-dev/2003-November/040343.html>`__


--------------------
Project ideas galore
--------------------
Last month I asked for ideas for a masters thesis.  This led to a huge 
response with various project ideas.  After going through them all I 
posted an annotated list of all the ideas presented to me (see the 
contributing thread for the email link).  If you are ever bored for say, 
a month or more, you can try to tackle one of these projects to keep 
yourself occupied.

Contributing threads:
   - `Thesis ideas list 
<http://mail.python.org/pipermail/python-dev/2003-November/040387.html>`__


-------------------------
No more hex/oct warnings!
-------------------------
The warnings from 2.3 about how hex/oct constants were going to change 
in 2.4 are now gone since the change has been checked into CVS.  `PEP 
237`_, which specified this change, also mentioned warnings of the 
change in 2.4 along with changes to repr for longs.

The warnings are not going to exist in Python 2.4 .  It was felt the 
warnings in 2.3 were visible and noisy enough to get the point across.

As for the repr change for longs, that is not going to happen.  The repr 
of a long will continue to have an "L" appended to the end of it.

.. _PEP 237: http://www.python.org/peps/pep_0237.html

Contributing threads:
   - `Int FutureWarnings and other 2.4 TODOs 
<http://mail.python.org/pipermail/python-dev/2003-November/040411.html>`__


------------------------------
Backticks to go bye-bye in 3.0
------------------------------
Consider yourselves warned; backticks will be removed in Python 3.0 .

Contributing threads:
   - `Banishing apply(), buffer(), coerce(), and intern() 
<http://mail.python.org/pipermail/python-dev/2003-November/040433.html>`__


-----------------------
itertools grows groupby
-----------------------
itertools.groupby takes an iterable and a key-accessing function and 
returns a tuple of the key and an iterator containing items that match 
the current key.  Read the documentation for a more thorough explanation.

Contributing threads:
   - `"groupby" iterator 
<http://mail.python.org/pipermail/python-dev/2003-November/040469.html>`__


From fincher.8 at osu.edu  Sat Dec  6 19:00:31 2003
From: fincher.8 at osu.edu (Jeremy Fincher)
Date: Sat Dec  6 18:02:39 2003
Subject: [Python-Dev] list.sorted()
In-Reply-To: <005201c3bc3b$20069320$e841fea9@oemcomputer>
References: <005201c3bc3b$20069320$e841fea9@oemcomputer>
Message-ID: <200312061900.31919.fincher.8@osu.edu>

On Saturday 06 December 2003 03:54 pm, Raymond Hettinger wrote:
> It's up to someone else to propose the obvious solutions -- I've already
> gone over my quota and am running low on asbestos ;-)

I remember someone, somewhere (comp.lang.python, I think) proposing a 
"consumers" module for the new ideas people are having now that generator 
expressions will be unleashed in 2.4.  A "sorted" consumer would seem to fit 
right in, I would think.

Jeremy

From oussoren at cistron.nl  Sun Dec  7 00:35:22 2003
From: oussoren at cistron.nl (Ronald Oussoren)
Date: Sun Dec  7 00:35:26 2003
Subject: [Python-Dev] list.sorted()
In-Reply-To: <005201c3bc3b$20069320$e841fea9@oemcomputer>
References: <005201c3bc3b$20069320$e841fea9@oemcomputer>
Message-ID: <26D718AD-2877-11D8-BB3A-0003931CFE24@cistron.nl>


On 6 dec 2003, at 21:54, Raymond Hettinger wrote:

> The two disadvantages of having it as a classmethod are the mandatory
> list dot prefix and that it is confusing when called with an existing
> list:
>
>   [f(elem, arg) for elem in [3,2,1].sorted(anotherseq)]
>

The latter problem is a generic problem of class methods, e.g. 
someDict.fromKeys(seq). One way to fix this would be to make sure that 
class methods are no longer accessible through instances.

Ronald


From raymond.hettinger at verizon.net  Sun Dec  7 07:38:43 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec  7 07:39:43 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
Message-ID: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>

I think it would be worthwhile to occasionally (every 3 months or so)
package a Py2.4 pre-alpha release. My feel is that a number of people
without compilers (Windows users especially) would enjoy working with
the latest python if it were an easy thing to do (has an installer,
etc).
 
Besides increasing community involvement, this could open up a whole new
stream of user feedback so we can discover issues sooner rather than
later.  Since non-developers stress the system in different ways, they
are more likely to surface various documentation, usability, and
integration bugs.
 
An opening message could be added that clearly states that everything is
subject to change, under development, non-production, etc.
 
 
 
Raymond Hettinger
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031207/47a7d9e1/attachment-0001.html
From skip at manatee.mojam.com  Sun Dec  7 08:00:52 2003
From: skip at manatee.mojam.com (Skip Montanaro)
Date: Sun Dec  7 08:01:01 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200312071300.hB7D0qvo023090@manatee.mojam.com>


Bug/Patch Summary
-----------------

593 open / 4407 total bugs (+50)
217 open / 2492 total patches (+24)

New Bugs
--------

Can't send Apple Events without WindowServer connection (2003-12-01)
	http://python.org/sf/852150
add support for cjkcodecs to Python email (2003-12-01)
	http://python.org/sf/852347
^$ won't split on empty line (2003-12-02)
	http://python.org/sf/852532
bz2:compress empty string inconsistent (2003-12-02)
	http://python.org/sf/853061
After fork in subthread, signals are blocked (2003-12-03)
	http://python.org/sf/853411
IP6 address parsing i sgmllib (2003-12-03)
	http://python.org/sf/853506
socket.recv() raises MemoryError exception (WindowsXP ONLY) (2003-12-03)
	http://python.org/sf/853507
new _Launch module includes useless deprecated functions (2003-12-03)
	http://python.org/sf/853558
Carbon.CF.CFURLRef should be easier to create (2003-12-03)
	http://python.org/sf/853656
No documentation for zipimport module (2003-12-03)
	http://python.org/sf/853800
'Demo/embed/' , It crash (2003-12-03)
	http://python.org/sf/853862
Message.get_filename() problem (2003-12-04)
	http://python.org/sf/854102
Python-2.3.3c1, Solaris 2.7: socketmodule does not compile (2003-12-05)
	http://python.org/sf/854823

New Patches
-----------

urllib2 CacheFTPHandler doesn't work on multiple dirs (2003-11-30)
	http://python.org/sf/851736
urllib2 silently returns None when auth_uri is mismatched (2003-11-30)
	http://python.org/sf/851752
bugfix for [Bugs-850964](optparse) (2003-11-30)
	http://python.org/sf/851902
Refactored patch #815911 for logging with SocketHandler (2003-12-01)
	http://python.org/sf/851993
broken url in colorsys documentation (2003-12-01)
	http://python.org/sf/852236
Replace backticks with repr() (2003-12-01)
	http://python.org/sf/852334
tests and processors patch for urllib2 (2003-12-02)
	http://python.org/sf/852995
Tix hlist missing entry_configure and entry_cget methods (2003-12-03)
	http://python.org/sf/853488
Optional keyword unicode args not handled correctly (2003-12-04)
	http://python.org/sf/853890
Implement lazy read for sockets (2003-12-04)
	http://python.org/sf/853963
ConfigParser should raise an error for duplicate options (2003-12-04)
	http://python.org/sf/854484
shutil.copyfile destroys hard links (Bug [ 851123 ]) (2003-12-05)
	http://python.org/sf/854853
fix typos (2003-12-05)
	http://python.org/sf/855195

Closed Bugs
-----------

os.strerror doesn't understand windows error codes (2003-10-14)
	http://python.org/sf/823672
Windows installer 2.3.2 leaves old version in control panel (2003-11-05)
	http://python.org/sf/836515
Typos in the docs (Extending/Embedding + Python/C API) (2003-11-09)
	http://python.org/sf/838938
tkFileDialog.Open is broken (2003-11-17)
	http://python.org/sf/843999
"up" instead of "down" in turtle module documentation (2003-11-17)
	http://python.org/sf/844123
couple of new list.sort bugs (2003-11-25)
	http://python.org/sf/848856
winreg can segfault (2003-11-28)
	http://python.org/sf/851056
Item out of order on builtin function page (2003-11-29)
	http://python.org/sf/851152
Bug tracker page asks for login even when logged in (2003-11-29)
	http://python.org/sf/851156
New-style classes with __eq__ but not __hash__ are hashable (2003-11-29)
	http://python.org/sf/851449

Closed Patches
--------------

Better output for unittest (2003-04-16)
	http://python.org/sf/722638
whichdb not ID'ing dbm files with GDBM backend (2003-07-06)
	http://python.org/sf/766650
unittest's main don't handle TestSuite in command line (2003-09-10)
	http://python.org/sf/804212

From aahz at pythoncraft.com  Sun Dec  7 08:29:59 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sun Dec  7 08:30:03 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
Message-ID: <20031207132959.GA26733@panix.com>

On Sun, Dec 07, 2003, Raymond Hettinger wrote:
>
> I think it would be worthwhile to occasionally (every 3 months or so)
> package a Py2.4 pre-alpha release. My feel is that a number of people
> without compilers (Windows users especially) would enjoy working with
> the latest python if it were an easy thing to do (has an installer,
> etc).
>  
> Besides increasing community involvement, this could open up a whole new
> stream of user feedback so we can discover issues sooner rather than
> later.  Since non-developers stress the system in different ways, they
> are more likely to surface various documentation, usability, and
> integration bugs.

Let's see if I can channel Tim correctly: "The number of people who have
historically downloaded beta and candidate releases indicates that it's
unlikely that people will download pre-alpha releases."

OTOH, David Goodger has been reasonably successful with docutils having
snapshots.  It's certainly a worthwhile question about whether people
would be more likely to download beta and candidate releases if snapshots
were regularly available.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From perky at i18n.org  Sun Dec  7 09:36:20 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Sun Dec  7 09:36:25 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <20031207132959.GA26733@panix.com>
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com>
Message-ID: <20031207143620.GA32893@i18n.org>

On Sun, Dec 07, 2003 at 08:29:59AM -0500, Aahz wrote:
> On Sun, Dec 07, 2003, Raymond Hettinger wrote:
> >
> > I think it would be worthwhile to occasionally (every 3 months or so)
> > package a Py2.4 pre-alpha release. My feel is that a number of people
> > without compilers (Windows users especially) would enjoy working with
> > the latest python if it were an easy thing to do (has an installer,
> > etc).
> >  
> > Besides increasing community involvement, this could open up a whole new
> > stream of user feedback so we can discover issues sooner rather than
> > later.  Since non-developers stress the system in different ways, they
> > are more likely to surface various documentation, usability, and
> > integration bugs.
> 
> Let's see if I can channel Tim correctly: "The number of people who have
> historically downloaded beta and candidate releases indicates that it's
> unlikely that people will download pre-alpha releases."

JFYI, FreeBSD distributes its own unofficial snapshot of Python 2.4.
http://www.freshports.org/lang/python-devel/

Looking at download statistics, it seems that about 60~80 users
install FreeBSD's python-devel snapshot for every updates. (the
count can be much more because downloads from many mirror sites
isn't counted.)


Hye-Shik

From fredrik at pythonware.com  Sun Dec  7 09:29:58 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun Dec  7 09:50:23 2003
Subject: [Python-Dev] Re: Py2.4 pre-alpha snapshot
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com>
Message-ID: <bqvdhj$g9f$1@sea.gmane.org>

Aahz wrote:

> OTOH, David Goodger has been reasonably successful with docutils having
> snapshots.

well, that's not that strange, given that the docutils page says that
the latest official release is 18 months old [1], and is named 0.3, and
that users should avoid using official releases.

for some reason, I'm not sure that's the right approach for Python.

</F>

1) that appears to be a typo, though.  the effbot's 89'th law of
software releases: if you don't release stuff often enough to notice
a typo on your release page in 6 months, you're not releasing stuff
often enough ;-)




From pje at telecommunity.com  Sun Dec  7 10:37:05 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Dec  7 10:35:03 2003
Subject: [Python-Dev] Where to report PyPI problem?
Message-ID: <5.1.0.14.0.20031207103226.03d88bd0@mail.telecommunity.com>

I was updating some release information today, and got this traceback:

Traceback (most recent call last):
   File "/usr/local/pypi/lib/pypi/webui.py", line 99, in run
     self.inner_run()
   File "/usr/local/pypi/lib/pypi/webui.py", line 374, in inner_run
     getattr(self, action)()
   File "/usr/local/pypi/lib/pypi/webui.py", line 1209, in pkg_edit
     info = reldict[ver]
KeyError: 0.9.2%20%28bug%20fix%20release%29

It looks rather like PyPI's not decoding URL-escaped field names.  I'd 
submit a patch, but I'm not sure where to find the source.  I poked around 
the nondist tree in CVS, but didn't see it.


From aahz at pythoncraft.com  Sun Dec  7 10:58:36 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sun Dec  7 10:58:39 2003
Subject: [Python-Dev] Where to report PyPI problem?
In-Reply-To: <5.1.0.14.0.20031207103226.03d88bd0@mail.telecommunity.com>
References: <5.1.0.14.0.20031207103226.03d88bd0@mail.telecommunity.com>
Message-ID: <20031207155836.GB6153@panix.com>

On Sun, Dec 07, 2003, Phillip J. Eby wrote:
>
> I was updating some release information today, and got this traceback:
> 
> Traceback (most recent call last):
>   File "/usr/local/pypi/lib/pypi/webui.py", line 99, in run
>     self.inner_run()
>   File "/usr/local/pypi/lib/pypi/webui.py", line 374, in inner_run
>     getattr(self, action)()
>   File "/usr/local/pypi/lib/pypi/webui.py", line 1209, in pkg_edit
>     info = reldict[ver]
> KeyError: 0.9.2%20%28bug%20fix%20release%29
> 
> It looks rather like PyPI's not decoding URL-escaped field names.  I'd 
> submit a patch, but I'm not sure where to find the source.  I poked around 
> the nondist tree in CVS, but didn't see it.

Didja see the "Contact Us" section on http://www.python.org/pypi
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From Jack.Jansen at cwi.nl  Sun Dec  7 11:32:23 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Sun Dec  7 11:32:30 2003
Subject: [Python-Dev] test_unicode_file failing on Mac OS X
In-Reply-To: <16338.5731.362994.497561@montanaro.dyndns.org>
References: <16338.5731.362994.497561@montanaro.dyndns.org>
Message-ID: <EF939B2E-28D2-11D8-B88E-000A27B19B96@cwi.nl>


On 6-dec-03, at 18:48, Skip Montanaro wrote:

> Two of the test_unicode_file began failing on my Mac today (fresh cvs 
> up, OS
> X 10.2.8, vanilla unix-style build):
>
>     
> ======================================================================
>     FAIL: test_directories (__main__.TestUnicodeFiles)
>     
> ----------------------------------------------------------------------
>     Traceback (most recent call last):
>       File "../Lib/test/test_unicode_file.py", line 155, in 
> test_directories
>         self._do_directory(TESTFN_ENCODED+ext, TESTFN_ENCODED+ext, 
> os.getcwd)
>       File "../Lib/test/test_unicode_file.py", line 103, in 
> _do_directory
>         make_name)
>     AssertionError: '@test-a\xcc\x80o\xcc\x80.dir' != 
> '@test-\xc3\xa0\xc3\xb2.dir'

This is probably related to the two flavors of unicode there are, one 
which prefers to have all accents separately from the letters as much 
as possible and one which prefers the reverse. I keep forgetting the 
names of the two, they're somewhat silly.

But the problem is that Python prefers to represent the string "?" as 
the two characters "a" and "umlaut on the previous char", and MacOSX 
prefers to represent the same string as "a with umlaut on it". Or the 
other way around, this is something else I always forget.

And while there are algorithms to convert the combined form of unicode 
to the uncombined form and vice versa there are no Python codecs to do 
this. The OSX system calls do the right thing (convert both forms to 
what it prefers), but when you do a readdir() you don't get the string 
back you put it.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From pf_moore at yahoo.co.uk  Sun Dec  7 11:57:26 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Dec  7 11:57:12 2003
Subject: [Python-Dev] Re: Py2.4 pre-alpha snapshot
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
Message-ID: <vfosva4p.fsf@yahoo.co.uk>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> I think it would be worthwhile to occasionally (every 3 months or so)
> package a Py2.4 pre-alpha release. My feel is that a number of people
> without compilers (Windows users especially) would enjoy working with
> the latest python if it were an easy thing to do (has an installer,
> etc).

Speaking as a "latest release" junkie with not enough time to build
for myself, I'd say it might be nice. But the killer is that you need
extensions. On Windows, for instance, you'd need to prevail on Mark
Hammond to produce 2.4-compatible win32all builds (pretty much
essential on Windows, and a serious pain to build by hand, I believe).

Also, you'd need an installer which would sit happily alongside the
production version (install in a separate directory, include a
python24.exe so as not to clash in terms of command names, etc).

> Besides increasing community involvement, this could open up a whole new
> stream of user feedback so we can discover issues sooner rather than
> later.  Since non-developers stress the system in different ways, they
> are more likely to surface various documentation, usability, and
> integration bugs.

A compromise may just be a simple zipped up compile. That helps people
without compilers, but leaves the effort of "installing" to the user.
It still doesn't help with the extensions issue, though...

Paul.
-- 
This signature intentionally left blank


From kbk at shore.net  Sun Dec  7 12:14:42 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sun Dec  7 12:14:46 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <20031207132959.GA26733@panix.com> (Aahz's message of "Sun, 7
	Dec 2003 08:29:59 -0500")
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com>
Message-ID: <m3u14cwnwd.fsf@float.localdomain>

Aahz <aahz@pythoncraft.com> writes:

> On Sun, Dec 07, 2003, Raymond Hettinger wrote:
>>
>> I think it would be worthwhile to occasionally (every 3 months or so)
>> package a Py2.4 pre-alpha release. My feel is that a number of people
>> without compilers (Windows users especially) would enjoy working with
>> the latest python if it were an easy thing to do (has an installer,
>> etc).
>>  
>> Besides increasing community involvement, this could open up a whole new
>> stream of user feedback so we can discover issues sooner rather than
>> later.  Since non-developers stress the system in different ways, they
>> are more likely to surface various documentation, usability, and
>> integration bugs.
>
> Let's see if I can channel Tim correctly: "The number of people who have
> historically downloaded beta and candidate releases indicates that it's
> unlikely that people will download pre-alpha releases."
>
> OTOH, David Goodger has been reasonably successful with docutils having
> snapshots.  It's certainly a worthwhile question about whether people
> would be more likely to download beta and candidate releases if snapshots
> were regularly available.

In my case, I can no longer compile Python on Windows (VC5 worked up
to 2.3).  So to test IDLE against 2.4 on that platform, I need a
Windows installer.

As I recollect, there was discussion about moving to VC7 which has
a free compiler available.  Is that going to happen on 2.4?

-- 
KBK

From pedronis at bluewin.ch  Sun Dec  7 12:43:42 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Sun Dec  7 12:40:34 2003
Subject: [Python-Dev] source-level backward compatibility
  requirements
In-Reply-To: <200312051654.hB5Gs9e08643@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Fri, 05 Dec 2003 11:22:06 EST."
	<16336.45230.749132.305648@grendel.fdrake.net>
	<200312051523.hB5FNJTU010018@maxim.off.ekorp.com>
	<16900000.1070638518@leeloo.intern.geht.de>
	<16336.44564.604153.644072@grendel.fdrake.net>
	<200312051615.hB5GFvW08552@c-24-5-183-134.client.comcast.net>
	<16336.45230.749132.305648@grendel.fdrake.net>
Message-ID: <5.2.1.1.0.20031207183758.02fd16c0@pop.bluewin.ch>

At 08:54 05.12.2003 -0800, Guido van Rossum wrote:
> > On the other hand, I find I'm not really a fan of deprecating apply()
> > in the first place, so I can think of other solutions.  ;-)
>
>We could deprecate it in the docs but remove the silent warning which
>slows it down.  Then we could leave it in the logging module until
>3.0.

Btw, maybe this is known to everybody, but the python layers of SWIG 
produced Python
bindings still use apply all over the place.



From martin at v.loewis.de  Sun Dec  7 12:47:37 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Dec  7 12:47:55 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
Message-ID: <m3ekvgtt8m.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> I think it would be worthwhile to occasionally (every 3 months or so)
> package a Py2.4 pre-alpha release. My feel is that a number of people
> without compilers (Windows users especially) would enjoy working with
> the latest python if it were an easy thing to do (has an installer,
> etc).

Anybody could do that, without any official blessing. For example, I'm
distributing Itanium binaries at my own leisure, and I plan to provide
Itanium binaries for 2.4-in-progress, once I complete fixing the
pending issues.

So if you want to provide snapshots, just go ahead. If you need
hyperlinks from Python.org, or disk space on python.org: that can be
provided.

OTOH, I would consider it a wastage of resources if the regular
packagers would put any effort into snapshots (unless they really want
to do this on their own).

Regards,
Martin

From martin at v.loewis.de  Sun Dec  7 12:50:08 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Dec  7 12:50:23 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <m3u14cwnwd.fsf@float.localdomain>
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com> <m3u14cwnwd.fsf@float.localdomain>
Message-ID: <m38ylott4f.fsf@mira.informatik.hu-berlin.de>

kbk@shore.net (Kurt B. Kaiser) writes:

> As I recollect, there was discussion about moving to VC7 which has
> a free compiler available.  Is that going to happen on 2.4?

Yes. If nobody beats me in that, I'll do it over Christmas.

Regards,
Martin

From martin at v.loewis.de  Sun Dec  7 12:56:54 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Dec  7 12:57:28 2003
Subject: [Python-Dev] test_unicode_file failing on Mac OS X
In-Reply-To: <EF939B2E-28D2-11D8-B88E-000A27B19B96@cwi.nl>
References: <16338.5731.362994.497561@montanaro.dyndns.org>
	<EF939B2E-28D2-11D8-B88E-000A27B19B96@cwi.nl>
Message-ID: <m34qwctst5.fsf@mira.informatik.hu-berlin.de>

Jack Jansen <Jack.Jansen@cwi.nl> writes:

> This is probably related to the two flavors of unicode there are, one
> which prefers to have all accents separately from the letters as much
> as possible and one which prefers the reverse. I keep forgetting the
> names of the two, they're somewhat silly.

OS X uses what is called the "decomposed normal form", splitting
combined characters into the base character and the combining accent.

Python supports either form, but will use precomposed characters more
often than not.

> And while there are algorithms to convert the combined form of unicode
> to the uncombined form and vice versa there are no Python codecs to do
> this. 

Not as a codec, but as unicodedata.normalize. If you do

unicodedata.normalize(composed_string, "NFD")

you get the string that OS X wants you to use.

Of course, with Unicode-on-Windows, the story is mostly vice-versa.
NTFS/Win32 does not perform any normalization, so you can actually
store the precomposed and the decomposed string simultaneously in the
same directory (which is confusing). The platform codecs always
generate the precomposed form, though, so you are more likely to find
the precomposed form on disk.

For the test, it would be best to compare normal forms, and have the
test pass if the normal forms (NFD) are equal.

Regards,
Martin

From pje at telecommunity.com  Sun Dec  7 13:26:03 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sun Dec  7 13:24:04 2003
Subject: [Python-Dev] Where to report PyPI problem?
In-Reply-To: <20031207155836.GB6153@panix.com>
References: <5.1.0.14.0.20031207103226.03d88bd0@mail.telecommunity.com>
	<5.1.0.14.0.20031207103226.03d88bd0@mail.telecommunity.com>
Message-ID: <5.1.0.14.0.20031207132430.03e25bc0@mail.telecommunity.com>

At 10:58 AM 12/7/03 -0500, Aahz wrote:
>On Sun, Dec 07, 2003, Phillip J. Eby wrote:
> >
> > I was updating some release information today, and got this traceback:
> >
> > Traceback (most recent call last):
> >   File "/usr/local/pypi/lib/pypi/webui.py", line 99, in run
> >     self.inner_run()
> >   File "/usr/local/pypi/lib/pypi/webui.py", line 374, in inner_run
> >     getattr(self, action)()
> >   File "/usr/local/pypi/lib/pypi/webui.py", line 1209, in pkg_edit
> >     info = reldict[ver]
> > KeyError: 0.9.2%20%28bug%20fix%20release%29
> >
> > It looks rather like PyPI's not decoding URL-escaped field names.  I'd
> > submit a patch, but I'm not sure where to find the source.  I poked around
> > the nondist tree in CVS, but didn't see it.
>
>Didja see the "Contact Us" section on http://www.python.org/pypi

Aha.  When I had hovered the "Bug Reports" link and saw a SourceForge link, 
I assumed it was the general Python bug reports page, not 
PyPI-specific.  Sorry about that.  I'll pop in a bug report there.  Thanks.


From theller at python.net  Sun Dec  7 14:46:27 2003
From: theller at python.net (Thomas Heller)
Date: Sun Dec  7 14:46:44 2003
Subject: [Python-Dev] Re: Py2.4 pre-alpha snapshot
In-Reply-To: <vfosva4p.fsf@yahoo.co.uk> (Paul Moore's message of "Sun, 07
	Dec 2003 16:57:26 +0000")
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<vfosva4p.fsf@yahoo.co.uk>
Message-ID: <k758wgvg.fsf@python.net>

Paul Moore <pf_moore@yahoo.co.uk> writes:

> "Raymond Hettinger" <raymond.hettinger@verizon.net> writes:
>
>> I think it would be worthwhile to occasionally (every 3 months or so)
>> package a Py2.4 pre-alpha release. My feel is that a number of people
>> without compilers (Windows users especially) would enjoy working with
>> the latest python if it were an easy thing to do (has an installer,
>> etc).
>
> Speaking as a "latest release" junkie with not enough time to build
> for myself, I'd say it might be nice. But the killer is that you need
> extensions. On Windows, for instance, you'd need to prevail on Mark
> Hammond to produce 2.4-compatible win32all builds (pretty much
> essential on Windows, and a serious pain to build by hand, I believe).

It has been a pain, but not any longer.  win32all can now be built with
a distutils setup script - although there is no official release built
with that.

> Also, you'd need an installer which would sit happily alongside the
> production version (install in a separate directory, include a
> python24.exe so as not to clash in terms of command names, etc).

There's no problem having different major Python versions installed on
one machine, even if there are *also* CVS builds.  It can all be hidden
in batch scripts calling python.exe in different directories.

>> Besides increasing community involvement, this could open up a whole new
>> stream of user feedback so we can discover issues sooner rather than
>> later.  Since non-developers stress the system in different ways, they
>> are more likely to surface various documentation, usability, and
>> integration bugs.
>
> A compromise may just be a simple zipped up compile. That helps people
> without compilers, but leaves the effort of "installing" to the user.
> It still doesn't help with the extensions issue, though...

Then there's the possibility to include the compiled files (even for
different python versions, if you don't care too much about the download
size) in a source distribution prepared by distutils.

Thomas


From eppstein at ics.uci.edu  Sun Dec  7 14:46:57 2003
From: eppstein at ics.uci.edu (David Eppstein)
Date: Sun Dec  7 14:47:01 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
References: <16338.5731.362994.497561@montanaro.dyndns.org>
	<EF939B2E-28D2-11D8-B88E-000A27B19B96@cwi.nl>
	<m34qwctst5.fsf@mira.informatik.hu-berlin.de>
Message-ID: <eppstein-CE0812.11465707122003@sea.gmane.org>

In article <m34qwctst5.fsf@mira.informatik.hu-berlin.de>,
 martin@v.loewis.de (Martin v. Lowis) wrote:

> For the test, it would be best to compare normal forms, and have the
> test pass if the normal forms (NFD) are equal.

Shouldn't that be what happens in general for equality testing of 
unicodes, not just for this test?

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From pf_moore at yahoo.co.uk  Sun Dec  7 15:51:39 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sun Dec  7 15:51:37 2003
Subject: [Python-Dev] Re: Py2.4 pre-alpha snapshot
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com>
	<m3u14cwnwd.fsf@float.localdomain>
	<m38ylott4f.fsf@mira.informatik.hu-berlin.de>
Message-ID: <wu98uzac.fsf@yahoo.co.uk>

martin@v.loewis.de (Martin v. L?wis) writes:

> kbk@shore.net (Kurt B. Kaiser) writes:
>
>> As I recollect, there was discussion about moving to VC7 which has
>> a free compiler available.  Is that going to happen on 2.4?
>
> Yes. If nobody beats me in that, I'll do it over Christmas.

What exactly does that imply? Will VC6 no longer work, or will you
just add extra build files etc for VC7?

How will the free VC7 work (I assume Kurt means the one in the .NET
SDK)? AFAIK, it doesn't support Visual Studio project files. It also
doesn't include an optimizer, what effect will that have on build
compatibility, etc?

Paul.
-- 
This signature intentionally left blank


From kbk at shore.net  Sun Dec  7 17:53:28 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Sun Dec  7 17:53:31 2003
Subject: [Python-Dev] Re: Py2.4 pre-alpha snapshot
In-Reply-To: <wu98uzac.fsf@yahoo.co.uk> (Paul Moore's message of "Sun, 07
	Dec 2003 20:51:39 +0000")
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com> <m3u14cwnwd.fsf@float.localdomain>
	<m38ylott4f.fsf@mira.informatik.hu-berlin.de>
	<wu98uzac.fsf@yahoo.co.uk>
Message-ID: <m33cbww87r.fsf@float.localdomain>

Paul Moore <pf_moore@yahoo.co.uk> writes:

> How will the free VC7 work (I assume Kurt means the one in the .NET
> SDK)? AFAIK, it doesn't support Visual Studio project files. It also
> doesn't include an optimizer, what effect will that have on build
> compatibility, etc?

Here are the pertinent messages from last spring:

-- 
KBK

=====================================================================
From: Guido van Rossum <guido@python.org>
Subject: Re: [Python-Dev] MS VC 7 offer
To: phil@riverbankcomputing.co.uk
cc: python-dev@python.org
Date: Tue, 06 May 2003 14:55:15 -0400

> How do we get hold of the free VC 7 compilers?

Here's the info Nick sent me:

| We offer as part of the .NET Framework SDK each of the compilers that
| comprise our Visual Studio tool - including C++.  The caveat here is
| that we don't yet ship the full CRT or STL with this distribution -
| this will be changing.  Also, the 64bit C++ compilers ship for free as
| part of the Windows Platform SDK.  All of this is available on
| msdn.microsoft.com.
[...]
| Here are the links to the SDKs.  But so you aren't surprised, these are
| NOT low-overhead downloads or installs...
| 
| .NET Framework 1.1
| 
| http://www.msdn.microsoft.com/netframework/downloads/howtoget.aspx
| 
| Platform SDK
| 
| http://msdn.microsoft.com/library/default.asp?url=/library/en-us/sdkintro/sdkintro/obtaining_the_complete_sdk.asp

--Guido van Rossum (home page: http://www.python.org/~guido/)

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev

From: Tim Peters <tim.one@comcast.net>
Subject: RE: [Python-Dev] MS VC 7 offer
To: phil@riverbankcomputing.co.uk
Cc: python-dev@python.org
Date: Tue, 06 May 2003 14:56:52 -0400

[Phil Thompson]
> How do we get hold of the free VC 7 compilers?

Part of the 100+ MB .NET Framework 1.1 SDK:

http://www.msdn.microsoft.com/netframework/downloads/howtoget.aspx

Note that this requires Win2K minimum.

_______________________________________________
Python-Dev mailing list
Python-Dev@python.org
http://mail.python.org/mailman/listinfo/python-dev

From: Chad Netzer <cnetzer@mail.arc.nasa.gov>
Subject: Re: [Python-Dev] MS VC 7 offer
To: python-dev@python.org
Date: 06 May 2003 15:37:30 -0700
Organization: 

On Tue, 2003-05-06 at 15:06, Guido van Rossum wrote:

> Part of the offer was:
> 
> | Potentially we can even figure out how to enable anyone to
> | build Python using the freely downloadable compilers I mentioned
> | above...

Which would seem to exclude building on Win98 machines (or WinME
*snort*, or even Win NT 4).  Those platforms still have a huge installed
base, and I would assume a not insignificant developer base.

Is offering a MSVC6 version along with a more recent compiler version an
option?

-- 

Chad Netzer
(any opinion expressed is my own and not NASA's or my employer's)

===============================================================================

From tdelaney at avaya.com  Sun Dec  7 17:57:18 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun Dec  7 17:57:26 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEF5FF10@au3010avexu1.global.avaya.com>

> From: Aahz
> 
> Hmmmm...  How important is the "less memory" angle versus the "faster"
> angle?  Why not just add a few bytes to the long type to store an int?
> If you're using the int, the pointer is NULL, and you're saving that
> memory, at least.  The only harm is some extra memory in the type
> object.

One thing I can immediately think of is that you're likely to see cache effects much sooner. In that case, you'll probably find that you've ended up with higher memory usage *and* lower performance. The best of both worlds :)

Tim Delaney

From greg at cosc.canterbury.ac.nz  Sun Dec  7 19:05:58 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Dec  7 19:06:08 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <2mfzfzwmk2.fsf@starship.python.net>
Message-ID: <200312080005.hB805wS08108@oma.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> As things stand, I don't think cPickle *can* be careful enough.
> Perhaps we could just add a "name" field to CObjects and have module
> init functions check that.

That would certainly be a huge improvement over the current
situation!

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun Dec  7 19:37:06 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Dec  7 19:37:11 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
Message-ID: <200312080037.hB80b6Q08173@oma.cosc.canterbury.ac.nz>

Guido:

> And the docs should be updated to explain the description better
> (currently at one point the description is called "extra callback data
> for the destructor function" which seems a rather odd feature).

Indeed! That comment completely misled me about the purpose of
the "desc". Ironically, if that sentence simply hadn't been 
there, I probably would have correctly guessed what the desc
was meant for.

Another API suggestion: a function such as

  void* PyCObject_AsVoidPtrWithDesc(PyObject* self, void *expected_desc)

which checks the desc and raises an exception if it doesn't
match the expected value.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Sun Dec  7 19:43:37 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Dec  7 19:43:42 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <2m7k1bwb6a.fsf@starship.python.net>
Message-ID: <200312080043.hB80hba08180@oma.cosc.canterbury.ac.nz>

Michael Hudson <mwh@python.net>:

> Why a void* then?  That's particularly unassertive.  char* would be
> better, surely...

A string isn't the only thing that could usefully be used
as a descriptor.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From jjl at pobox.com  Sun Dec  7 19:51:00 2003
From: jjl at pobox.com (John J Lee)
Date: Sun Dec  7 19:51:07 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
Message-ID: <Pine.LNX.4.58.0312072353060.11373@alice>

First, it seems the standard place for functional tests is in

if __name__ == "__main__"

right?

Some (most?) of the standard library functional tests seem to be a bunch
of commands that are simply supposed not to crash (urllib2) or that are
little test programs that need to be invoked with the correct arguments
(ftplib).

Would it be a good thing to attempt to PyUnit-ize these functional tests
so you can run them with no command line arguments and get clear
indications of how many tests are passing and failing?  Or, if not PyUnit,
just ad-hoc tests that indicate success / failure?

Should some be left intact and new functional tests added?  For example,
ftplib has a little test program tacked on at its end, perhaps that should
be retained in some form?


John

From raymond.hettinger at verizon.net  Sun Dec  7 19:59:26 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec  7 19:59:32 2003
Subject: [Python-Dev] New developer
Message-ID: <001601c3bd26$877e15a0$0705a044@oemcomputer>

Please welcome Hye-Shik Chang as a new python developer.

He hails from Seoul, Korea and has interests in implementing generator
expressions and in i18n projects such as CJK codecs.  Most recently, he
has worked on the code and tests implementing itertools.groupby().


Raymond Hettinger


His Bio (slightly edited):
"""
I work for a linux company for a military service replacement. (Korean
military service duty is 2 years but we can work for defense industry or
IT company for 3 years instead.) My company sells AAA and DHCP solutions
for Korean major ISPs. They are all based on python. And even the ISPs'
operators use python interactive interpreter as a command line
interface.

I'm a commit-privileged developer on FreeBSD project since 2002. I
maintain package infrastructure for python stuffs named "bsd.python.mk",
the python package itself and FreeBSD packages for 80+ python 3rd party
modules. (But it's easy thank to distutils.)

After starting to use python on my company, I'm distributing python
extension modules I developed as opensource licensed as possible:
CJKCodecs, py-rrdtool, py-freebsd, py-fibheap, coolwater and iconvcodec.

"""


From aahz at pythoncraft.com  Sun Dec  7 20:04:54 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sun Dec  7 20:04:56 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEF5FF10@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF5FF10@au3010avexu1.global.avaya.com>
Message-ID: <20031208010454.GB19111@panix.com>

On Mon, Dec 08, 2003, Delaney, Timothy C (Timothy) wrote:
> From: Aahz
>> 
>> Hmmmm...  How important is the "less memory" angle versus the "faster"
>> angle?  Why not just add a few bytes to the long type to store an int?
>> If you're using the int, the pointer is NULL, and you're saving that
>> memory, at least.  The only harm is some extra memory in the type
>> object.
> 
> One thing I can immediately think of is that you're likely to see
> cache effects much sooner. In that case, you'll probably find that
> you've ended up with higher memory usage *and* lower performance. The
> best of both worlds :)

While there's generally some truth to that, I have a suspicion that the
tricks we'll need to perform in order hide the int/long dichotomy at the
Python level will also be cache-unfriendly.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From python at rcn.com  Sun Dec  7 20:19:09 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sun Dec  7 20:19:16 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
In-Reply-To: <Pine.LNX.4.58.0312072353060.11373@alice>
Message-ID: <001701c3bd29$4878f020$0705a044@oemcomputer>

> First, it seems the standard place for functional tests is in
> 
> if __name__ == "__main__"
> 
> right?
> 
> Some (most?) of the standard library functional tests seem to be a
bunch
> of commands that are simply supposed not to crash (urllib2) or that
are
> little test programs that need to be invoked with the correct
arguments
> (ftplib).
> 
> Would it be a good thing to attempt to PyUnit-ize these functional
tests
> so you can run them with no command line arguments and get clear
> indications of how many tests are passing and failing?  Or, if not
PyUnit,
> just ad-hoc tests that indicate success / failure?
> 
> Should some be left intact and new functional tests added?  For
example,
> ftplib has a little test program tacked on at its end, perhaps that
should
> be retained in some form?


Ideally, all repeatable, self verifying tests should be coded using the
unittest module and placed in Lib/test.

Walter Dorwald is actively working on converting tests to unittest form.
At the same time, he is running a coverage analysis tool and adding
tests to increase coverage.  Also, he is factoring out some tests that
are common to multiple modules.  If you want to assist, I recommend
working with him directly and having him give a 100% review to any
proposed changes.  For an example of that work, see:
www.python.org/sf/736962


Some of tests in the module itself are being left intact for various
reasons.  For instance, the random module still has a few tests that
need a human eye to authenticate.  Some, like the ones in urllib2 are
normally only run when someone updates the module -- they should be left
in for convenience but could also have a counterpart in Lib/test so long
as the test were marked to run only when -u network was enabled.  As you
pointed out, some are run interactively using command line arguments --
these are a development tool and should be left in.  Also, some modules
need to be left alone because they are maintained elsewhere.


Raymond Hettinger


From anthony at ekit-inc.com  Mon Dec  8 00:01:37 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Mon Dec  8 00:01:51 2003
Subject: [Python-Dev] PEP0004: audioop deprecated?
Message-ID: <200312080501.hB851bP7000438@maxim.off.ekorp.com>



According to PEP-0004 and PEP-0320, the audioop module is deprecated,
and scheduled for removal in 2.4. I hadn't noticed this before, but
I'd like to ask that it be kept. I've a number of pieces of code using
it - both internal (our unified messaging server makes heavy use of
it) and external (the shtoom python SIP phone uses it).

Anthony

From guido at python.org  Mon Dec  8 00:08:04 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec  8 00:08:11 2003
Subject: [Python-Dev] Int FutureWarnings and other 2.4 TODOs
In-Reply-To: Your message of "Sun, 07 Dec 2003 20:04:54 EST."
	<20031208010454.GB19111@panix.com> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF5FF10@au3010avexu1.global.avaya.com>
	<20031208010454.GB19111@panix.com> 
Message-ID: <200312080508.hB8584E23041@c-24-5-183-134.client.comcast.net>

I've figured the whole int/short/long thing out now.  Jack may be
happy to know that it happened on a long bike ride. :-)

There will be two classes, and they can both be user-visible, but
users will generally deal in terms of the class called 'int', which is
the base class.  It is called 'int', but its implementation is what we
currently call 'long'.  The subclass will be called 'short',
implemented as what we currently call 'int'.  The 'short' class will
not be subclassable.  The 'int' class can tell the difference between
the two representations by looking at the class pointer -- if it's
'short' then the short representation applies.  Both will
automatically return an object of the other class if the size of the
return value requires/allows it.  The new 'short', the current 'int',
already does this -- the new 'int', currently 'long' will have to be
adapted.  But (unlike 'short') it doesn't have to be perfect...  (If
'int' is subclassed, it may return an instance of the subclass always,
to make subclassing int more useful -- "contagiousness" helps in this
case I think.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From sjoerd at acm.org  Mon Dec  8 04:42:15 2003
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Mon Dec  8 04:42:19 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/PCbuild
 make_versioninfo.dsp, 1.1.2.1, 1.1.2.2
In-Reply-To: <E1ATHbx-0007h5-00@sc8-pr-cvs1.sourceforge.net>
References: <E1ATHbx-0007h5-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <3FD44777.2090107@acm.org>

theller@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/PCbuild
> In directory sc8-pr-cvs1:/tmp/cvs-serv29575
> 
> Modified Files:
>       Tag: release23-maint
> 	make_versioninfo.dsp 
> Log Message:
> Sigh.  dsp files must be marked as binary files for cvs, otherwise
> MSVC isn't able to read them.  Thanks to David Rushby.

Actually, this is not entirely true.
They must be marked with -ko, i.e. CVS keywords must not be expanded. 
But it is ok if they are treated as text files.

-- 
Sjoerd Mullender <sjoerd@acm.org>

From theller at python.net  Mon Dec  8 04:59:35 2003
From: theller at python.net (Thomas Heller)
Date: Mon Dec  8 04:59:47 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/PCbuild
	make_versioninfo.dsp, 1.1.2.1, 1.1.2.2
In-Reply-To: <3FD44777.2090107@acm.org> (Sjoerd Mullender's message of "Mon,
	08 Dec 2003 10:42:15 +0100")
References: <E1ATHbx-0007h5-00@sc8-pr-cvs1.sourceforge.net>
	<3FD44777.2090107@acm.org>
Message-ID: <iskreik8.fsf@python.net>

Sjoerd Mullender <sjoerd@acm.org> writes:

> theller@users.sourceforge.net wrote:
>> Update of /cvsroot/python/python/dist/src/PCbuild
>> In directory sc8-pr-cvs1:/tmp/cvs-serv29575
>> Modified Files:
>>       Tag: release23-maint
>> 	make_versioninfo.dsp Log Message:
>> Sigh.  dsp files must be marked as binary files for cvs, otherwise
>> MSVC isn't able to read them.  Thanks to David Rushby.
>
> Actually, this is not entirely true.
> They must be marked with -ko, i.e. CVS keywords must not be
> expanded. But it is ok if they are treated as text files.

That may be, but the real problem is that MSVC refuses to load them if
they have unix line endings.  And the source tarball is built on a unix
like system.

Thomas


From theller at python.net  Mon Dec  8 05:01:34 2003
From: theller at python.net (Thomas Heller)
Date: Mon Dec  8 05:01:40 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <m38ylott4f.fsf@mira.informatik.hu-berlin.de> (Martin v.'s
	message of "07 Dec 2003 18:50:08 +0100")
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com> <m3u14cwnwd.fsf@float.localdomain>
	<m38ylott4f.fsf@mira.informatik.hu-berlin.de>
Message-ID: <d6azeigx.fsf@python.net>

martin@v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) writes:

> kbk@shore.net (Kurt B. Kaiser) writes:
>
>> As I recollect, there was discussion about moving to VC7 which has
>> a free compiler available.  Is that going to happen on 2.4?
>
> Yes. If nobody beats me in that, I'll do it over Christmas.

How's your msi builder going?  I would be interested to try it out -
although I probably don't have too much time over Christmas.

Thomas


From mwh at python.net  Mon Dec  8 05:02:32 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec  8 05:02:36 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <200312080043.hB80hba08180@oma.cosc.canterbury.ac.nz> (Greg
	Ewing's message of "Mon, 08 Dec 2003 13:43:37 +1300 (NZDT)")
References: <200312080043.hB80hba08180@oma.cosc.canterbury.ac.nz>
Message-ID: <2mk757tyo7.fsf@starship.python.net>

Greg Ewing <greg@cosc.canterbury.ac.nz> writes:

> Michael Hudson <mwh@python.net>:
>
>> Why a void* then?  That's particularly unassertive.  char* would be
>> better, surely...
>
> A string isn't the only thing that could usefully be used
> as a descriptor.

What else would you consider useful?  I'm only really aware of the
most straightforward uses of CObjects (and wonder if the advanced uses
actually see much use...).

Cheers,
mhw

-- 
  Just point your web browser at http://www.python.org/search/ and
  look for "program", "doesn't", "work", or "my". Whenever you find
  someone else whose program didn't work, don't do what they
  did. Repeat as needed.    -- Tim Peters, on python-help, 16 Jun 1998

From jim at zope.com  Mon Dec  8 05:03:05 2003
From: jim at zope.com (Jim Fulton)
Date: Mon Dec  8 05:03:52 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>
	<2mfzfzwmk2.fsf@starship.python.net>
	<200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
Message-ID: <3FD44C59.1070701@zope.com>

Guido van Rossum wrote:
>>>Indeed, CObjects seem fundamentally dangerous to me, unless
>>>the modules which create and use them are extremely careful
>>>to make sure they can tell whether they've got the right
>>>kind of CObject.
>>>
>>
>>Well, the right *CObject* -- it's CObject identity that matters.
> 
> 
> I seem to recall that the CObject design contains a feature to avoid
> using the wrong CObject, but it's not used by any of the CObject
> implementations in the standard library, and the CObject docs don't
> describe how it's meant to use (they were clearly reverse-engineered
> from the code).
> 
> Several early checkin messages of cobject.c mention Jim Fulton as the
> author, so maybe we could ask him. 

Yup, I'm the one who proposed CObject.

 > (I'm cc'ing Jim in case he
> remembers the true intention for the description; he might also know
> whether Zope 2 still uses it;  I can only check Zope 3, and it
> doesn't.)

What "it" are you refering to.

I don't really remember what the description was for. I also
don't have the original email.

> But even without Jim's confirmation, ISTM from memory and looking at
> the source that the 'desc' field (which stands for description)

I now hate abbreviations like that btw. :)

 > was
> intended as a clue that could be checked by the user of a C Object to
> verify it had gotten the right one: in a shared header (needed anyway
> to describe the layout of the struct pointer contained in the C
> Object) you could define a magic word or a magic string value that
> would be stored as the description, and the users would check that
> value before believing the struct pointer.

I vaguely remember that such a clue was one possible application
of the extra data.  I really imagined that the destructor could need the
extra data, but, again, it's been a long time.  Too bad there's not a PEP. :)

> Unfortunately the same CObject version that introduced the description
> field also introduced the convenience function PyCObject_Import(),
> which doesn't return the description, and returns the void pointer
> contained in the CObject without returning the CObject itself, leaving
> its caller without a clue about the description...

That's because I don't think that the description was primarily intended
as a fingerprint.

> Anyway, I think it can be fixed by starting to use the description
> field and adding a new convenience C API, perhaps named
> PyCObject_ImportEx(), whch takes a description and checks it.  We
> should try to standardize on what kind of thing to use as the
> description -- if we use a magic word, PyCObject_ImportEx() should do
> a simple compare, but if we use a string, it should do a strcmp().  Or
> the API could simply return the description into an output parameter,
> making the caller responsible for doing the checking.
> 
> And the docs should be updated to explain the description better
> (currently at one point the description is called "extra callback data
> for the destructor function" which seems a rather odd feature).

I like the idea of adding this use of the description.  That is,
the description is really a marker, that is obtained from an include file
and checked against the value in the CObject.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org


From mwh at python.net  Mon Dec  8 05:30:06 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec  8 05:30:10 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <3FD44C59.1070701@zope.com> (Jim Fulton's message of "Mon, 08
	Dec 2003 05:03:05 -0500")
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>
	<2mfzfzwmk2.fsf@starship.python.net>
	<200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
	<3FD44C59.1070701@zope.com>
Message-ID: <2mk757sitt.fsf@starship.python.net>

Jim Fulton <jim@zope.com> writes:

>  > (I'm cc'ing Jim in case he
>> remembers the true intention for the description; he might also know
>> whether Zope 2 still uses it;  I can only check Zope 3, and it
>> doesn't.)
>
> What "it" are you refering to.

The desc field in a CObject.

> I don't really remember what the description was for. I also
> don't have the original email.

http://mail.python.org/pipermail/python-dev/2003-December/040702.html

>> But even without Jim's confirmation, ISTM from memory and looking at
>> the source that the 'desc' field (which stands for description)
>
> I now hate abbreviations like that btw. :)

Good :-)

>  > was
>> intended as a clue that could be checked by the user of a C Object to
>> verify it had gotten the right one: in a shared header (needed anyway
>> to describe the layout of the struct pointer contained in the C
>> Object) you could define a magic word or a magic string value that
>> would be stored as the description, and the users would check that
>> value before believing the struct pointer.
>
> I vaguely remember that such a clue was one possible application
> of the extra data.  I really imagined that the destructor could need the
> extra data, but, again, it's been a long time.  Too bad there's not a PEP. :)
> 
>> Unfortunately the same CObject version that introduced the description
>> field also introduced the convenience function PyCObject_Import(),
>> which doesn't return the description, and returns the void pointer
>> contained in the CObject without returning the CObject itself, leaving
>> its caller without a clue about the description...
>
> That's because I don't think that the description was primarily intended
> as a fingerprint.

I feel a retcon coming on.

>> Anyway, I think it can be fixed by starting to use the description
>> field and adding a new convenience C API, perhaps named
>> PyCObject_ImportEx(), whch takes a description and checks it.  We
>> should try to standardize on what kind of thing to use as the
>> description -- if we use a magic word, PyCObject_ImportEx() should do
>> a simple compare, but if we use a string, it should do a strcmp().  Or
>> the API could simply return the description into an output parameter,
>> making the caller responsible for doing the checking.
>> And the docs should be updated to explain the description better
>> (currently at one point the description is called "extra callback data
>> for the destructor function" which seems a rather odd feature).
>
> I like the idea of adding this use of the description.  That is,
> the description is really a marker, that is obtained from an include file
> and checked against the value in the CObject.

Right.  What should we use as the description object?  A good old C
string?  Seems sensible enough to me, but Greg Ewing seemed to have
some other use in mind for it...

Cheers,
mwh

-- 
6. The code definitely is not portable - it will produce incorrect 
   results if run from the surface of Mars.
               -- James Bonfield, http://www.ioccc.org/2000/rince.hint

From fredrik at pythonware.com  Mon Dec  8 05:54:31 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Dec  8 05:54:39 2003
Subject: [Python-Dev] Re: are CObjects inherently unsafe?
References: <200312080043.hB80hba08180@oma.cosc.canterbury.ac.nz>
	<2mk757tyo7.fsf@starship.python.net>
Message-ID: <br1l9a$gf6$1@sea.gmane.org>

Michael Hudson wrote:

> > A string isn't the only thing that could usefully be used
> > as a descriptor.
>
> What else would you consider useful?  I'm only really aware of the
> most straightforward uses of CObjects (and wonder if the advanced uses
> actually see much use...).

the descriptor is passed to the destructor, so it can be used to hold data
associated with the object.

checking our repository, I cannot find a single example where I haven't
used the descriptor at all; the most common use is to point to a "MAGIC"
string, and there's one case where I've used it to keep a reference to a
PyObject that's related to the CObject pointer.

</F>




From fredrik at pythonware.com  Mon Dec  8 05:59:18 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Mon Dec  8 06:00:24 2003
Subject: [Python-Dev] Re: are CObjects inherently unsafe?
References: <200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>
	<200312080037.hB80b6Q08173@oma.cosc.canterbury.ac.nz>
Message-ID: <br1li9$gv5$1@sea.gmane.org>

Greg Ewing wrote:

> Another API suggestion: a function such as
>
>   void* PyCObject_AsVoidPtrWithDesc(PyObject* self, void *expected_desc)
>
> which checks the desc and raises an exception if it doesn't
> match the expected value.

which should probably be

    void* PyCObject_AsVoidPtrWithStringDesc(PyObject* self, const char *expected_desc)
        /* uses desc && !strcmp(desc, expected_desc), and can still crash... */

    void* PyCObject_AsVoidPtrWithPtrDesc(PyObject* self, void *expected_desc)
        /* uses desc == expected_desc */

</F>




From jim at zope.com  Mon Dec  8 07:00:32 2003
From: jim at zope.com (Jim Fulton)
Date: Mon Dec  8 07:01:20 2003
Subject: [Python-Dev] are CObjects inherently unsafe?
In-Reply-To: <2mk757sitt.fsf@starship.python.net>
References: <200312050056.hB50uh722511@oma.cosc.canterbury.ac.nz>	<2mfzfzwmk2.fsf@starship.python.net>	<200312051441.hB5EfKv08101@c-24-5-183-134.client.comcast.net>	<3FD44C59.1070701@zope.com>
	<2mk757sitt.fsf@starship.python.net>
Message-ID: <3FD467E0.4030508@zope.com>

Michael Hudson wrote:
> Jim Fulton <jim@zope.com> writes:
> 
> 
>> > (I'm cc'ing Jim in case he
>>
>>>remembers the true intention for the description; he might also know
>>>whether Zope 2 still uses it;  I can only check Zope 3, and it
>>>doesn't.)
>>
>>What "it" are you refering to.
> 
> 
> The desc field in a CObject.
> 
> 
>>I don't really remember what the description was for. I also
>>don't have the original email.
> 
> 
> http://mail.python.org/pipermail/python-dev/2003-December/040702.html
> 
> 
>>>But even without Jim's confirmation, ISTM from memory and looking at
>>>the source that the 'desc' field (which stands for description)
>>
>>I now hate abbreviations like that btw. :)
> 
> 
> Good :-)
> 
> 
>> > was
>>
>>>intended as a clue that could be checked by the user of a C Object to
>>>verify it had gotten the right one: in a shared header (needed anyway
>>>to describe the layout of the struct pointer contained in the C
>>>Object) you could define a magic word or a magic string value that
>>>would be stored as the description, and the users would check that
>>>value before believing the struct pointer.
>>
>>I vaguely remember that such a clue was one possible application
>>of the extra data.  I really imagined that the destructor could need the
>>extra data, but, again, it's been a long time.  Too bad there's not a PEP. :)
>>
>>
>>>Unfortunately the same CObject version that introduced the description
>>>field also introduced the convenience function PyCObject_Import(),
>>>which doesn't return the description, and returns the void pointer
>>>contained in the CObject without returning the CObject itself, leaving
>>>its caller without a clue about the description...
>>
>>That's because I don't think that the description was primarily intended
>>as a fingerprint.
> 
> 
> I feel a retcon coming on.
> 
> 
>>>Anyway, I think it can be fixed by starting to use the description
>>>field and adding a new convenience C API, perhaps named
>>>PyCObject_ImportEx(), whch takes a description and checks it.  We
>>>should try to standardize on what kind of thing to use as the
>>>description -- if we use a magic word, PyCObject_ImportEx() should do
>>>a simple compare, but if we use a string, it should do a strcmp().  Or
>>>the API could simply return the description into an output parameter,
>>>making the caller responsible for doing the checking.
>>>And the docs should be updated to explain the description better
>>>(currently at one point the description is called "extra callback data
>>>for the destructor function" which seems a rather odd feature).
>>
>>I like the idea of adding this use of the description.  That is,
>>the description is really a marker, that is obtained from an include file
>>and checked against the value in the CObject.
> 
> 
> Right.  What should we use as the description object?  A good old C
> string? 

Probably.  I don't have a strong opinion on it.

Jim

-- 
Jim Fulton           mailto:jim@zope.com       Python Powered!
CTO                  (540) 361-1714            http://www.python.org
Zope Corporation     http://www.zope.com       http://www.zope.org


From vinay_sajip at red-dove.com  Mon Dec  8 07:05:33 2003
From: vinay_sajip at red-dove.com (Vinay Sajip)
Date: Mon Dec  8 07:03:19 2003
Subject: [Python-Dev] source-level backward compatibility requirements -
	logging and apply()
Message-ID: <00f101c3bd83$96117dc0$652b6992@alpha>

Sorry that this post starts a new thread - I have python-dev on digest mode
and have not been following the list as closely as I ought to have done.

[Guido]
>> We could deprecate it in the docs but remove the silent warning which
>> slows it down.  Then we could leave it in the logging module until
>> 3.0.

[Fred]
>+42
>Does this warrant a c2 release?  I'm happy to make the changes right
>now.

+1 on deprecating apply() in the docs but not in the code. As Raymond
originally said, the only "problem" with apply() and coerce() is that while
they are useful, they are not useful in "mainstream" situations and could
use up learning bandwidth. However, I think apply() is a useful idiom to
have at one's disposal, though it only finds application here and there.

If Raymond's "all-builtins-have-at-least-one-friend" comment is not true for
apply(), I'd like to nominate myself as that friend :-)

Regards,

Vinay Sajip



From tim.one at comcast.net  Mon Dec  8 11:17:00 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec  8 11:16:59 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/PCbuild
	make_versioninfo.dsp, 1.1.2.1, 1.1.2.2
In-Reply-To: <3FD44777.2090107@acm.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEFFHKAB.tim.one@comcast.net>

>> Sigh.  dsp files must be marked as binary files for cvs, otherwise
>> MSVC isn't able to read them.  Thanks to David Rushby.

[Sjoerd Mullender, on Windows .dsp files]
> Actually, this is not entirely true.
> They must be marked with -ko, i.e. CVS keywords must not be expanded.
> But it is ok if they are treated as text files.

We used to, and that doesn't work -- someone ends up editing a .dsp file on
Linux then, or builds a distro from Linux, and the \r\n line ends get lost.
MSVC indeed does not tolerate a .dsp file with plain \n line ends.  Since
the exact line end characters used are crucial, it really is a binary file.


From martin at v.loewis.de  Mon Dec  8 12:46:14 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Dec  8 12:46:40 2003
Subject: [Python-Dev] Re: Py2.4 pre-alpha snapshot
In-Reply-To: <wu98uzac.fsf@yahoo.co.uk>
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com> <m3u14cwnwd.fsf@float.localdomain>
	<m38ylott4f.fsf@mira.informatik.hu-berlin.de>
	<wu98uzac.fsf@yahoo.co.uk>
Message-ID: <m3k757w6c9.fsf@mira.informatik.hu-berlin.de>

Paul Moore <pf_moore@yahoo.co.uk> writes:

> What exactly does that imply? Will VC6 no longer work, or will you
> just add extra build files etc for VC7?

I will remove the VC6 project files, and restore them in a different
place. They will become unmaintained.

> How will the free VC7 work (I assume Kurt means the one in the .NET
> SDK)? AFAIK, it doesn't support Visual Studio project files. It also
> doesn't include an optimizer, what effect will that have on build
> compatibility, etc?

I don't know, I don't use the VC7 SDK compiler. I cannot understand
your question on build compatibility, etc.

Regards,
Martin


From martin at v.loewis.de  Mon Dec  8 12:50:01 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Dec  8 12:54:22 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <d6azeigx.fsf@python.net>
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com> <m3u14cwnwd.fsf@float.localdomain>
	<m38ylott4f.fsf@mira.informatik.hu-berlin.de>
	<d6azeigx.fsf@python.net>
Message-ID: <m3fzfvw65y.fsf@mira.informatik.hu-berlin.de>

Thomas Heller <theller@python.net> writes:

> > Yes. If nobody beats me in that, I'll do it over Christmas.
> 
> How's your msi builder going?  I would be interested to try it out -
> although I probably don't have too much time over Christmas.

I got so far that I can initialize the schema, i.e. I have code that
takes the current schema, and dumps it into a Python source file,
which, when run, will restore the schema. This includes data for the
validation and install sequence tables.

I was next looking at including the actual files, and was planning to
supply a single CAB stream. I got stuck finding a CAB API, and plan to
invoke cabarc.exe instead to generate the CAB file.

This was a couple of weeks ago, I have made no progress since.

Regards,
Martin


From martin at v.loewis.de  Mon Dec  8 12:53:37 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Dec  8 12:55:05 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
In-Reply-To: <eppstein-CE0812.11465707122003@sea.gmane.org>
References: <16338.5731.362994.497561@montanaro.dyndns.org>
	<EF939B2E-28D2-11D8-B88E-000A27B19B96@cwi.nl>
	<m34qwctst5.fsf@mira.informatik.hu-berlin.de>
	<eppstein-CE0812.11465707122003@sea.gmane.org>
Message-ID: <m3brqjw5zy.fsf@mira.informatik.hu-berlin.de>

David Eppstein <eppstein@ics.uci.edu> writes:

> > For the test, it would be best to compare normal forms, and have the
> > test pass if the normal forms (NFD) are equal.
> 
> Shouldn't that be what happens in general for equality testing of 
> unicodes, not just for this test?

There was a BDFL pronouncement once that this should not be done
automatically, in general. Normalization is a very slow algorithm, and
it might not be meaningful in all cases.

E.g. XML 1.1 will require that all documents are in NFC. So the
XML-generating application will have to normalize on output; all
XML-processing applications can then assume that all strings are
normalized. Converting them to NFD all the time would be wasteful.

Python is still lacking an efficient test function to determine
whether a string is in normal form already; reportedly, a yes-no-maybe
function with a reasonably slow rate of maybe answers can be
implemented much more efficiently than performing the actual
conversion.

Regards,
Martin

From martin at v.loewis.de  Mon Dec  8 12:57:23 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Dec  8 12:58:55 2003
Subject: [Python-Dev] New developer
In-Reply-To: <001601c3bd26$877e15a0$0705a044@oemcomputer>
References: <001601c3bd26$877e15a0$0705a044@oemcomputer>
Message-ID: <m37k17w5to.fsf@mira.informatik.hu-berlin.de>

"Raymond Hettinger" <raymond.hettinger@verizon.net> writes:

> Please welcome Hye-Shik Chang as a new python developer.
> 
> He hails from Seoul, Korea and has interests in implementing generator
> expressions and in i18n projects such as CJK codecs.  Most recently, he
> has worked on the code and tests implementing itertools.groupby().

Does that mean we get CJKCodecs in 2.4 ?-)

(I'm actually serious here; I would welcome this addition)

Regards,
Martin

From barry at python.org  Mon Dec  8 13:37:48 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec  8 13:37:53 2003
Subject: [Python-Dev] New developer
In-Reply-To: <m37k17w5to.fsf@mira.informatik.hu-berlin.de>
References: <001601c3bd26$877e15a0$0705a044@oemcomputer>
	<m37k17w5to.fsf@mira.informatik.hu-berlin.de>
Message-ID: <1070908667.25704.17.camel@anthem>

On Mon, 2003-12-08 at 12:57, Martin v. L?wis wrote:

> Does that mean we get CJKCodecs in 2.4 ?-)
> 
> (I'm actually serious here; I would welcome this addition)

Me too!
-Barry



From perky at i18n.org  Mon Dec  8 14:35:46 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Mon Dec  8 14:35:52 2003
Subject: [Python-Dev] New developer
In-Reply-To: <m37k17w5to.fsf@mira.informatik.hu-berlin.de>
References: <001601c3bd26$877e15a0$0705a044@oemcomputer>
	<m37k17w5to.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031208193546.GA17282@i18n.org>

On Mon, Dec 08, 2003 at 06:57:23PM +0100, Martin v. L?wis wrote:
> "Raymond Hettinger" <raymond.hettinger@verizon.net> writes:
> 
> > Please welcome Hye-Shik Chang as a new python developer.
> > 
> > He hails from Seoul, Korea and has interests in implementing generator
> > expressions and in i18n projects such as CJK codecs.  Most recently, he
> > has worked on the code and tests implementing itertools.groupby().

Hello! I'm him. :)

As Raymond said, I have interests in implementing PEP 289 as well
as i18n stuffs such as CJK codecs or richer file.encoding support.
I'm currently doing some reviews for list.sort and str.rsplit
following Raymond's worth instructions.

> Does that mean we get CJKCodecs in 2.4 ?-)
> 
> (I'm actually serious here; I would welcome this addition)

I'd like. CJKCodecs got enough compatibility and standard conformance
to be merged into python to my thinking. And source size (500kB in
tar.gz) and installed binary size (1500kB on i386) may be reasonable
enough for most of people.


Hye-Shik

From walter at livinglogic.de  Mon Dec  8 18:16:26 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Mon Dec  8 18:16:43 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
In-Reply-To: <001701c3bd29$4878f020$0705a044@oemcomputer>
References: <001701c3bd29$4878f020$0705a044@oemcomputer>
Message-ID: <3FD5064A.7020201@livinglogic.de>

Raymond Hettinger wrote:

> [...]
> Ideally, all repeatable, self verifying tests should be coded using the
> unittest module and placed in Lib/test.
> 
> Walter Dorwald is actively working on converting tests to unittest form.
> At the same time, he is running a coverage analysis tool and adding
> tests to increase coverage.

BTW, the results of the coverage tests are available at:
http://coverage.livinglogic.de/
Currently updates are done once a month.

> Also, he is factoring out some tests that
> are common to multiple modules.

This not only reduces the number of tests that have to be written, but
it helps to keep similar APIs consistent.

> [...]

Bye,
    Walter D?rwald


From vinay_sajip at red-dove.com  Mon Dec  8 18:40:02 2003
From: vinay_sajip at red-dove.com (Vinay Sajip)
Date: Mon Dec  8 18:37:49 2003
Subject: [Python-Dev] Source-level backward compatibility requirements -
	logging and apply() - addendum
Message-ID: <002901c3bde4$9b069980$652b6992@alpha>

> If Raymond's "all-builtins-have-at-least-one-friend" comment is not true
for
> apply(), I'd like to nominate myself as that friend :-)

Raymond has very tactfully shown me the error of my ways, with respect to
the now-I-see-it-was-misplaced friendship with apply(). I hereby cut it
loose from me, and shall shed no tears when it departs in Python 3.0.

Regards,

Vinay


From fredrik at pythonware.com  Tue Dec  9 03:21:22 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Dec  9 03:21:34 2003
Subject: [Python-Dev] Re: Source-level backward compatibility requirements
	-logging and apply() - addendum
References: <002901c3bde4$9b069980$652b6992@alpha>
Message-ID: <br40m7$lgg$1@sea.gmane.org>

Vinay Sajip wrote_

> Raymond has very tactfully shown me the error of my ways, with respect to
> the now-I-see-it-was-misplaced friendship with apply(). I hereby cut it
> loose from me, and shall shed no tears when it departs in Python 3.0.

can python-dev perhaps provide a downloadable version of Raymond,
that I can bring to my Python training sessions?

in my experience, you always lose your audience when you introduce the
callable(*args, **kwargs) syntax, and you don't get them back until you
explain that the notion is apply(callable, args, kwargs).

whoever decided that the new calling syntax is "better" obviously didn't
spend many seconds on user testing.  a foolish consistency etc.

</F>




From theller at python.net  Tue Dec  9 06:41:44 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec  9 06:41:52 2003
Subject: [Python-Dev] Py2.4 pre-alpha snapshot
In-Reply-To: <m3fzfvw65y.fsf@mira.informatik.hu-berlin.de> (Martin v.'s
	message of "08 Dec 2003 18:50:01 +0100")
References: <000201c3bcbf$2d1c2320$f705a044@oemcomputer>
	<20031207132959.GA26733@panix.com> <m3u14cwnwd.fsf@float.localdomain>
	<m38ylott4f.fsf@mira.informatik.hu-berlin.de>
	<d6azeigx.fsf@python.net>
	<m3fzfvw65y.fsf@mira.informatik.hu-berlin.de>
Message-ID: <vfoqi5fr.fsf@python.net>

martin@v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=) writes:

> Thomas Heller <theller@python.net> writes:
>
>> > Yes. If nobody beats me in that, I'll do it over Christmas.
>> 
>> How's your msi builder going?  I would be interested to try it out -
>> although I probably don't have too much time over Christmas.
>
> I got so far that I can initialize the schema, i.e. I have code that
> takes the current schema, and dumps it into a Python source file,
> which, when run, will restore the schema. This includes data for the
> validation and install sequence tables.
>
> I was next looking at including the actual files, and was planning to
> supply a single CAB stream. I got stuck finding a CAB API, and plan to
> invoke cabarc.exe instead to generate the CAB file.

I hacked together a very simple cab creator using ctypes <wink>, maybe
you find it useful - there's also a link to the Cabinet SDK on the page,
which is not required but contains the docs (it is only 400kB or so).

http://starship.python.net/crew/theller/moin.cgi/CreateCab

> This was a couple of weeks ago, I have made no progress since.
>

Thomas


From smi at srasys.co.in  Tue Dec  9 07:28:46 2003
From: smi at srasys.co.in (Mythili Madhav)
Date: Tue Dec  9 07:26:48 2003
Subject: [Python-Dev] Help on MSAA
Message-ID: <002d01c3be4f$fe0f5f60$980110ac@3mts.com>

Hi,

I need to write a wrapper around MSAA in Python for having my own control
classes like MyApplication, MyWindow, MyPushButton etc which I think is
basically to subclass from MSAA's classes. I have downloaded pyAA but I
don't know how and from what to subclass. Has anybody worked on this? If
yes, kindly revert.

Thanks in advance,
SMI
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031209/7336d06c/attachment.html
From karthik19_20 at yahoo.com  Tue Dec  9 07:43:15 2003
From: karthik19_20 at yahoo.com (karthik karthik)
Date: Tue Dec  9 07:43:26 2003
Subject: [Python-Dev] help on MSAA
Message-ID: <20031209124315.94658.qmail@web11905.mail.yahoo.com>

hi!!

my task is to develop a wrapper to the MSAA using 
python. for this i'm developing a module. the class
design is:

class Wrapper:

    class Application:
        
    class Window:

    class PushButton:

    class EditBox:

in my main module i create an object for the Wrapper
class and when i say:

wrapper.Application, it should enable me to access the
application..

similarly if i say,
wrapper.PushButton, it should enable me to access the
pushbutton...

in other words its a generic class tht gives me the
respective handlers. i inherit this class and user for
specific purposes.

thank u.
sha. 


__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/

From aahz at pythoncraft.com  Tue Dec  9 08:03:51 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Dec  9 08:03:54 2003
Subject: [Python-Dev] help on MSAA
In-Reply-To: <20031209124315.94658.qmail@web11905.mail.yahoo.com>
References: <20031209124315.94658.qmail@web11905.mail.yahoo.com>
Message-ID: <20031209130351.GA22221@panix.com>

On Tue, Dec 09, 2003, karthik karthik wrote:
> 
> my task is to develop a wrapper to the MSAA using python. for this i'm
> developing a module. the class design is:

python-dev is for discussion of features and bugfixes for Python, not
creating programs in Python.  Please restrict future posts like this to
comp.lang.python (or the python-list gateway).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From Jack.Jansen at cwi.nl  Tue Dec  9 08:25:40 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue Dec  9 08:25:11 2003
Subject: [Python-Dev] Re: Source-level backward compatibility requirements
	-logging and apply() - addendum
In-Reply-To: <br40m7$lgg$1@sea.gmane.org>
References: <002901c3bde4$9b069980$652b6992@alpha> <br40m7$lgg$1@sea.gmane.org>
Message-ID: <2F1E9273-2A4B-11D8-A5D7-0030655234CE@cwi.nl>


On 9 Dec 2003, at 9:21, Fredrik Lundh wrote:
> in my experience, you always lose your audience when you introduce the
> callable(*args, **kwargs) syntax, and you don't get them back until you
> explain that the notion is apply(callable, args, kwargs).

This is a good point.

For most other language constructs that we have considered dropping the
new construct was more readable than the old one (think repr() versus 
backticks)
but for this one that isn't true: apply() is much more readable than 
the * and **
notation in some situations.

In other situations the * notation is much easier, I really don't want 
to
write callable(1, a=2, *args, **kwargs) in apply notation anymore. But 
still
apply is easy to understand for easy cases.
--
Jack Jansen        <Jack.Jansen@cwi.nl>        http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From fredrik at pythonware.com  Tue Dec  9 08:31:35 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Dec  9 08:31:47 2003
Subject: [Python-Dev] Re: Source-level backward compatibility
	requirements-logging and apply() - addendum
References: <002901c3bde4$9b069980$652b6992@alpha> <br40m7$lgg$1@sea.gmane.org>
Message-ID: <br4irr$nl1$1@sea.gmane.org>

> in my experience, you always lose your audience when you introduce the
> callable(*args, **kwargs) syntax, and you don't get them back until you
> explain that the notion is apply(callable, args, kwargs).

missing an "old" in there:

> explain that the old notion is apply(callable, args, kwargs).

</F>




From Scott.Daniels at Acm.Org  Wed Dec 10 08:28:46 2003
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Tue Dec  9 09:27:10 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
In-Reply-To: <E1AThE0-0003cc-I2@mail.python.org>
References: <E1AThE0-0003cc-I2@mail.python.org>
Message-ID: <3FD71F8E.6030307@Acm.Org>

martin@v.loewis.de (Martin v. L?wis) wrote:
> David Eppstein <eppstein@ics.uci.edu> writes:
>>>For the test, it would be best to compare normal forms, and have the
>>>test pass if the normal forms (NFD) are equal.
>>Shouldn't that be what happens in general for equality testing of 
>>unicodes, not just for this test?
> There was a BDFL pronouncement once that this should not be done
> automatically, in general. Normalization is a very slow algorithm, and
> it might not be meaningful in all cases.

Could we perhaps use a comparison that, in effect, did:

     def uni_equal(first, second):
         if first == second:
             return True
         return first.normalize() == second.normalize()

That is, take advantage of the fact that normalization is often
unnecessary for "trivial" reasons.

-Scott David Daniels
Scott.Daniels@Acm.Org

From jjl at pobox.com  Tue Dec  9 11:01:53 2003
From: jjl at pobox.com (John J Lee)
Date: Tue Dec  9 11:02:05 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
In-Reply-To: <001701c3bd29$4878f020$0705a044@oemcomputer>
References: <001701c3bd29$4878f020$0705a044@oemcomputer>
Message-ID: <Pine.LNX.4.58.0312080138140.11373@alice>

On Sun, 7 Dec 2003, Raymond Hettinger wrote:
[...]
> Ideally, all repeatable, self verifying tests should be coded using the
> unittest module and placed in Lib/test.
[...]
> Some of tests in the module itself are being left intact for various
> reasons.  For instance, the random module still has a few tests that
> need a human eye to authenticate.  Some, like the ones in urllib2 are
> normally only run when someone updates the module -- they should be left
> in for convenience but could also have a counterpart in Lib/test so long
> as the test were marked to run only when -u network was enabled.  As you

(Ah, hadn't seen the test runner script, so didn't know about -u network,
thanks.)

OK, so I can PyUnit-ize the urllib2 functional tests (but leave them there
so functional tests can conveniently be run separately), and link them to
Lib/test (so all self-verifying tests get run there).

One problem, though: doesn't putting functional tests in Lib/test throw
off Walter Dorwald's unit test coverage numbers?  Perhaps there should be
a 'functional' resource for test_support.use_resources (so the tests can
be run with -uall, -functional for coverage measurements)?

Thanks


John

From Jack.Jansen at cwi.nl  Tue Dec  9 11:31:34 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue Dec  9 11:31:44 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
In-Reply-To: <3FD71F8E.6030307@Acm.Org>
References: <E1AThE0-0003cc-I2@mail.python.org> <3FD71F8E.6030307@Acm.Org>
Message-ID: <275ADB90-2A65-11D8-BACE-000A27B19B96@cwi.nl>


On 10-dec-03, at 14:28, Scott David Daniels wrote:

> martin@v.loewis.de (Martin v. L?wis) wrote:
>> David Eppstein <eppstein@ics.uci.edu> writes:
>>>> For the test, it would be best to compare normal forms, and have the
>>>> test pass if the normal forms (NFD) are equal.
>>> Shouldn't that be what happens in general for equality testing of 
>>> unicodes, not just for this test?
>> There was a BDFL pronouncement once that this should not be done
>> automatically, in general. Normalization is a very slow algorithm, and
>> it might not be meaningful in all cases.
>
> Could we perhaps use a comparison that, in effect, did:
>
>     def uni_equal(first, second):
>         if first == second:
>             return True
>         return first.normalize() == second.normalize()
>
> That is, take advantage of the fact that normalization is often
> unnecessary for "trivial" reasons.

It helps, but only in 50% of the comparisons:-)
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From eppstein at ics.uci.edu  Tue Dec  9 12:06:03 2003
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Dec  9 12:06:06 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
References: <E1AThE0-0003cc-I2@mail.python.org> <3FD71F8E.6030307@Acm.Org>
	<275ADB90-2A65-11D8-BACE-000A27B19B96@cwi.nl>
Message-ID: <eppstein-AA8550.09060309122003@sea.gmane.org>

In article <275ADB90-2A65-11D8-BACE-000A27B19B96@cwi.nl>,
 Jack Jansen <Jack.Jansen@cwi.nl> wrote:

> > Could we perhaps use a comparison that, in effect, did:
> >
> >     def uni_equal(first, second):
> >         if first == second:
> >             return True
> >         return first.normalize() == second.normalize()
> >
> > That is, take advantage of the fact that normalization is often
> > unnecessary for "trivial" reasons.
> 
> It helps, but only in 50% of the comparisons:-)

One could imagine a test that also quickly shortcutted most of the 
unequal comparisons.

A possibly bigger problem with allowing a more general definition of 
equality for unicodes might be that it could be incompatible with the 
ordering given by < and <=.

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From guido at python.org  Tue Dec  9 12:23:10 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec  9 12:23:18 2003
Subject: [Python-Dev] Re: Source-level backward compatibility requirements
	-logging and apply() - addendum
In-Reply-To: Your message of "Tue, 09 Dec 2003 09:21:22 +0100."
	<br40m7$lgg$1@sea.gmane.org> 
References: <002901c3bde4$9b069980$652b6992@alpha>  
	<br40m7$lgg$1@sea.gmane.org> 
Message-ID: <200312091723.hB9HNAM25939@c-24-5-183-134.client.comcast.net>

> in my experience, you always lose your audience when you introduce the
> callable(*args, **kwargs) syntax, and you don't get them back until you
> explain that the notion is apply(callable, args, kwargs).

Maybe it depends on the audience?

In my experience (admitted it was a while ago), the desire to call a
function with arguments that are already given as an array often comes
to relative newbies who've never heard of apply in any other language
(isn't it a Lisp thing?).  Sometimes they even ask why

  a = (1, 2, 3)
  f(a)

isn't equivalent to

  f(1, 2, 3)

and telling them about

  f(*a)

usually makes them very happy.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Tue Dec  9 12:29:45 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Dec  9 12:29:48 2003
Subject: [Python-Dev] Re: Source-level backward compatibility requirements
	-logging and apply() - addendum
In-Reply-To: <200312091723.hB9HNAM25939@c-24-5-183-134.client.comcast.net>
References: <002901c3bde4$9b069980$652b6992@alpha> <br40m7$lgg$1@sea.gmane.org>
	<200312091723.hB9HNAM25939@c-24-5-183-134.client.comcast.net>
Message-ID: <20031209172944.GA22672@panix.com>

On Tue, Dec 09, 2003, Guido van Rossum wrote:
>Fredrik
>>
>> in my experience, you always lose your audience when you introduce the
>> callable(*args, **kwargs) syntax, and you don't get them back until you
>> explain that the notion is apply(callable, args, kwargs).
> 
> Maybe it depends on the audience?
> 
> In my experience (admitted it was a while ago), the desire to call a
> function with arguments that are already given as an array often comes
> to relative newbies who've never heard of apply in any other language
> (isn't it a Lisp thing?).  

There's also the issue that IME teaching apply() usually comes after
teaching

    def foo(*args, **kwargs):

which makes the symmetry easier to understand, IMO.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From martin at v.loewis.de  Tue Dec  9 14:51:45 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Dec  9 14:52:25 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
In-Reply-To: <3FD71F8E.6030307@Acm.Org>
References: <E1AThE0-0003cc-I2@mail.python.org> <3FD71F8E.6030307@Acm.Org>
Message-ID: <m3r7zdpy5q.fsf@mira.informatik.hu-berlin.de>

Scott David Daniels <Scott.Daniels@Acm.Org> writes:

> Could we perhaps use a comparison that, in effect, did:
> 
>      def uni_equal(first, second):
>          if first == second:
>              return True
>          return first.normalize() == second.normalize()
> 
> That is, take advantage of the fact that normalization is often
> unnecessary for "trivial" reasons.

It also affects hashing, if Unicode objects are used as dictionary
keys. Objects that compare equal need to hash equal.

Regards,
Martin


From walter at livinglogic.de  Tue Dec  9 16:20:01 2003
From: walter at livinglogic.de (=?ISO-8859-1?Q?Walter_D=F6rwald?=)
Date: Tue Dec  9 16:20:27 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
In-Reply-To: <Pine.LNX.4.58.0312080138140.11373@alice>
References: <001701c3bd29$4878f020$0705a044@oemcomputer>
	<Pine.LNX.4.58.0312080138140.11373@alice>
Message-ID: <3FD63C81.20506@livinglogic.de>

John J Lee wrote:

> On Sun, 7 Dec 2003, Raymond Hettinger wrote:
> [...]
> 
>>Ideally, all repeatable, self verifying tests should be coded using the
>>unittest module and placed in Lib/test.
> 
> [...]
> 
>>Some of tests in the module itself are being left intact for various
>>reasons.  For instance, the random module still has a few tests that
>>need a human eye to authenticate.  Some, like the ones in urllib2 are
>>normally only run when someone updates the module -- they should be left
>>in for convenience but could also have a counterpart in Lib/test so long
>>as the test were marked to run only when -u network was enabled.  As you
> 
> 
> (Ah, hadn't seen the test runner script, so didn't know about -u network,
> thanks.)
> 
> OK, so I can PyUnit-ize the urllib2 functional tests (but leave them there
> so functional tests can conveniently be run separately), and link them to
> Lib/test (so all self-verifying tests get run there).

Personally I'd prefer all tests to be in the test scripts. Otherwise
they clutter up the module without serving any real purpose (except
maybe as documentation).

> One problem, though: doesn't putting functional tests in Lib/test throw
> off Walter Dorwald's unit test coverage numbers?  Perhaps there should be
> a 'functional' resource for test_support.use_resources (so the tests can
> be run with -uall, -functional for coverage measurements)?

What exactly do you mean with 'functional' tests? We certainly don't
need non-functional tests! ;) And why would additional test scripts
throw off the coverage numbers?

Bye,
    Walter D?rwald



From greg at cosc.canterbury.ac.nz  Tue Dec  9 19:40:06 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec  9 19:40:12 2003
Subject: [Python-Dev] Re: Source-level backward compatibility requirements
	-logging and apply() - addendum
In-Reply-To: <br40m7$lgg$1@sea.gmane.org>
Message-ID: <200312100040.hBA0e6225446@oma.cosc.canterbury.ac.nz>

Fredrik Lundh <fredrik@pythonware.com>:

> in my experience, you always lose your audience when you introduce the
> callable(*args, **kwargs) syntax, and you don't get them back until you
> explain that the notion is apply(callable, args, kwargs).

What sort of example are you using to introduce it? Have
you taught them about the * and ** syntaxes in function
definitions first?

The main use case motivating it is passing on * and **
arguments to an overridden method, and when seen in that
context it looks almost self-explanatory, in my opinion.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From raymond.hettinger at verizon.net  Tue Dec  9 21:55:55 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Dec  9 21:56:26 2003
Subject: [Python-Dev] PEP 292 and templating
Message-ID: <001601c3bec9$2f258d80$e841fea9@oemcomputer>

Is there interest in having a templating module with two functions one
for simple substitutions and the other with more tools?
 
The first would be Barry's simple substitutions using only $name or
${name} for templates exposed to the user.
 
The second would extend the first with Cheetah style dotted names for
more advanced templates controlled by the programmer.
 
 
Raymond Hettinger
 
 
 
 
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031209/c8aada22/attachment.html
From fredrik at pythonware.com  Wed Dec 10 03:34:47 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Wed Dec 10 03:35:04 2003
Subject: [Python-Dev] Re: PEP 292 and templating
References: <001601c3bec9$2f258d80$e841fea9@oemcomputer>
Message-ID: <br6lrj$bil$1@sea.gmane.org>

Raymond Hettinger wrote:

> The first would be Barry's simple substitutions using only
> $name or ${name} for templates exposed to the user.
>
> The second would extend the first with Cheetah style
> dotted names for more advanced templates controlled
> by the programmer.

time for a "retools" module, perhaps?

some code snippets to consider:

http://www.effbot.org/zone/re-sub.htm
http://www.effbot.org/zone/template-language.htm

a collection of "cookbook REs" would also be nice.  any takers?

</F>




From kdart at kdart.com  Wed Dec 10 05:23:02 2003
From: kdart at kdart.com (Keith Dart)
Date: Wed Dec 10 05:23:08 2003
Subject: [Python-Dev] Greetings
Message-ID: <1071051781.14243.25.camel@leviathan.kdart.com>

Hello all. I just joined this list, but I have been a full-time Python
developer for several years now. 

My goal here, if possible, would be to contribute some of my code to the
the standard library. I have some more time now to clean up and
contribute some of my own library code since I am, alas, unemployed. 

I look forward to working with you all.

Sincerely,
Keith Dart


-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031210/4f4e6bff/attachment.bin
From Jack.Jansen at cwi.nl  Wed Dec 10 06:59:23 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Dec 10 06:58:59 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle)
In-Reply-To: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
References: <200312041341.hB4DfLcp015502@maxim.off.ekorp.com>
Message-ID: <4B7BF5A7-2B08-11D8-A7AA-0030655234CE@cwi.nl>

Anthony,
I would like to do a 2.3.3 release for MacOS9 this time, so after 
you've finished
with the general 2.3.3 release could you please hand over ownership
of the branch to me, and tell the crowd to refrain from fiddling with
the branch until I'm done?

Thanks,
--
Jack Jansen        <Jack.Jansen@cwi.nl>        http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From anthony at ekit-inc.com  Wed Dec 10 08:31:50 2003
From: anthony at ekit-inc.com (Anthony Baxter)
Date: Wed Dec 10 08:32:16 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle) 
In-Reply-To: Message from Jack Jansen <Jack.Jansen@cwi.nl> of "Wed,
	10 Dec 2003 12:59:23 BST."
	<4B7BF5A7-2B08-11D8-A7AA-0030655234CE@cwi.nl> 
Message-ID: <200312101331.hBADVo7A019516@maxim.off.ekorp.com>

>>> Jack Jansen wrote
> Anthony,
> I would like to do a 2.3.3 release for MacOS9 this time, so after 
> you've finished
> with the general 2.3.3 release could you please hand over ownership
> of the branch to me, and tell the crowd to refrain from fiddling with
> the branch until I'm done?

Not a problem. Were you going to do an OS X release as well?

Assuming we get the 2.3.3 release on Friday the 19th (AU time), how
long do you anticipate you'll want the branch for? (Guesstimate, 
obviously).

Anthony

From Jack.Jansen at cwi.nl  Wed Dec 10 09:24:16 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Dec 10 09:23:50 2003
Subject: [Python-Dev] release23-maint branch CLOSED for release (was Re:
	2.3.3 cycle) 
In-Reply-To: <200312101331.hBADVo7A019516@maxim.off.ekorp.com>
References: <200312101331.hBADVo7A019516@maxim.off.ekorp.com>
Message-ID: <890803F8-2B1C-11D8-BC8B-0030655234CE@cwi.nl>


On 10 Dec 2003, at 14:31, Anthony Baxter wrote:

>>>> Jack Jansen wrote
>> Anthony,
>> I would like to do a 2.3.3 release for MacOS9 this time, so after
>> you've finished
>> with the general 2.3.3 release could you please hand over ownership
>> of the branch to me, and tell the crowd to refrain from fiddling with
>> the branch until I'm done?
>
> Not a problem. Were you going to do an OS X release as well?

Yes. But that's done from the standard unix sources (maybe with one
or two version numbers of installers changed, but that's all).

For MacOS9 I often have to add some last second fixes.

> Assuming we get the 2.3.3 release on Friday the 19th (AU time), how
> long do you anticipate you'll want the branch for? (Guesstimate,
> obviously).

The christmas-newyear break would be a good time to do this,
but I'll try to get to it earlier.
--
Jack Jansen        <Jack.Jansen@cwi.nl>        http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From guido at python.org  Wed Dec 10 10:13:37 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 10 10:13:42 2003
Subject: [Python-Dev] Greetings
In-Reply-To: Your message of "Wed, 10 Dec 2003 02:23:02 PST."
	<1071051781.14243.25.camel@leviathan.kdart.com> 
References: <1071051781.14243.25.camel@leviathan.kdart.com> 
Message-ID: <200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>

> Hello all. I just joined this list, but I have been a full-time Python
> developer for several years now.=20
> 
> My goal here, if possible, would be to contribute some of my code to the
> the standard library. I have some more time now to clean up and
> contribute some of my own library code since I am, alas, unemployed.
> 
> I look forward to working with you all.

Sure!  Welcome.  Sorry to hear about your job loss.  Have you read the
developer docs at http://www.python.org/dev/?  Depending on the size
of your contributions, the patch submission manager at SourceForge or
a PEP describing the functionality would seem the best way to get the
process started.  Feel free to ask more directed questions here.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From Scott.Daniels at Acm.Org  Thu Dec 11 09:27:59 2003
From: Scott.Daniels at Acm.Org (Scott David Daniels)
Date: Wed Dec 10 10:26:24 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
In-Reply-To: <m3r7zdpy5q.fsf@mira.informatik.hu-berlin.de>
References: <E1AThE0-0003cc-I2@mail.python.org> <3FD71F8E.6030307@Acm.Org>
	<m3r7zdpy5q.fsf@mira.informatik.hu-berlin.de>
Message-ID: <3FD87EEF.8060506@Acm.Org>

I na?vely wrote:
 >Could we perhaps use a comparison that, in effect, did:
 >     def uni_equal(first, second):
 >         if first == second:
 >             return True
 >         return first.normalize() == second.normalize()
 >That is, take advantage of the fact that normalization is often
 >unnecessary for "trivial" reasons.

This works, and a similar "unequal" trick may be constructible.
Ordering is certainly trickier (assuring we have a total order
given new equalities, so that we cannot choose a, b, and c where:
     a < b = c > a   is  True.

But, Martin v. L?wis points out:
> It also affects hashing, if Unicode objects are used as dictionary
> keys. Objects that compare equal need to hash equal.

Still not disgusting, _but_ unicode strings must hash equal to
the corresponding "plain" string.  I am not certain about this
requirement for non-ASCII characters, but I expect we are stuck
with matching hashes in the range ord(' ') through ord('~') and
probably for all character values from 0 through 127.  We might
be able to classify UTF-16 code units into three groups:
   1) matches base ASCII character
   2) diacritical or combining
   3) definitely distinct from any ASCII or combining form.
If we map the group 1 entries to the corresponding ASCII code,
skip the group 2s, and take the group 3s separately (probably
remapping to another set), we might come up with a hash that
used only the map results as elements contributing to the hash.

Are we stuck with the current hash for unicode?  If so, there is
little hope.  If not, this might bear further investigation.

-Scott David Daniels
Scott.Daniels@Acm.Org

From jjl at pobox.com  Wed Dec 10 12:14:39 2003
From: jjl at pobox.com (John J Lee)
Date: Wed Dec 10 12:14:49 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
In-Reply-To: <3FD63C81.20506@livinglogic.de>
References: <001701c3bd29$4878f020$0705a044@oemcomputer>
	<Pine.LNX.4.58.0312080138140.11373@alice>
	<3FD63C81.20506@livinglogic.de>
Message-ID: <Pine.LNX.4.58.0312100300450.31525@alice>

On Tue, 9 Dec 2003, [ISO-8859-1] Walter D=F6rwald wrote:
[...]
> > One problem, though: doesn't putting functional tests in Lib/test throw
> > off Walter Dorwald's unit test coverage numbers?  Perhaps there should =
be
> > a 'functional' resource for test_support.use_resources (so the tests ca=
n
> > be run with -uall, -functional for coverage measurements)?
>
> What exactly do you mean with 'functional' tests? We certainly don't
> need non-functional tests! ;) And why would additional test scripts
> throw off the coverage numbers?

http://c2.com/cgi/wiki?FunctionalTest

I had assumed you were measuring unit test coverage, but maybe there are
lots of functional tests in Lib/test too.

BTW, I had a look at your coverage page, and it looked as if it would be
useful and interesting, if I understood what all those little graphs &c.
were. :-/


John

From martin at v.loewis.de  Wed Dec 10 12:16:15 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Dec 10 12:16:23 2003
Subject: [Python-Dev] Re: test_unicode_file failing on Mac OS X
In-Reply-To: <3FD87EEF.8060506@Acm.Org>
References: <E1AThE0-0003cc-I2@mail.python.org> <3FD71F8E.6030307@Acm.Org>
	<m3r7zdpy5q.fsf@mira.informatik.hu-berlin.de>
	<3FD87EEF.8060506@Acm.Org>
Message-ID: <m3he084mqo.fsf@mira.informatik.hu-berlin.de>

Scott David Daniels <Scott.Daniels@Acm.Org> writes:

> Still not disgusting, _but_ unicode strings must hash equal to
> the corresponding "plain" string.  I am not certain about this
> requirement for non-ASCII characters, but I expect we are stuck
> with matching hashes in the range ord(' ') through ord('~') and
> probably for all character values from 0 through 127.

Strictly speaking, unicode and byte strings must hash equal if they
convert to each other through the system encoding. As a side effect,
this really means that the system encoding currently must be ASCII, as
the current hash function won't otherwise hash equal (latin-1 may also
be correct as the system encoding).

Fortunately, plain ASCII strings are in all applicable normal forms
(NFC, NFKC, NFD, NFKD).

Regards,
Martin

From edloper at gradient.cis.upenn.edu  Wed Dec 10 13:32:22 2003
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Wed Dec 10 12:31:10 2003
Subject: [Python-Dev] Normalizing unicode? (was: Re: test_unicode_file
 failing on Mac OS X)
In-Reply-To: <E1AU7iL-0006qv-FD@mail.python.org>
References: <E1AU7iL-0006qv-FD@mail.python.org>
Message-ID: <3FD766B6.8000204@gradient.cis.upenn.edu>

Scott David Daniels wrote:
> I na?vely wrote:
>  >Could we perhaps use a comparison that, in effect, did:
>  >     def uni_equal(first, second):
>  >         if first == second:
>  >             return True
>  >         return first.normalize() == second.normalize()
>  >That is, take advantage of the fact that normalization is often
>  >unnecessary for "trivial" reasons.
> 
> [...]

Before we start considering how it's possible to make unicode.__equal__ 
act encoding-insensitively[1], I think we need to consider whether 
that's really the behavior we want.  In some ways, this seems like 
case-insensitive equality to me: it's certainly a useful operation, but 
I don't think it should be the object's builtin notion of equality..
   - I think people will be confused if s1==s2 but s1[0]!=s2[0].
   - Sometimes you might *want* to distinguish different encodings of
     the "same" string; a "normalized" equality test makes that very
     difficult.

And if you *do* want unicode objects to act normalized, then I think 
that the right way to do it is to normalize them at creation time.  Then 
all the right hash/eq/cmp stuff just falls out.

But since some people will may want to distinguish different encodings 
of the same string, I think that the most sensible alternative is to add 
a new subclass to unicode -- something like "normalized_unicode."  It 
would normalize itself at construction time; and when combined with 
other unicode strings (eg by +), the result would be normalized (so 
unicode+normalized_unicode -> normalized_unicode).  It's possible that 
the normalized unicode class would be more useful to people (and 
therefore more widely used?), but the non-normalized version would still 
be available for people who want it.

(or we could just leave things as they are now, and force people to do 
any normalization themselves. :) )

-Edward

[1] I don't think that "encoding" is the right technical term here, but 
I'm not sure what the right term is.  I mean insensitive to the 
difference between separated diacritics & unified diacritics.



From guido at python.org  Wed Dec 10 12:39:16 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 10 12:39:24 2003
Subject: [Python-Dev] Normalizing unicode? (was: Re: test_unicode_file
	failing on Mac OS X)
In-Reply-To: Your message of "Wed, 10 Dec 2003 12:32:22 CST."
	<3FD766B6.8000204@gradient.cis.upenn.edu> 
References: <E1AU7iL-0006qv-FD@mail.python.org>  
	<3FD766B6.8000204@gradient.cis.upenn.edu> 
Message-ID: <200312101739.hBAHdGx27967@c-24-5-183-134.client.comcast.net>

> Before we start considering how it's possible to make unicode.__equal__ 
> act encoding-insensitively[1], I think we need to consider whether 
> that's really the behavior we want.  In some ways, this seems like 
> case-insensitive equality to me: it's certainly a useful operation, but 
> I don't think it should be the object's builtin notion of equality..
>    - I think people will be confused if s1==s2 but s1[0]!=s2[0].
>    - Sometimes you might *want* to distinguish different encodings of
>      the "same" string; a "normalized" equality test makes that very
>      difficult.

Right.  Couldn't have said it better myself.

> And if you *do* want unicode objects to act normalized, then I think 
> that the right way to do it is to normalize them at creation time.  Then 
> all the right hash/eq/cmp stuff just falls out.

Exactly.

> But since some people will may want to distinguish different encodings 
> of the same string, I think that the most sensible alternative is to add 
> a new subclass to unicode -- something like "normalized_unicode."  It 
> would normalize itself at construction time; and when combined with 
> other unicode strings (eg by +), the result would be normalized (so 
> unicode+normalized_unicode -> normalized_unicode).  It's possible that 
> the normalized unicode class would be more useful to people (and 
> therefore more widely used?), but the non-normalized version would still 
> be available for people who want it.

Works for me.  I recomment that someone try this approach as a user
subclass first -- this should be easy enough, right?

> (or we could just leave things as they are now, and force people to do 
> any normalization themselves. :) )

Do we even have normalization code in core Python?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Wed Dec 10 12:43:36 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Wed Dec 10 12:44:02 2003
Subject: [Python-Dev] Normalizing unicode? (was: Re: test_unicode_file
	failing on Mac OS X)
In-Reply-To: <200312101739.hBAHdGx27967@c-24-5-183-134.client.comcast.net>
References: <E1AU7iL-0006qv-FD@mail.python.org>
	<3FD766B6.8000204@gradient.cis.upenn.edu>
	<200312101739.hBAHdGx27967@c-24-5-183-134.client.comcast.net>
Message-ID: <m38ylk4lh3.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> > (or we could just leave things as they are now, and force people to do 
> > any normalization themselves. :) )
> 
> Do we even have normalization code in core Python?

Yes, in unicodedata.normalize, since 2.3.

I agree that having == operate as if strings where normalized is
undesirable.

Regards,
Martin

From walter at livinglogic.de  Wed Dec 10 13:03:47 2003
From: walter at livinglogic.de (=?ISO-8859-15?Q?Walter_D=F6rwald?=)
Date: Wed Dec 10 13:04:03 2003
Subject: [Python-Dev] Conventions for functional tests, PyUnit?
In-Reply-To: <Pine.LNX.4.58.0312100300450.31525@alice>
References: <001701c3bd29$4878f020$0705a044@oemcomputer>
	<Pine.LNX.4.58.0312080138140.11373@alice>
	<3FD63C81.20506@livinglogic.de>
	<Pine.LNX.4.58.0312100300450.31525@alice>
Message-ID: <3FD76003.3030809@livinglogic.de>

John J Lee wrote:

> On Tue, 9 Dec 2003, [ISO-8859-1] Walter D?rwald wrote:
> [...]
> 
>>>One problem, though: doesn't putting functional tests in Lib/test throw
>>>off Walter Dorwald's unit test coverage numbers?  Perhaps there should be
>>>a 'functional' resource for test_support.use_resources (so the tests can
>>>be run with -uall, -functional for coverage measurements)?
>>
>>What exactly do you mean with 'functional' tests? We certainly don't
>>need non-functional tests! ;) And why would additional test scripts
>>throw off the coverage numbers?
> 
> 
> http://c2.com/cgi/wiki?FunctionalTest
> 
> I had assumed you were measuring unit test coverage, but maybe there are
> lots of functional tests in Lib/test too.

What I'm doing is compiling the Python source with profiling options
and running Lib/test/regrtest.py via Lib/trace.py.

After that the script calls gcov for every .c file to get C coverage
info and checks the coverage files generated by trace.py to get Python
coverage info.

> BTW, I had a look at your coverage page, and it looked as if it would be
> useful and interesting, if I understood what all those little graphs &c.
> were. :-/

I guess a little explanation would help. There are four types of source
code lines:

1) Unknown: The status of the line can not be determined, because
    gcov or trace.py didn't generate a coverage file
    (color black)
2) Uncoverable: Can not be executed (Comment or empty line)
    (color grey)
3) Uncovered (but coverable): Has not been executed (but is executable)
    (color red)
4) Covered: Has been executed at least once during the test run.
    (color green)

For files:

1) Unknown: A coverage file hasn't been created.
2) Uncoverable: The file consists only of uncoverable lines
    (e.g. a file with comments only).
3) Uncovered: The file contains coverable lines, but none of
    these lines have been executed.
4) Covered: The file contains at least one covered line.


Note that the web application isn't exactly fast. The machine is
rather old and the Postgres database seems to have a few problems
with tables that contain 10 million records. :-/

Bye,
    Walter D?rwald



From kdart at kdart.com  Wed Dec 10 16:14:17 2003
From: kdart at kdart.com (Keith Dart)
Date: Wed Dec 10 16:14:21 2003
Subject: [Python-Dev] Greetings
In-Reply-To: <200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>
References: <1071051781.14243.25.camel@leviathan.kdart.com>
	<200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>
Message-ID: <1071090857.14245.36.camel@leviathan.kdart.com>

On Wed, 2003-12-10 at 07:13, Guido van Rossum wrote:

> Sure!  Welcome.  Sorry to hear about your job loss.  Have you read the
> developer docs at http://www.python.org/dev/?  Depending on the size
> of your contributions, the patch submission manager at SourceForge or
> a PEP describing the functionality would seem the best way to get the
> process started.  Feel free to ask more directed questions here.

Thank you. I have read the dev page. So, suppose one wanted to add a new
module. Should I write PEP for it? 

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031210/99fe04d5/attachment.bin
From guido at python.org  Wed Dec 10 16:30:52 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 10 16:30:58 2003
Subject: [Python-Dev] Greetings
In-Reply-To: Your message of "Wed, 10 Dec 2003 13:14:17 PST."
	<1071090857.14245.36.camel@leviathan.kdart.com> 
References: <1071051781.14243.25.camel@leviathan.kdart.com>
	<200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net> 
	<1071090857.14245.36.camel@leviathan.kdart.com> 
Message-ID: <200312102130.hBALUq328562@c-24-5-183-134.client.comcast.net>

> Thank you. I have read the dev page. So, suppose one wanted to add a new
> module. Should I write PEP for it?

That question can't be answered in general; I suggest presenting the
new module you'd like to see added here (in just a few lines or
paragraphs), with motivation why you think it's useful for many Python
users, and see what kind of feedback you get.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From danmil at shore.net  Wed Dec 10 17:05:56 2003
From: danmil at shore.net (Dan Milstein)
Date: Wed Dec 10 17:05:41 2003
Subject: [Python-Dev] Experimental Macro System
Message-ID: <3FD798C4.6040709@shore.net>

As a way to learn more about Python (and for fun), I'm building an 
experimental Dylan- and Lisp-inspired Macro system for Python.  I've just 
put up a 0.1 release on Sourceforge at:

https://sourceforge.net/projects/pymac/

...if anyone is interested.  It's pretty primitive at the moment, but it is 
possible to write macros and use them to expand and run actual Python code. 
  It's currently implemented as a preprocessor (written in Python).

More details in the README file in the project.

-Dan Milstein


From aleaxit at yahoo.com  Wed Dec 10 18:35:46 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Wed Dec 10 18:35:53 2003
Subject: [Python-Dev] a serious threat to 2.3's speed?
Message-ID: <200312110035.46217.aleaxit@yahoo.com>

According to:

http://primates.ximian.com/~miguel/archive/2003/Dec-09.html

Hugunin's new Ironpython (Python on .NET) project is 70% faster
than CPython-2.3 on the standard pystone benchmark.  That's all
I know about it, but it may need checking out (dunno if parrot can
take some tricks from this Ironpython thing, but, who knows... we'd
better preemptively see if there's anything we can take for CPython!).


Alex


From tdelaney at avaya.com  Wed Dec 10 18:46:04 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 10 18:46:13 2003
Subject: [Python-Dev] a serious threat to 2.3's speed?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>

> From: Alex Martelli
> 
> http://primates.ximian.com/~miguel/archive/2003/Dec-09.html
> 
> Hugunin's new Ironpython (Python on .NET) project is 70% faster
> than CPython-2.3 on the standard pystone benchmark.  That's all
> I know about it, but it may need checking out (dunno if parrot can
> take some tricks from this Ironpython thing, but, who knows... we'd
> better preemptively see if there's anything we can take for CPython!).

>From the microbenchmarks, it looks like function calls are the big bottlenecks.

Another option for a masters thesis Brett - speeding up function calls :)

Tim Delaney

From tdelaney at avaya.com  Wed Dec 10 19:58:06 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 10 19:58:13 2003
Subject: [Python-Dev] a serious threat to 2.3's speed?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEF605DB@au3010avexu1.global.avaya.com>

> >From the microbenchmarks, it looks like function calls are 
> the big bottlenecks.

BTW the above is my analysis - not Alex's - obviously I got the cursor on the wrong line ... my apologies.

Tim Delaney

From bac at OCF.Berkeley.EDU  Wed Dec 10 23:06:39 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Wed Dec 10 23:06:47 2003
Subject: [Python-Dev] a serious threat to 2.3's speed?
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
Message-ID: <3FD7ED4F.5060708@ocf.berkeley.edu>

Delaney, Timothy C (Timothy) wrote:
>>From: Alex Martelli
>>
>>http://primates.ximian.com/~miguel/archive/2003/Dec-09.html
>>
>>Hugunin's new Ironpython (Python on .NET) project is 70% faster
>>than CPython-2.3 on the standard pystone benchmark.  That's all
>>I know about it, but it may need checking out (dunno if parrot can
>>take some tricks from this Ironpython thing, but, who knows... we'd
>>better preemptively see if there's anything we can take for CPython!).
> 
> 
>>From the microbenchmarks, it looks like function calls are the big bottlenecks.
> 
> Another option for a masters thesis Brett - speeding up function calls :)
> 


=)  Maybe.

Just a quick update for people: I talked to my adviser and my possible 
topics as been thinned out considerably.  I have my last final tomorrow 
at which point I will be free, so to speak, and thus plan on making the 
time to post the list again to get another round of comments on what 
people see as beneficial, etc.

-Brett

From martin at v.loewis.de  Thu Dec 11 02:21:40 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 11 02:21:51 2003
Subject: [Python-Dev] Greetings
In-Reply-To: <200312102130.hBALUq328562@c-24-5-183-134.client.comcast.net>
References: <1071051781.14243.25.camel@leviathan.kdart.com>
	<200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>
	<1071090857.14245.36.camel@leviathan.kdart.com>
	<200312102130.hBALUq328562@c-24-5-183-134.client.comcast.net>
Message-ID: <m3wu93izuj.fsf@mira.informatik.hu-berlin.de>

Guido van Rossum <guido@python.org> writes:

> That question can't be answered in general; I suggest presenting the
> new module you'd like to see added here (in just a few lines or
> paragraphs), with motivation why you think it's useful for many Python
> users, and see what kind of feedback you get.

In addition, you should realize that there are many ways of publishing
Python modules; including them in the Python distribution is just one
way.

Regards,
Martin


From martin at v.loewis.de  Thu Dec 11 02:23:56 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 11 02:24:20 2003
Subject: [Python-Dev] a serious threat to 2.3's speed?
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEF605DB@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF605DB@au3010avexu1.global.avaya.com>
Message-ID: <m3smjrizqr.fsf@mira.informatik.hu-berlin.de>

"Delaney, Timothy C (Timothy)" <tdelaney@avaya.com> writes:

> > >From the microbenchmarks, it looks like function calls are 
> > the big bottlenecks.

> BTW the above is my analysis - not Alex's - obviously I got the
> cursor on the wrong line ... my apologies.

It's not that obvious. More obviously, it's a line starting with
"From", and your mailer has inserted a greater-than sign
automatically.

Regards,
Martin

From kdart at kdart.com  Thu Dec 11 02:32:44 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 02:32:48 2003
Subject: [Python-Dev] distributing modules
In-Reply-To: <m3wu93izuj.fsf@mira.informatik.hu-berlin.de>
References: <1071051781.14243.25.camel@leviathan.kdart.com>
	<200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>
	<1071090857.14245.36.camel@leviathan.kdart.com>
	<200312102130.hBALUq328562@c-24-5-183-134.client.comcast.net>
	<m3wu93izuj.fsf@mira.informatik.hu-berlin.de>
Message-ID: <1071127964.14245.62.camel@leviathan.kdart.com>

On Wed, 2003-12-10 at 23:21, Martin v. L?wis wrote:
> In addition, you should realize that there are many ways of publishing
> Python modules; including them in the Python distribution is just one
> way.

That's true. In fact, I do distribute some from sourceforge.net (in the
pyNMS project) now. But that is a large package that does contain some
general and unrelated modules that others might find useful. It is also
limited to a tarball download or CVS access. That limits it's user base
a bit. Since we currently don't have a CPAN for Python, I thought I
might try this route. 

Also, I know it is often said that Python has "batteries included".
However, in my experience the "batteries" are often inadequate for the
job. Many modules are incomplete, or otherwise like some key
functionality. I have had to extend and/or rewrite many of the "stock"
libraries over the past few years. I would like to fold my additions
back into the core, if possible. 


-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031210/74167065/attachment.bin
From karthik19_20 at yahoo.com  Thu Dec 11 03:31:58 2003
From: karthik19_20 at yahoo.com (karthik karthik)
Date: Thu Dec 11 03:32:02 2003
Subject: [Python-Dev] prolem in running AAIE demo in pyAA.py
Message-ID: <20031211083158.63563.qmail@web11901.mail.yahoo.com>

hi!!

when i'm trying to run the AAIE demo in pyAA.py i got
this error message:


    from wxPython.wx import *
ImportError: No module named wxPython.wx


can anyone provide me the link to download
wxPython.wx?

regards,
Sha

__________________________________
Do you Yahoo!?
New Yahoo! Photos - easier uploading and sharing.
http://photos.yahoo.com/

From fredrik at pythonware.com  Thu Dec 11 05:16:58 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu Dec 11 05:17:12 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
Message-ID: <br9g72$o66$1@sea.gmane.org>

Tim Delaney wrote:

> From the microbenchmarks, it looks like function calls are the big
> bottlenecks.

in my pytte1 experiment from last year, I was able to run pystone about 25
times faster than a stock 2.1 interpreter.

(that would be, say, about 20 times faster than 2.3)

key features contributing to this is true gc, call-site caching, and "traditional"
(C-style) argument passing for common cases.

not sure if any of that can be applied any of that to today's C python, though
(but that doesn't mean it cannot be done ;-).

> Another option for a masters thesis Brett - speeding up function calls :)

that would be most excellent.

</F>




From mwh at python.net  Thu Dec 11 05:32:17 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 11 05:32:20 2003
Subject: [Python-Dev] distributing modules
In-Reply-To: <1071127964.14245.62.camel@leviathan.kdart.com> (Keith Dart's
	message of "Wed, 10 Dec 2003 23:32:44 -0800")
References: <1071051781.14243.25.camel@leviathan.kdart.com>
	<200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>
	<1071090857.14245.36.camel@leviathan.kdart.com>
	<200312102130.hBALUq328562@c-24-5-183-134.client.comcast.net>
	<m3wu93izuj.fsf@mira.informatik.hu-berlin.de>
	<1071127964.14245.62.camel@leviathan.kdart.com>
Message-ID: <2mad5zodam.fsf@starship.python.net>

Keith Dart <kdart@kdart.com> writes:

> Also, I know it is often said that Python has "batteries included".
> However, in my experience the "batteries" are often inadequate for the
> job. Many modules are incomplete, or otherwise like some key
> functionality. I have had to extend and/or rewrite many of the "stock"
> libraries over the past few years. I would like to fold my additions
> back into the core, if possible. 

I think improving existing library modules is a much less contentious
issue than adding new ones...

Cheers,
mwh

-- 
  Darned confusing, unless you have that magic ingredient coffee, of
  which I can pay you Tuesday for a couple pounds of extra-special
  grind today.                           -- John Mitchell, 11 Jan 1999

From mwh at python.net  Thu Dec 11 05:34:37 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 11 05:34:40 2003
Subject: [Python-Dev] Normalizing unicode?
In-Reply-To: <3FD766B6.8000204@gradient.cis.upenn.edu> (Edward Loper's
	message of "Wed, 10 Dec 2003 12:32:22 -0600")
References: <E1AU7iL-0006qv-FD@mail.python.org>
	<3FD766B6.8000204@gradient.cis.upenn.edu>
Message-ID: <2m65gnod6q.fsf@starship.python.net>

Edward Loper <edloper@gradient.cis.upenn.edu> writes:

> Scott David Daniels wrote:
>> I na?vely wrote:
>>  >Could we perhaps use a comparison that, in effect, did:
>>  >     def uni_equal(first, second):
>>  >         if first == second:
>>  >             return True
>>  >         return first.normalize() == second.normalize()
>>  >That is, take advantage of the fact that normalization is often
>>  >unnecessary for "trivial" reasons.
>> [...]
>
> Before we start considering how it's possible to make
> unicode.__equal__ act encoding-insensitively[1], I think we need to
> consider whether that's really the behavior we want.  In some ways,
> this seems like case-insensitive equality to me: it's certainly a
> useful operation, but I don't think it should be the object's builtin
> notion of equality..
>    - I think people will be confused if s1==s2 but s1[0]!=s2[0].
>    - Sometimes you might *want* to distinguish different encodings of
>      the "same" string; a "normalized" equality test makes that very
>      difficult.

In general it seems to me that == should, given a choice, err on the
side of being an overly tight equivalence relation -- i.e. return True
less often.

Cheers,
mwh

-- 
81. In computing, turning the obvious into the useful is a living
    definition of the word "frustration".
  -- Alan Perlis, http://www.cs.yale.edu/homes/perlis-alan/quotes.html

From martin at v.loewis.de  Thu Dec 11 06:34:14 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 11 06:36:13 2003
Subject: [Python-Dev] Re: distributing modules
In-Reply-To: <1071127964.14245.62.camel@leviathan.kdart.com>
References: <1071051781.14243.25.camel@leviathan.kdart.com>
	<200312101513.hBAFDbi27677@c-24-5-183-134.client.comcast.net>
	<1071090857.14245.36.camel@leviathan.kdart.com>
	<200312102130.hBALUq328562@c-24-5-183-134.client.comcast.net>
	<m3wu93izuj.fsf@mira.informatik.hu-berlin.de>
	<1071127964.14245.62.camel@leviathan.kdart.com>
Message-ID: <m3smjrmvux.fsf@mira.informatik.hu-berlin.de>

Keith Dart <kdart@kdart.com> writes:

> That's true. In fact, I do distribute some from sourceforge.net (in the
> pyNMS project) now. But that is a large package that does contain some
> general and unrelated modules that others might find useful.

As the first step, you should make use of distutils, then generate
binary packages for the platforms you care about (e.g. bdist_wininst,
bdist_rpm).

As the second step, you should register your packages with PyPI:

http://www.python.org/pypi

> Also, I know it is often said that Python has "batteries included".
> However, in my experience the "batteries" are often inadequate for the
> job. Many modules are incomplete, or otherwise like some key
> functionality. I have had to extend and/or rewrite many of the "stock"
> libraries over the past few years.

As Michael explains, improvements on existing libraries are very much
appreciated. You should upload each improvement individually to SF,
indicating what bug it fixes or what feature it adds. I would be
concerned about rewrites, though, because they might break backwards
compatibility.

Regards,
Martin


From jeremy at alum.mit.edu  Thu Dec 11 10:04:30 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Thu Dec 11 10:07:49 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <br9g72$o66$1@sea.gmane.org>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
	<br9g72$o66$1@sea.gmane.org>
Message-ID: <1071155070.15985.583.camel@localhost.localdomain>

On Thu, 2003-12-11 at 05:16, Fredrik Lundh wrote:
> Tim Delaney wrote:
> 
> > From the microbenchmarks, it looks like function calls are the big
> > bottlenecks.
> 
> in my pytte1 experiment from last year, I was able to run pystone about 25
> times faster than a stock 2.1 interpreter.

what is pyttel?  I don't think I've heard of it.

Jeremy



From skip at pobox.com  Thu Dec 11 10:09:03 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 11 10:09:21 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <br9g72$o66$1@sea.gmane.org>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
	<br9g72$o66$1@sea.gmane.org>
Message-ID: <16344.34959.580835.583077@montanaro.dyndns.org>


    >> Another option for a masters thesis Brett - speeding up function
    >> calls :)

    Fredrik> that would be most excellent.

Indeed.  We should make sure we're comparing apples to apples when looking
at Jim's benchmark figures though.  It's not clear to me that his current
system supports all the different types of callables which CPython supports.
That alone might streamline his function call code a lot.

Skip


From fredrik at pythonware.com  Thu Dec 11 10:17:35 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Thu Dec 11 10:17:43 2003
Subject: [Python-Dev] Re: Re: a serious threat to 2.3's speed?
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com><br9g72$o66$1@sea.gmane.org>
	<1071155070.15985.583.camel@localhost.localdomain>
Message-ID: <bra1qi$tcq$1@sea.gmane.org>

Jeremy Hylton wrote:

> > > From the microbenchmarks, it looks like function calls are the big
> > > bottlenecks.
> >
> > in my pytte1 experiment from last year, I was able to run pystone about 25
> > times faster than a stock 2.1 interpreter.
>
> what is pyttel?  I don't think I've heard of it.

google is your friend (at least if you're using the right font ;-)

    http://effbot.org/zone/pytte.htm

</F>




From aleax at aleax.it  Thu Dec 11 10:40:18 2003
From: aleax at aleax.it (Alex Martelli)
Date: Thu Dec 11 10:40:25 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <16344.34959.580835.583077@montanaro.dyndns.org>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
	<br9g72$o66$1@sea.gmane.org>
	<16344.34959.580835.583077@montanaro.dyndns.org>
Message-ID: <200312111640.18085.aleax@aleax.it>

On Thursday 11 December 2003 04:09 pm, Skip Montanaro wrote:
>     >> Another option for a masters thesis Brett - speeding up function
>     >> calls :)
>
>     Fredrik> that would be most excellent.
>
> Indeed.  We should make sure we're comparing apples to apples when looking
> at Jim's benchmark figures though.  It's not clear to me that his current
> system supports all the different types of callables which CPython
> supports. That alone might streamline his function call code a lot.

How so?  Isn't one "indirecting" through the tp_call slot anyway?  Supporting
two or two thousand types of callables should make function calls just about
the same cost -- or am I missing something...?


Alex


From skip at pobox.com  Thu Dec 11 11:06:03 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 11 11:06:12 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <200312111640.18085.aleax@aleax.it>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
	<br9g72$o66$1@sea.gmane.org>
	<16344.34959.580835.583077@montanaro.dyndns.org>
	<200312111640.18085.aleax@aleax.it>
Message-ID: <16344.38379.89674.176847@montanaro.dyndns.org>


    >> It's not clear to me that his current system supports all the
    >> different types of callables which CPython supports. That alone might
    >> streamline his function call code a lot.

    Alex> How so?  Isn't one "indirecting" through the tp_call slot anyway?
    Alex> Supporting two or two thousand types of callables should make
    Alex> function calls just about the same cost -- or am I missing
    Alex> something...?

Python's function call machinery is complex, mostly in the argument
marshalling area I think.  Take a look in Python/ceval.c at

    call_function
    fast_function
    do_call
    ext_do_call
    update_keyword_args
    update_star_args

If Jim's compiler doesn't yet support any of varargs, keyword args, calling
C functions or calling bound method objects, I suspect his code is more
streamlined.

Skip

From duncan at rcp.co.uk  Thu Dec 11 11:35:57 2003
From: duncan at rcp.co.uk (Duncan Booth)
Date: Thu Dec 11 11:36:08 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
	<br9g72$o66$1@sea.gmane.org>
	<16344.34959.580835.583077@montanaro.dyndns.org>
	<200312111640.18085.aleax@aleax.it>
	<16344.38379.89674.176847@montanaro.dyndns.org>
Message-ID: <n2m-g.Xns944EA861C333Aduncanrcpcouk@127.0.0.1>

Skip Montanaro <skip@pobox.com> wrote in
news:16344.38379.89674.176847@montanaro.dyndns.org: 

> Python's function call machinery is complex, mostly in the argument
> marshalling area I think.  Take a look in Python/ceval.c at
> 
>     call_function
>     fast_function
>     do_call
>     ext_do_call
>     update_keyword_args
>     update_star_args
> 
> If Jim's compiler doesn't yet support any of varargs, keyword args,
> calling C functions or calling bound method objects, I suspect his
> code is more streamlined.

I've been working on my own implementation of Python for .Net, although 
I've now emailed Jim to ask if I can help him with his since he seems to be 
further on that I am with my version.

The way I was planning to do function calls (and had partially implemented 
it), was to have an IPyCallable interface, and a set of classes that 
implement that interface.

The interface includes methods:

   PyObject __call__();
   PyObject __call__(PyObject arg1);
   PyObject __call__(PyObject arg1, PyObject arg2);
   PyObject __call__(PyObject arg1, PyObject arg2, PyObject arg3);
   PyObject __call__(PyObject[] args);
   PyObject __call__(PyObject[] args, PyDict keywordArgs);

Concrete classes exist for functions and for bound methods taking 0, 1, 2, 
3 or more arguments. The constructor for a function object takes a 
delegate, a list of argument names and a list of defaults.

The idea was that, say you had a function taking 3 arguments and you call 
it with 3 arguments, the call can be passed straight through without doing 
any checking at all for default args or keyword args, so one virtual call 
then a delegate call. If you call it with fewer arguments the earlier 
overloads simply add in the missing defaults. This should allow pretty well 
all calls to be streamlined so you only pay for the overhead of variable 
number of arguments or keywords if you actually use them in the call, and 
even then you don't have a string to interpret at runtime.

I was planning to create function wrappers at runtime, so that a method 
that took say a string and an integer would be wrapped by a static function 
taking three PyObject arguments and casting them appropriately before 
forwarding to the wrapped method. So a Python call to a method would get 
directed through the function object which calls the delegate to the 
wrapper which does any necessary casts and calls the underlying method. In 
practice some of these nested calls should be optimisable by the JIT.

Anyway, that was my scheme. I wonder what Jim's is.

-- 
Duncan Booth                                             duncan@rcp.co.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

From pje at telecommunity.com  Thu Dec 11 12:06:17 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec 11 12:06:35 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <n2m-g.Xns944EA861C333Aduncanrcpcouk@127.0.0.1>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEF60595@au3010avexu1.global.avaya.com>
	<br9g72$o66$1@sea.gmane.org>
	<16344.34959.580835.583077@montanaro.dyndns.org>
	<200312111640.18085.aleax@aleax.it>
	<16344.38379.89674.176847@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20031211115334.02259ec0@telecommunity.com>

At 04:35 PM 12/11/03 +0000, Duncan Booth wrote:
>The way I was planning to do function calls (and had partially implemented
>it), was to have an IPyCallable interface, and a set of classes that
>implement that interface.
>
>The interface includes methods:
>
>    PyObject __call__();
>    PyObject __call__(PyObject arg1);
>    PyObject __call__(PyObject arg1, PyObject arg2);
>    PyObject __call__(PyObject arg1, PyObject arg2, PyObject arg3);
>    PyObject __call__(PyObject[] args);
>    PyObject __call__(PyObject[] args, PyDict keywordArgs);
>
>Concrete classes exist for functions and for bound methods taking 0, 1, 2,
>3 or more arguments. The constructor for a function object takes a
>delegate, a list of argument names and a list of defaults.

Hm.  We could steal this exact same idea for use in CPython, although it'd 
be tedious to implement.  The current tp_call would cover the last two 
cases shown, but adding a tp_call0 through tp_call3 or so might speed up a 
*lot* of function calls.  Specialized bytecodes for these calls would have 
very short implementations.  The tp_call_N slots could have default 
implementations in 'object' that delegate back to the regular tp_call, so 
that end-user types wouldn't need to reimplement them.

Hm.  Actually, this'd be *really* tedious to implement right, because of 
all the custom bound method types needed, and the C multiple wrappers for 
__call__ methods written in Python.  OTOH, this might be an "opportunity" 
to write generic leftcurry and rightcurry functions, and use rightcurry for 
default arguments...

Bleah.  What this *really* seems to want is to generate a new type on the 
fly, that is precisely coded to do the necessary left and right currying as 
part of C-level code invocation.  A JIT, in other words.

Ah well, it sounded cool at first.  :)


From tjreedy at udel.edu  Thu Dec 11 14:13:45 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Thu Dec 11 14:13:49 2003
Subject: [Python-Dev] Re: prolem in running AAIE demo in pyAA.py
References: <20031211083158.63563.qmail@web11901.mail.yahoo.com>
Message-ID: <brafl7$uet$1@sea.gmane.org>


"karthik karthik" <karthik19_20@yahoo.com> wrote in message
news:20031211083158.63563.qmail@web11901.mail.yahoo.com...
> when i'm trying to run the AAIE demo in pyAA.py i got
> this error message:
>
>
>     from wxPython.wx import *
> ImportError: No module named wxPython.wx
>
>
> can anyone provide me the link to download
> wxPython.wx?

PyDev is for developing future Python versions.  Please direct usage
questions like this to the general Python mailing list (or comp.lang.python)
or to application specific lists (there is one for wxPython, I believe).

TJR




From tdelaney at avaya.com  Thu Dec 11 14:50:40 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec 11 14:50:47 2003
Subject: [Python-Dev] a serious threat to 2.3's speed?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0456@au3010avexu1.global.avaya.com>

> From: martin@v.loewis.de [mailto:martin@v.loewis.de]
> 
> > > >From the microbenchmarks, it looks like function calls are 
> > > the big bottlenecks.
> 
> > BTW the above is my analysis - not Alex's - obviously I got the
> > cursor on the wrong line ... my apologies.
> 
> It's not that obvious. More obviously, it's a line starting with
> "From", and your mailer has inserted a greater-than sign
> automatically.

What I meant was that the line has an extra indent, and the cause is most likely that when I trimmed the original message I accidentally left the cursor on the wrong line when I started typing and didn't notice it until after the message was sent. Hence the use of "obviously" - as in, that was obviously the cause of the error - not that it's obvious that it *wasn't* Alex's comment.

Tim Delaney

From tdelaney at avaya.com  Thu Dec 11 14:55:09 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec 11 14:55:17 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0458@au3010avexu1.global.avaya.com>

> From: Fredrik Lundh
> 
> key features contributing to this is true gc, call-site 
> caching, and "traditional"
> (C-style) argument passing for common cases.

Did you collect stats on how much garbage collection actually occurred during the tests? How did it compare to stock CPython?

One of the big flaws in benchmarking early versions of Java of course was that most tests didn't end up invoking the garbage collector - hence showing an artificially good result that could not be observed in real-world usage.

Tim Delaney

From tdelaney at avaya.com  Thu Dec 11 14:57:12 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec 11 14:57:19 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0459@au3010avexu1.global.avaya.com>

> From: Duncan Booth
> 
> The interface includes methods:
> 
>    PyObject __call__();
>    PyObject __call__(PyObject arg1);
>    PyObject __call__(PyObject arg1, PyObject arg2);
>    PyObject __call__(PyObject arg1, PyObject arg2, PyObject arg3);
>    PyObject __call__(PyObject[] args);
>    PyObject __call__(PyObject[] args, PyDict keywordArgs);

Hmm - IIRC that's how Jython does it.

Tim Delaney

From pedronis at bluewin.ch  Thu Dec 11 15:52:54 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Thu Dec 11 15:49:45 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0459@au3010avexu1.global
	.avaya.com>
Message-ID: <5.2.1.1.0.20031211211518.0294e900@pop.bluewin.ch>

At 06:57 12.12.2003 +1100, Delaney, Timothy C (Timothy) wrote:
> > From: Duncan Booth
> >
> > The interface includes methods:
> >
> >    PyObject __call__();
> >    PyObject __call__(PyObject arg1);
> >    PyObject __call__(PyObject arg1, PyObject arg2);
> >    PyObject __call__(PyObject arg1, PyObject arg2, PyObject arg3);
> >    PyObject __call__(PyObject[] args);
> >    PyObject __call__(PyObject[] args, PyDict keywordArgs);
>
>Hmm - IIRC that's how Jython does it.

yup, but it doesn't try to profit from this organization as much as it 
could, and it queries some ad-hoc per-thread storage at each function call, 
instead of passing
the data along from caller to callee, which would be possibly speedier.

So there are probably margins for speeding up also Jython. OTOH I expect 
.NET to allow a larger range of tricks.

My pet peeves with .NET are that it does not garbage collect dynamically 
loaded code,
but maybe their are finally planning to support that, or will at some point.
And that there is a strong pressure (market numbers etc ...) to favor 
platform integration versus keeping the ported languages true to their 
original semantics.
(rather purist peeves). But considering things like Groovy on Java side 
maybe it is what people want. [ I think there is a tension between 
scripting the platform and having languages that can survive beyond the 
platform. OTOH given that covering with a portability layer the huge class 
libraries that come with .NET or Java is probably something people don't 
have time to, the point is rather moot and only of theoretical interest, 
OTOH I know of some few CPython/Jython projects using one or the other on 
different tier and reusing domain logic ]

.NET, Mono, Parrot, PyPy... funny times ahead. 


From kdart at kdart.com  Thu Dec 11 17:17:22 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 17:17:29 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
Message-ID: <1071181041.14243.76.camel@leviathan.kdart.com>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/2f0728a3/attachment-0001.bin
From skip at pobox.com  Thu Dec 11 17:24:34 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 11 17:24:37 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <1071181041.14243.76.camel@leviathan.kdart.com>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
Message-ID: <16344.61090.472327.729253@montanaro.dyndns.org>


    Keith> class TextFile(UserFile):
    Keith>      """TextFile(self, name, mode="r", bufsize=-1, linesep=None)
    Keith> A file object that handles different line separator
    Keith> conventions. This file object's readline() method returns the
    Keith> line with trailing line separator stripped, and raises EOFError
    Keith> on end of file. The writelines() method will append an
    Keith> appropriate line separator when writing. Thus, this file object
    Keith> allows reading and writeing non-native text files.
    Keith>      """

Is there a reason the universal newline mode doesn't cover this case
already?

    Keith>      """MergedIO(outfile, infile)
    Keith> Combines a write stream and a read stream into one read/write
    Keith> object.""" 

And isn't this subsumed by mode='r+'?

Skip

From kdart at kdart.com  Thu Dec 11 17:29:34 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 17:29:39 2003
Subject: [Python-Dev] New module proposal: timelib
Message-ID: <1071181774.14243.81.camel@leviathan.kdart.com>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/9b87a23f/attachment.bin
From kdart at kdart.com  Thu Dec 11 17:37:04 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 17:37:09 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <16344.61090.472327.729253@montanaro.dyndns.org>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
Message-ID: <1071182224.14245.89.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 14:24, Skip Montanaro wrote:
>     Keith> class TextFile(UserFile):
>     Keith>      """TextFile(self, name, mode="r", bufsize=-1, linesep=None)
>     Keith> A file object that handles different line separator
>     Keith> conventions. This file object's readline() method returns the
>     Keith> line with trailing line separator stripped, and raises EOFError
>     Keith> on end of file. The writelines() method will append an
>     Keith> appropriate line separator when writing. Thus, this file object
>     Keith> allows reading and writeing non-native text files.
>     Keith>      """
> 
> Is there a reason the universal newline mode doesn't cover this case
> already?

This module lets you explicitly create a text data file for any
platform. The universal newline feature of python is for Python modules
only, and lets you transparently import and exec Python source files
with the different line ending formats. That is a different thing. 


>     Keith>      """MergedIO(outfile, infile)
>     Keith> Combines a write stream and a read stream into one read/write
>     Keith> object.""" 
> 
> And isn't this subsumed by mode='r+'?

No, mode r+ lets you create a _single_ file descriptor that is in
read-write mode. The MergedIO lets you take who file objects (with
different file descriptors), one readable the other writable, and
combine them at the method-call level into one read-write object. This
is useful for pipes, or when you get file objects from libraries. 
 

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/92d5c829/attachment.bin
From bac at OCF.Berkeley.EDU  Thu Dec 11 17:48:39 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Dec 11 17:48:47 2003
Subject: [Python-Dev] New module proposal: timelib
In-Reply-To: <1071181774.14243.81.camel@leviathan.kdart.com>
References: <1071181774.14243.81.camel@leviathan.kdart.com>
Message-ID: <3FD8F447.7080300@ocf.berkeley.edu>

Keith Dart wrote:
> Attached is a module that I call timelib that extends the standard time
> module. This module allows easier time manipulation with a mutable time
> object, and provides some extra utility functions.
>

Browsing the code I didn't see anything that was that much better than 
what datetime offered.  The mutability is the only big difference but I 
don't see that as a critical thing in this case since they are 
relatively cheap objects to create.

And speaking of datetime, that module pretty much supercedes time.  The 
only thing in time that datetime lacks is basically strptime from my 
experience.

Personally, I would rather see any nice, new functionality be added to 
datetime so that time itself can possibly just go away.  datetime is 
just so much nicer to use in my opinion.

-Brett

From kdart at kdart.com  Thu Dec 11 17:48:59 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 17:49:05 2003
Subject: [Python-Dev] new module proposal: daemonize
Message-ID: <1071182938.14238.95.camel@leviathan.kdart.com>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/9b97aa8e/attachment.bin
From bac at OCF.Berkeley.EDU  Thu Dec 11 17:51:32 2003
From: bac at OCF.Berkeley.EDU (Brett C.)
Date: Thu Dec 11 17:51:39 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <1071182224.14245.89.camel@leviathan.kdart.com>
References: <1071181041.14243.76.camel@leviathan.kdart.com>	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
Message-ID: <3FD8F4F4.3060904@ocf.berkeley.edu>

Keith Dart wrote:
> On Thu, 2003-12-11 at 14:24, Skip Montanaro wrote:
> 
>>    Keith> class TextFile(UserFile):
>>    Keith>      """TextFile(self, name, mode="r", bufsize=-1, linesep=None)
>>    Keith> A file object that handles different line separator
>>    Keith> conventions. This file object's readline() method returns the
>>    Keith> line with trailing line separator stripped, and raises EOFError
>>    Keith> on end of file. The writelines() method will append an
>>    Keith> appropriate line separator when writing. Thus, this file object
>>    Keith> allows reading and writeing non-native text files.
>>    Keith>      """
>>
>>Is there a reason the universal newline mode doesn't cover this case
>>already?
> 
> 
> This module lets you explicitly create a text data file for any
> platform. The universal newline feature of python is for Python modules
> only, and lets you transparently import and exec Python source files
> with the different line ending formats. That is a different thing. 
> 

Not true.  Python 2.3 added the "U" option for file opening to use 
universal newline support for any file you open.  See 
http://www.python.org/doc/2.3/whatsnew/node7.html .

-Brett

From pje at telecommunity.com  Thu Dec 11 17:51:39 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec 11 17:51:54 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <1071182224.14245.89.camel@leviathan.kdart.com>
References: <16344.61090.472327.729253@montanaro.dyndns.org>
	<1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
Message-ID: <5.1.1.6.0.20031211175002.028d0470@telecommunity.com>

At 02:37 PM 12/11/03 -0800, Keith Dart wrote:
>This module lets you explicitly create a text data file for any
>platform. The universal newline feature of python is for Python modules
>only, and lets you transparently import and exec Python source files
>with the different line ending formats. That is a different thing.

Not so.  See:

http://www.python.org/doc/current/lib/built-in-funcs.html

under 'file()', which notes that 'U' or 'rU' enable universal newline mode 
for any input file.


From kdart at kdart.com  Thu Dec 11 17:52:59 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 17:53:03 2003
Subject: [Python-Dev] new module proposal: logfile
Message-ID: <1071183178.14244.99.camel@leviathan.kdart.com>

Attached is a module that implements a self-rotating log file object.
You simply write to it perpetually, and if it grows over a specified
size it is automatically truncated, and a certain number of archived
copies are kept as well. 

It also has an object that allows redirecting your stdio to a log file.
Useful for daemon processes. 

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/ac8c37c9/attachment.bin
From kdart at kdart.com  Thu Dec 11 17:56:50 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 17:56:56 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <3FD8F4F4.3060904@ocf.berkeley.edu>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
	<3FD8F4F4.3060904@ocf.berkeley.edu>
Message-ID: <1071183410.14243.103.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 14:51, Brett C. wrote:
> Keith Dart wrote:
> > This module lets you explicitly create a text data file for any
> > platform. The universal newline feature of python is for Python modules
> > only, and lets you transparently import and exec Python source files
> > with the different line ending formats. That is a different thing. 
> > 
> 
> Not true.  Python 2.3 added the "U" option for file opening to use 
> universal newline support for any file you open.  See 
> http://www.python.org/doc/2.3/whatsnew/node7.html .

Whups, something new! Anyway, that link doesn't say anything about
writing, only reading. It does not appear that you can specify what
platform to write to with U mode.

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/a6a5c62f/attachment.bin
From guido at python.org  Thu Dec 11 18:00:27 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 17:58:25 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: Your message of "Thu, 11 Dec 2003 14:37:04 PST."
	<1071182224.14245.89.camel@leviathan.kdart.com> 
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org> 
	<1071182224.14245.89.camel@leviathan.kdart.com> 
Message-ID: <200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net>

> > Is there a reason the universal newline mode doesn't cover this case
> > already?
> 
> This module lets you explicitly create a text data file for any
> platform. The universal newline feature of python is for Python modules
> only, and lets you transparently import and exec Python source files
> with the different line ending formats. That is a different thing.

I suggest you go back and read the docs for open() in Python 2.3
again, you are confused with a different feature.

I also note that you change the meaning of readline() to return the
line with the trailing separator stripped, and raise EOFError when you
have reached the end.  That's a big deviation from the standard file
semantics, and shouldn't be done lightly in a subclass (it means that
a TextFile instance is not substitutable for a regular file instance
in most cases).

Reviewing the rest of UserFile, I see several possibly useful
additions (locking, fstat() etc.) and one semantic change that
deserves to be a separate subclass or mix-in (the EINTR behavior).
And nothing that warrants the name "UserFile" (in fact, all the
other UserXXX classes are specifically *not* subclasses of the native
XXX class).

> No, mode r+ lets you create a _single_ file descriptor that is in
> read-write mode. The MergedIO lets you take who file objects (with
> different file descriptors), one readable the other writable, and
> combine them at the method-call level into one read-write object. This
> is useful for pipes, or when you get file objects from libraries.

It looks like you are simply making all write operations go to the
writable object and all the read operations go to the readable object.
That's not a very sophisticated notion of "merging", and I'm not sure
what the point is.  Perhaps some examples of how you use this would
help.

I note that in general your proposals are low on motivation, which is
going to do a lot of damage to your case if you don't fix this soon.
You may be underestimating the level of motivation needed to get
features added to the stdlib -- it's "batteries included" but also (to
some extent) "TOOWTDI".

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Thu Dec 11 17:59:54 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec 11 18:00:00 2003
Subject: [Python-Dev] new module proposal: logfile
In-Reply-To: <1071183178.14244.99.camel@leviathan.kdart.com>
Message-ID: <5.1.1.6.0.20031211175420.028d8ec0@telecommunity.com>

At 02:52 PM 12/11/03 -0800, Keith Dart wrote:
>Attached is a module that implements a self-rotating log file object.

This is already available in Python 2.3; please see 
http://www.python.org/doc/current/lib/node285.html

Perhaps it would be better for you to publish these modules on the web 
someplace, and then post the URL to python-announce?  That way, interested 
parties could look them over and try them out more easily.  You would also 
be able to get feedback from a wider audience.


From guido at python.org  Thu Dec 11 18:07:14 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 18:05:19 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: Your message of "Thu, 11 Dec 2003 14:48:59 PST."
	<1071182938.14238.95.camel@leviathan.kdart.com> 
References: <1071182938.14238.95.camel@leviathan.kdart.com> 
Message-ID: <200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>

> The attached module encapsulates all that is necessary to make your
> python script into a Unix daemon. It also provides a way to redirect
> your process's stdout to a log file. You call daemonize.daemonize() and
> when it returns your code is now a daemon (disconnected from a terminal
> and stdio).

This (or something like it) looks like a useful addition.

I remember writing a much more elaborate version for Zope:

http://cvs.zope.org/Zope/lib/python/zdaemon/

(Ignore Daemon.py, which is the old Zope API for the same thing.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec 11 18:09:23 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 18:07:20 2003
Subject: [Python-Dev] new module proposal: logfile
In-Reply-To: Your message of "Thu, 11 Dec 2003 14:52:59 PST."
	<1071183178.14244.99.camel@leviathan.kdart.com> 
References: <1071183178.14244.99.camel@leviathan.kdart.com> 
Message-ID: <200312112309.hBBN9Nd02235@c-24-5-183-134.client.comcast.net>

> Attached is a module that implements a self-rotating log file object.
> You simply write to it perpetually, and if it grows over a specified
> size it is automatically truncated, and a certain number of archived
> copies are kept as well.

The logging module also supports this.

> It also has an object that allows redirecting your stdio to a log file.
> Useful for daemon processes.

I'd be more impressed if it was integrated with the loggingmodule.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From kdart at kdart.com  Thu Dec 11 18:10:12 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 18:10:21 2003
Subject: [Python-Dev] new module proposal: logfile
In-Reply-To: <5.1.1.6.0.20031211175420.028d8ec0@telecommunity.com>
References: <5.1.1.6.0.20031211175420.028d8ec0@telecommunity.com>
Message-ID: <1071184212.14243.112.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 14:59, Phillip J. Eby wrote:
> At 02:52 PM 12/11/03 -0800, Keith Dart wrote:
> >Attached is a module that implements a self-rotating log file object.
> 
> This is already available in Python 2.3; please see 
> http://www.python.org/doc/current/lib/node285.html
> 
> Perhaps it would be better for you to publish these modules on the web 
> someplace, and then post the URL to python-announce?  That way, interested 
> parties could look them over and try them out more easily.  You would also 
> be able to get feedback from a wider audience.

They already are. These are part of pyNMS: http://sourceforge.net/projects/pynms
But I thought they might be general enough for common usage.

But I think the lesson I get from this is that I should study the Python
2.3 docs. I have not used Python 2.3 much yet. All of the stuff I have
been submitted is old, pre-2.3, and I was not aware that some of it has
been recently added. 

Sorry. 8-<

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/99b93b72/attachment.bin
From tim.one at comcast.net  Thu Dec 11 18:10:34 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Dec 11 18:10:33 2003
Subject: [Python-Dev] New module proposal: timelib
In-Reply-To: <3FD8F447.7080300@ocf.berkeley.edu>
Message-ID: <LNBBLJKPBEHFEDALKOLCCELHHLAB.tim.one@comcast.net>

[Keith Dart]
>> Attached is a module that I call timelib that extends the standard
>> time module. This module allows easier time manipulation with a
>> mutable time object, and provides some extra utility functions.

[Brett Cannon]
> Browsing the code I didn't see anything that was that much better than
> what datetime offered.  The mutability is the only big difference but
> I don't see that as a critical thing in this case since they are
> relatively cheap objects to create.

Just noting that it was considered critical for time and date objects *not*
to be mutable, else they couldn't be used as dict keys (an important use
case).


From kdart at kdart.com  Thu Dec 11 18:12:09 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 18:12:14 2003
Subject: [Python-Dev] new module proposal: logfile
In-Reply-To: <200312112309.hBBN9Nd02235@c-24-5-183-134.client.comcast.net>
References: <1071183178.14244.99.camel@leviathan.kdart.com>
	<200312112309.hBBN9Nd02235@c-24-5-183-134.client.comcast.net>
Message-ID: <1071184329.14244.115.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 15:09, Guido van Rossum wrote:
> > Attached is a module that implements a self-rotating log file object.
> > You simply write to it perpetually, and if it grows over a specified
> > size it is automatically truncated, and a certain number of archived
> > copies are kept as well.
> 
> The logging module also supports this.
> 
> > It also has an object that allows redirecting your stdio to a log file.
> > Useful for daemon processes.
> 
> I'd be more impressed if it was integrated with the loggingmodule.

Yes, ok. My modules predate the logging module (and 2.3 in general). I
should get more familiar with 2.3 first. Sorry about that.




-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/2e574e4b/attachment.bin
From tdelaney at avaya.com  Thu Dec 11 18:13:53 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec 11 18:14:01 2003
Subject: [Python-Dev] RE: Python and Temple of Elemental Evil
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04B1@au3010avexu1.global.avaya.com>

> From: Steve Moret [mailto:smoret@troikagames.com]
> 
> Hey Tim,

Hi Steve. I held off pestering you since you said you were going to do further work on ToEE over the Christmas break ;) But since you've obviously got so much time, there's that slight matter of an article about how Python was used in ToEE ... :)

I'm forwarding this to python-dev as it deals with potential modifications to the python code base (including offer of a patch :) Guido has the final say anyway ...

I'll also put some comments inline - it could be that the documentation needs to be improved rather than any code changes. In particular, I think the first two cases are simply resolved, but possibly not obvious.

BTW, I've really enjoyed ToEE (I'm currently running a 3-character party ;) but I do agree that as a game it suffered from being too close to the module (something Huy - I think - said in the wrap-up on IGN). It didn't have the Troika goodness from Fallout or Arcanum ... It's a pity that game publishers don't seem to realise that the majority of people who buy games are adults ...

> Steve Moret here, from Troika Games, now that I've had a few 
> minutes to step
> away from work (after Temple of Elemental Evil shipped) I've 
> been itching to
> give back some of the work we did on Python to the community.
> 
> One major thing (that us game developers use a lot) that the Python/C
> interface is missing (or that I just couldn't find) is a way 
> of replacing
> Python's IO with a set that will work for the game (or other custom
> project).
> 
> Two things that I've now done twice to python (and need to do 
> a third time
> in a more generic sense so I can release it to the community, 
> unless its
> already in there and I just don't know how to do it) is 1) replacing
> Python's IO (specifically stdout/stderr and stdin) so that it 
> can read/write
> to the game's libraries (traditionally this goes to a console or debug
> file),

Probably the easiest way to do this is to simply replace sys.stdin, etc. For example:

    import sys

    def main():
        pass

    if __name__ = '__main__':

        try:
            ferr = file(stderr_file, 'w')

            try:
                fout = file(stdout_file, 'w')

                try:
                    fin = file(stdin_file, 'r')

                    try:
                        sys.stdin, sys.stdout, sys.stderr = fin, fout, ferr
                        main()
                    finally:
                        fin.close()

                finally:
                    fout.close()

            finally:
                ferr.close()
        
        finally:
            sys.stdin, sys.stdout, sys.stderr = sys.__stdin__, sys.__stdout__, sys.__stderr__

(you'll note that I've done stderr as the outermost so that errors with the others will go to the error file).

stdin, stdout, stderr are in the Python 2.3 .chm index and the documentation it links to is (IMO) sufficient. Would you have been affected by the fact that changing these bindings does not affect stdin, etc as used by os.system, popen, etc?

> and 2) replacing Python's fopen/stat/fread calls that 
> do reading of
> .py and .pyc files (so that they can be placed in a game's 
> .dat file or
> other proprietary filesystem).

The normal way to do this is with an __import__ hook. This has been greatly improved in Python 2.3 (PEP 302).

> A third step can be taken 
> (which replaces the
> Python fileobject's calls to fopen/etc with the game ones, 
> but I think it's
> a better idea for the developer to write a python module that 
> encapsulates
> this on their own anyway.

Not quite sure what you mean here. What are the semantics of the file object calls that are different to what you needed?

> Traditionally we (here at Troika) have done all of the above with a
> structure/array of function pointers and initialize it to the 
> defaults. Just
> wanted to know if I were to provide a diff/patch would you 
> guys want to
> include it. Also if I do, what version should I make the 
> patch for, 2.3.2 or
> CVS or? Otherwise I'll just try to get Troika to host the patch on our
> website incase other developers want to use it.

A patch is definitely welcome, even if not accepted. In general, patches should be made against the latest development branch (2.4) and if appropriate backported to the maintenance branch (2.3.3). However, since your proposals sound like new functionality, they would not be appropriate for a 2.3.x release.

Cheers.

Tim Delaney

From fdrake at acm.org  Thu Dec 11 18:16:57 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec 11 18:19:28 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
Message-ID: <16344.64233.972116.99631@sftp.fdrake.net>


Guido van Rossum writes:
 > I remember writing a much more elaborate version for Zope:
 > 
 > http://cvs.zope.org/Zope/lib/python/zdaemon/

I'll note that we (Zope Corp.) have continued to use this package, and
have been successful making some truely ill-behaved programs work well
as daemons.  It would be *wonderful* to see something like this as
part of the standard library.

I think the only real complaint I have about the package is that the
command-line processing module (zdaemon.zdoptions) shouldn't be part
of the package, but should be somewhere else.  That shouldn't be hard
to achieve, though.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From pje at telecommunity.com  Thu Dec 11 18:20:03 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Thu Dec 11 18:20:19 2003
Subject: [Python-Dev] new module proposal: logfile
In-Reply-To: <1071184212.14243.112.camel@leviathan.kdart.com>
References: <5.1.1.6.0.20031211175420.028d8ec0@telecommunity.com>
	<5.1.1.6.0.20031211175420.028d8ec0@telecommunity.com>
Message-ID: <5.1.1.6.0.20031211181852.01db6e60@telecommunity.com>

At 03:10 PM 12/11/03 -0800, Keith Dart wrote:

>But I think the lesson I get from this is that I should study the Python
>2.3 docs. I have not used Python 2.3 much yet. All of the stuff I have
>been submitted is old, pre-2.3, and I was not aware that some of it has
>been recently added.

A good place to start is:

http://www.python.org/doc/current/whatsnew/whatsnew23.html


From barry at python.org  Thu Dec 11 18:25:02 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 11 18:25:21 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
Message-ID: <1071185101.9905.83.camel@anthem>

On Thu, 2003-12-11 at 18:07, Guido van Rossum wrote:

> This (or something like it) looks like a useful addition.
> 
> I remember writing a much more elaborate version for Zope:
> 
> http://cvs.zope.org/Zope/lib/python/zdaemon/
> 
> (Ignore Daemon.py, which is the old Zope API for the same thing.)

I have something somewhat similar in mailman (although with lots of
extra goo):

http://cvs.sourceforge.net/viewcvs.py/mailman/mailman/bin/mailmanctl?view=markup

Still, something like this would be a nice addition to the standard
library.  Save's having to read Stevens APitUE Ch. 13.  <wink>.

-Barry





From barry at python.org  Thu Dec 11 18:27:26 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 11 18:27:34 2003
Subject: [Python-Dev] new module proposal: logfile
In-Reply-To: <1071184212.14243.112.camel@leviathan.kdart.com>
References: <5.1.1.6.0.20031211175420.028d8ec0@telecommunity.com>
	<1071184212.14243.112.camel@leviathan.kdart.com>
Message-ID: <1071185246.9905.86.camel@anthem>

On Thu, 2003-12-11 at 18:10, Keith Dart wrote:

> But I think the lesson I get from this is that I should study the Python
> 2.3 docs. I have not used Python 2.3 much yet. All of the stuff I have
> been submitted is old, pre-2.3, and I was not aware that some of it has
> been recently added. 

Let me just encourage you to keep contributing to Python!  Don't let all
us ornery curmudgeons discourage you. :)  I'd also suggest that for the
next round, submit SourceForge patches containing your code.  It's way
too easy for code contributions to get lost in inboxes and archives.

-Barry
 


From tdelaney at avaya.com  Thu Dec 11 18:30:39 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec 11 18:30:46 2003
Subject: [Python-Dev] "What's New" in docs
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04BF@au3010avexu1.global.avaya.com>

The recent posts, primarily from Keith and Steve (forwarded by me) suggest to me that it might be useful to keep a historical "What's New" section in the documentation, rather than just for the current version.

For example, currently the table of contents in the python docs is:

Main Page
Global Module Index
What's New
    1 PEP 218: A Standard Set Datatype
    2 PEP 255: Simple Generators
.
.
.

Would anyone see any advantage to having the "What's New" section like:

What's New in...
    Python 2.3
        1 PEP 218: A Standard Set Datatype
    Python 2.2
        1 PEP 234: Iterators
.
.
.

I use different versions of Python for different projects (in particular, 2.1 and 2.3) and I think this would be a useful quick reference.

Tim Delaney

From barry at python.org  Thu Dec 11 18:34:38 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 11 18:34:47 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <16344.64233.972116.99631@sftp.fdrake.net>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
	<16344.64233.972116.99631@sftp.fdrake.net>
Message-ID: <1071185677.9905.92.camel@anthem>

On Thu, 2003-12-11 at 18:16, Fred L. Drake, Jr. wrote:

> I think the only real complaint I have about the package is that the
> command-line processing module (zdaemon.zdoptions) shouldn't be part
> of the package, but should be somewhere else.  That shouldn't be hard
> to achieve, though.

I've been trying to use Greg's optparse package (new in 2.3) on some
other ZC projects, and for the most part I like it.  It has a different
rhythm than getopt, but has some nice properties over what I had been
doing, especially I suspect for i18n'd help.

I have a few minor quibbles, nits, and bug reports on it, but I'm sold
enough to be converting all my getopt-based option parsing over to it
for Mailman3.

Does zdoptions provide anything that isn't already in supported in
optparse?

-Barry



From fumanchu at amor.org  Thu Dec 11 18:37:12 2003
From: fumanchu at amor.org (Robert Brewer)
Date: Thu Dec 11 18:42:34 2003
Subject: [Python-Dev] "What's New" in docs
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561B5B@opus.amorhq.net>

I'd appreciate this, or something like it. At the least, it'd be nice to
have a link at the bottom of the current "What's New" that points to the
last "What's New", or to a complete index on another page.


Robert Brewer
MIS
Amor Ministries
fumanchu@amor.org

> -----Original Message-----
> From: Delaney, Timothy C (Timothy) [mailto:tdelaney@avaya.com] 
> Sent: Thursday, December 11, 2003 3:31 PM
> To: Python-Dev (E-mail)
> Subject: [Python-Dev] "What's New" in docs
> 
> 
> The recent posts, primarily from Keith and Steve (forwarded 
> by me) suggest to me that it might be useful to keep a 
> historical "What's New" section in the documentation, rather 
> than just for the current version.
> 
> For example, currently the table of contents in the python docs is:
> 
> Main Page
> Global Module Index
> What's New
>     1 PEP 218: A Standard Set Datatype
>     2 PEP 255: Simple Generators
> .
> .
> .
> 
> Would anyone see any advantage to having the "What's New" 
> section like:
> 
> What's New in...
>     Python 2.3
>         1 PEP 218: A Standard Set Datatype
>     Python 2.2
>         1 PEP 234: Iterators
> .
> .
> .
> 
> I use different versions of Python for different projects (in 
> particular, 2.1 and 2.3) and I think this would be a useful 
> quick reference.
> 
> Tim Delaney
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python->
dev/fumanchu%40amor.org
> 

From fdrake at acm.org  Thu Dec 11 18:43:21 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec 11 18:43:31 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <1071185677.9905.92.camel@anthem>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
	<16344.64233.972116.99631@sftp.fdrake.net>
	<1071185677.9905.92.camel@anthem>
Message-ID: <16345.281.537830.208845@sftp.fdrake.net>


Barry Warsaw writes:
 > Does zdoptions provide anything that isn't already in supported in
 > optparse?

Direct support for ZConfig.  ;-)  That doesn't belong in the standard
library... yet.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From barry at python.org  Thu Dec 11 18:47:22 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 11 18:47:31 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <16345.281.537830.208845@sftp.fdrake.net>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
	<16344.64233.972116.99631@sftp.fdrake.net>
	<1071185677.9905.92.camel@anthem>
	<16345.281.537830.208845@sftp.fdrake.net>
Message-ID: <1071186441.9905.95.camel@anthem>

On Thu, 2003-12-11 at 18:43, Fred L. Drake, Jr. wrote:
> Barry Warsaw writes:
>  > Does zdoptions provide anything that isn't already in supported in
>  > optparse?
> 
> Direct support for ZConfig.  ;-)  That doesn't belong in the standard
> library... yet.

When you get your wiki going and fix my complaints, we can talk about it
<wink>.

Actually, ZConfig might indeed be a candidate for inclusion.  It does
provide a much richer configuration system than the existing standard
libraries.

-Barry



From guido at python.org  Thu Dec 11 18:53:41 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 18:52:08 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: Your message of "Thu, 11 Dec 2003 15:37:12 PST."
	<DE1CF2B4FEC4A342BF62B6B2B334601E561B5B@opus.amorhq.net> 
References: <DE1CF2B4FEC4A342BF62B6B2B334601E561B5B@opus.amorhq.net> 
Message-ID: <200312112353.hBBNrf502386@c-24-5-183-134.client.comcast.net>

> I'd appreciate this, or something like it. At the least, it'd be nice to
> have a link at the bottom of the current "What's New" that points to the
> last "What's New", or to a complete index on another page.

Well, it's not exactly difficult to find the previous versions of the
docs from http://www.python.org/doc/ (hint: search for "Previous
versions" :-).

Isn't that enough?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tdelaney at avaya.com  Thu Dec 11 19:01:58 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Thu Dec 11 19:02:05 2003
Subject: [Python-Dev] RE: Python and Temple of Elemental Evil
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04D8@au3010avexu1.global.avaya.com>

> From: Guido van Rossum
> 
> > I'm forwarding this to python-dev as it deals with potential
> > modifications to the python code base (including offer of a patch :)
> > Guido has the final say anyway ...
> 
> The formatting of the message and the extensive quoting and references
> to a game made me lose track of what was suggested.  Could you
> summarize a bit?

OK - distilling down to the minimum I think is useful ...

[ Steve was unable to work out how to redirect stderr, stdout and stdin ]

[ I suggested using sys.stdin, etc - subject to the caveat that it doesn't
  work for os.system, etc - there may be a documentation issue here though
  I don't think so - finding the relevant info in the help was simple ].

[ Steve was unable to work out how to import from a proprietory format (.dat) ]

[ I suggested an import hook and pointed to PEP 302 (although ToEE uses
  Python 2.2). There may be a need for further documentation/pointers here. ]

[ The following is relevant to python-dev IMO - I should have trimmed further
  than I did in the first case :( Steve suggests that the file object semantics
  did not meet his needs, and offers a patch which I expect would be rejected,
  but may at least give a use case for consideration. ]

> A third step can be taken 
> (which replaces the
> Python fileobject's calls to fopen/etc with the game ones, 
> but I think it's
> a better idea for the developer to write a python module that 
> encapsulates
> this on their own anyway.

Not quite sure what you mean here. What are the semantics of the file object calls that are different to what you needed?

> Traditionally we (here at Troika) have done all of the above with a
> structure/array of function pointers and initialize it to the 
> defaults. Just
> wanted to know if I were to provide a diff/patch would you 
> guys want to
> include it. Also if I do, what version should I make the 
> patch for, 2.3.2 or
> CVS or? Otherwise I'll just try to get Troika to host the patch on our
> website incase other developers want to use it.

A patch is definitely welcome, even if not accepted. In general, patches should be made against the latest development branch (2.4) and if appropriate backported to the maintenance branch (2.3.3). However, since your proposals sound like new functionality, they would not be appropriate for a 2.3.x release.

Cheers.

Tim Delaney

From kdart at kdart.com  Thu Dec 11 19:04:45 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 19:04:51 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
	<200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net>
Message-ID: <1071187485.14246.134.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 15:00, Guido van Rossum wrote:

> I also note that you change the meaning of readline() to return the
> line with the trailing separator stripped, and raise EOFError when you
> have reached the end.  That's a big deviation from the standard file
> semantics, and shouldn't be done lightly in a subclass (it means that
> a TextFile instance is not substitutable for a regular file instance
> in most cases).

Yes, that is true, but I have had to use that semantic in other places
(usually when using non-blocking sockets) and I personally prefer it.
But as others have pointed out this functionality is not needed in
Python 2.3 and so I will drop it.  

> Reviewing the rest of UserFile, I see several possibly useful
> additions (locking, fstat() etc.) and one semantic change that
> deserves to be a separate subclass or mix-in (the EINTR behavior).

> And nothing that warrants the name "UserFile" (in fact, all the
> other UserXXX classes are specifically *not* subclasses of the native
> XXX class).

Yes, a lot of extra stuff was added later. Perhaps some should be
factored out into a new module. Perhaps "filelib"? It was just more
consistent for me, since I also have UserInt, UserFloat, and UserLong
modules as well. But those are, thankfully, no longer needed either.


> > No, mode r+ lets you create a _single_ file descriptor that is in
> > read-write mode. The MergedIO lets you take who file objects (with
> > different file descriptors), one readable the other writable, and
> > combine them at the method-call level into one read-write object. This
> > is useful for pipes, or when you get file objects from libraries.
> 
> It looks like you are simply making all write operations go to the
> writable object and all the read operations go to the readable object.
> That's not a very sophisticated notion of "merging", and I'm not sure
> what the point is.  Perhaps some examples of how you use this would
> help.

A quick example:

import os
import UserFile

r, w = os.pipe()
rf = os.fdopen(r)
wf = os.fdopen(w)

fo = UserFile.MergedIO(rf, wf)

_or_

rwio = UserFile.MergedIO(sys.stdin, sys.stdout)


Perhaps "merged" is not the right word. It is not intended to merge
files, but combine a separate read file object and write file object
into one object that can be passed to other library methods that want a
single read/write object (such as my "expect" module...) 


> I note that in general your proposals are low on motivation, which is
> going to do a lot of damage to your case if you don't fix this soon.
> You may be underestimating the level of motivation needed to get
> features added to the stdlib -- it's "batteries included" but also (to
> some extent) "TOOWTDI".

Hm, ok. more use cases? What I have submitted, and will submit, exist
because I did need them for real work at some point, and have found them
to be re-usable.

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/912b733d/attachment-0001.bin
From guido at python.org  Thu Dec 11 19:16:37 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 19:14:36 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: Your message of "Thu, 11 Dec 2003 16:04:45 PST."
	<1071187485.14246.134.camel@leviathan.kdart.com> 
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
	<200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net> 
	<1071187485.14246.134.camel@leviathan.kdart.com> 
Message-ID: <200312120016.hBC0Gbp02447@c-24-5-183-134.client.comcast.net>

> > It looks like you are simply making all write operations go to the
> > writable object and all the read operations go to the readable object.
> > That's not a very sophisticated notion of "merging", and I'm not sure
> > what the point is.  Perhaps some examples of how you use this would
> > help.
> 
> A quick example:
> 
> import os
> import UserFile
> 
> r, w = os.pipe()
> rf = os.fdopen(r)
> wf = os.fdopen(w)
> 
> fo = UserFile.MergedIO(rf, wf)
> 
> _or_
> 
> rwio = UserFile.MergedIO(sys.stdin, sys.stdout)

That's not a motivation.  How does writing rwio.readline() and
rw.write() enable your program to do something that wasn't possible by
using sys.stdin and sys.stdout directly?

> Perhaps "merged" is not the right word. It is not intended to merge
> files, but combine a separate read file object and write file object
> into one object that can be passed to other library methods that want a
> single read/write object (such as my "expect" module...)

Sounds like an API design bug in the expect module to me. :-)

> > I note that in general your proposals are low on motivation, which is
> > going to do a lot of damage to your case if you don't fix this soon.
> > You may be underestimating the level of motivation needed to get
> > features added to the stdlib -- it's "batteries included" but also (to
> > some extent) "TOOWTDI".
> 
> Hm, ok. more use cases? What I have submitted, and will submit, exist
> because I did need them for real work at some point, and have found them
> to be re-usable.

I don't doubt that.  But you need to *show* how they are useful for
others, and you need to show that you have thought about how they
would work well together with the rest of the stdio library.  (This is
where strip-newline-raise-EOFError breaks down.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From andrew-pythondev at puzzling.org  Thu Dec 11 19:16:26 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu Dec 11 19:16:34 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <1071185101.9905.83.camel@anthem>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
	<1071185101.9905.83.camel@anthem>
Message-ID: <20031212001626.GA25007@frobozz>

On Thu, Dec 11, 2003 at 06:25:02PM -0500, Barry Warsaw wrote:
> On Thu, 2003-12-11 at 18:07, Guido van Rossum wrote:
> 
> > This (or something like it) looks like a useful addition.
> > 
> > I remember writing a much more elaborate version for Zope:
> > 
> > http://cvs.zope.org/Zope/lib/python/zdaemon/
> > 
> > (Ignore Daemon.py, which is the old Zope API for the same thing.)
> 
> I have something somewhat similar in mailman (although with lots of
> extra goo):
> 
> http://cvs.sourceforge.net/viewcvs.py/mailman/mailman/bin/mailmanctl?view=markup

...And Twisted has its own version of this stuff, too:

http://cvs.twistedmatrix.com/cvs/twisted/scripts/twistd.py?rev=1.65&content-type=text/vnd.viewcvs-markup

*And* there's even a cookbook recipe for it (with lots of debate about the
details):

http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012

I'm sure there are plenty of other projects that have reinvented this, too.
Having daemonization code in the standard library would be good.

-Andrew.


From kdart at kdart.com  Thu Dec 11 19:22:39 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 19:22:45 2003
Subject: [Python-Dev] new module: scheduler
Message-ID: <1071188559.14245.141.camel@leviathan.kdart.com>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/3662d96a/attachment.bin
From kdart at kdart.com  Thu Dec 11 19:26:15 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 19:26:20 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <20031212001626.GA25007@frobozz>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
	<1071185101.9905.83.camel@anthem>  <20031212001626.GA25007@frobozz>
Message-ID: <1071188775.14243.146.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 16:16, Andrew Bennetts wrote:
> ...And Twisted has its own version of this stuff, too:
> 
> http://cvs.twistedmatrix.com/cvs/twisted/scripts/twistd.py?rev=1.65&content-type=text/vnd.viewcvs-markup
> 
> *And* there's even a cookbook recipe for it (with lots of debate about the
> details):
> 
> http://aspn.activestate.com/ASPN/Cookbook/Python/Recipe/66012
> 
> I'm sure there are plenty of other projects that have reinvented this, too.
> Having daemonization code in the standard library would be good.

Yes, I think I got some of the code in daemonize from the cookbook. I
had been using a little C program for that (e.g.: "daemonize python
pydaemon.py" ) but that required invoking from a shell, or remebering to
use the daemonize tool. With the daemonize module I could do it from
within python in a standard way. It may need some tweaking to make it
more portable. 

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/04ce81d1/attachment.bin
From guido at python.org  Thu Dec 11 19:36:37 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 19:34:38 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: Your message of "Thu, 11 Dec 2003 16:26:15 PST."
	<1071188775.14243.146.camel@leviathan.kdart.com> 
References: <1071182938.14238.95.camel@leviathan.kdart.com>
	<200312112307.hBBN7EW02209@c-24-5-183-134.client.comcast.net>
	<1071185101.9905.83.camel@anthem> <20031212001626.GA25007@frobozz> 
	<1071188775.14243.146.camel@leviathan.kdart.com> 
Message-ID: <200312120036.hBC0ab102521@c-24-5-183-134.client.comcast.net>

> Yes, I think I got some of the code in daemonize from the cookbook. I
> had been using a little C program for that (e.g.: "daemonize python
> pydaemon.py" ) but that required invoking from a shell, or remebering to
> use the daemonize tool. With the daemonize module I could do it from
> within python in a standard way. It may need some tweaking to make it
> more portable.

I note that the original Zope approachg to daemonizing was also like
you do it, from inside.  But when we decided to rewrite it, we found
that it was much more general to have an external program...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From kdart at kdart.com  Thu Dec 11 19:35:28 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 19:35:33 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <200312120016.hBC0Gbp02447@c-24-5-183-134.client.comcast.net>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
	<200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net>
	<1071187485.14246.134.camel@leviathan.kdart.com>
	<200312120016.hBC0Gbp02447@c-24-5-183-134.client.comcast.net>
Message-ID: <1071189327.14246.154.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 16:16, Guido van Rossum wrote:
> > rwio = UserFile.MergedIO(sys.stdin, sys.stdout)
> 
> That's not a motivation.  How does writing rwio.readline() and
> rw.write() enable your program to do something that wasn't possible by
> using sys.stdin and sys.stdout directly?

_MY_ programs could do that, but what if you got some third party module
that wanted a single read/write file object and you wanted to use it on
stdio? 

It makes APIs more regular and consistent by requiring only one
read/write object. For example, the library method could be passed a
single read/write Unix socket, or a read/write TCP socket, or stdio
object (MergedIO object). 

> > Perhaps "merged" is not the right word. It is not intended to merge
> > files, but combine a separate read file object and write file object
> > into one object that can be passed to other library methods that want a
> > single read/write object (such as my "expect" module...)
> 
> Sounds like an API design bug in the expect module to me. :-)

Nope, that's a feature. ;-)


> I don't doubt that.  But you need to *show* how they are useful for
> others, and you need to show that you have thought about how they
> would work well together with the rest of the stdio library.  (This is
> where strip-newline-raise-EOFError breaks down.)

Ok, this one can be removed then. It would not be commonly used anyway.

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/3761f30d/attachment.bin
From kdart at kdart.com  Thu Dec 11 19:52:14 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 19:52:21 2003
Subject: [Python-Dev] new module proposal: expect
Message-ID: <1071190333.14243.163.camel@leviathan.kdart.com>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/10e66d73/attachment-0001.bin
From guido at python.org  Thu Dec 11 19:57:05 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 19:55:02 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: Your message of "Thu, 11 Dec 2003 16:35:28 PST."
	<1071189327.14246.154.camel@leviathan.kdart.com> 
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
	<200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net>
	<1071187485.14246.134.camel@leviathan.kdart.com>
	<200312120016.hBC0Gbp02447@c-24-5-183-134.client.comcast.net> 
	<1071189327.14246.154.camel@leviathan.kdart.com> 
Message-ID: <200312120057.hBC0v5602569@c-24-5-183-134.client.comcast.net>

> > > rwio = UserFile.MergedIO(sys.stdin, sys.stdout)

> > That's not a motivation.  How does writing rwio.readline() and
> > rw.write() enable your program to do something that wasn't
> > possible by using sys.stdin and sys.stdout directly?
> 
> _MY_ programs could do that, but what if you got some third party
> module that wanted a single read/write file object and you wanted to
> use it on stdio?

> It makes APIs more regular and consistent by requiring only one
> read/write object. For example, the library method could be passed a
> single read/write Unix socket, or a read/write TCP socket, or stdio
> object (MergedIO object).

Well, the standard way of wrapping a file object around a socket
pretty much forces you to have separate read and write objects; check
out the makefile() method. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Thu Dec 11 19:57:07 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Dec 11 19:57:10 2003
Subject: [Python-Dev] new module proposal: expect
In-Reply-To: <1071190333.14243.163.camel@leviathan.kdart.com>
References: <1071190333.14243.163.camel@leviathan.kdart.com>
Message-ID: <20031212005707.GA26221@panix.com>

On Thu, Dec 11, 2003, Keith Dart wrote:
>
> It is sometimes necessary to interact with other programs, or users, as
> a user would via a command line interface. This is especially common in
> QA environments. A program called "expect", which is based on the Tcl
> language, has commonly been used for this. But that program is, well,
> not Python. ;-) Having a general purpose "expect" module for Python
> means that Python could be used out-of-the-box for the same tasks that
> the older "expect" program is used for. The attached expect.py module
> provides most of the core functionality of the original expect tool.
> Other functionality from the "expect" tool is duplicated in other Python
> modules. 

How does this differ from http://pexpect.sourceforge.net/
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From guido at python.org  Thu Dec 11 19:59:22 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 19:57:19 2003
Subject: [Python-Dev] new module proposal: expect
In-Reply-To: Your message of "Thu, 11 Dec 2003 16:52:14 PST."
	<1071190333.14243.163.camel@leviathan.kdart.com> 
References: <1071190333.14243.163.camel@leviathan.kdart.com> 
Message-ID: <200312120059.hBC0xM502597@c-24-5-183-134.client.comcast.net>

> It is sometimes necessary to interact with other programs, or users, as
> a user would via a command line interface. This is especially common in
> QA environments. A program called "expect", which is based on the Tcl
> language, has commonly been used for this. But that program is, well,
> not Python. ;-) Having a general purpose "expect" module for Python
> means that Python could be used out-of-the-box for the same tasks that
> the older "expect" program is used for. The attached expect.py module
> provides most of the core functionality of the original expect tool.
> Other functionality from the "expect" tool is duplicated in other Python
> modules.
> 
> It is hoped that this will enable non-Python users to easily replace any
> "expect" installations they have with Python. Thus, growing the Python
> market. Having this functionality as part of the core distribution would
> facilitate that.

Good, this is the kind of module that would make the Python standard
library more useful.  However, I've seen many implementations of this
before.  Before yours gets accepted into the standard library, you
have to convince us that your version is "best of breed".  That's been
done before: the logging package and the optparse module came about
this way.  But you have to do the research to compare your version
with others.  I recommend that you take this to c.l.py first.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From kdart at kdart.com  Thu Dec 11 20:04:18 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 20:04:24 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <200312120057.hBC0v5602569@c-24-5-183-134.client.comcast.net>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<16344.61090.472327.729253@montanaro.dyndns.org>
	<1071182224.14245.89.camel@leviathan.kdart.com>
	<200312112300.hBBN0RU02156@c-24-5-183-134.client.comcast.net>
	<1071187485.14246.134.camel@leviathan.kdart.com>
	<200312120016.hBC0Gbp02447@c-24-5-183-134.client.comcast.net>
	<1071189327.14246.154.camel@leviathan.kdart.com>
	<200312120057.hBC0v5602569@c-24-5-183-134.client.comcast.net>
Message-ID: <1071191058.14244.168.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 16:57, Guido van Rossum wrote:
> > > > rwio = UserFile.MergedIO(sys.stdin, sys.stdout)
> 
> > > That's not a motivation.  How does writing rwio.readline() and
> > > rw.write() enable your program to do something that wasn't
> > > possible by using sys.stdin and sys.stdout directly?
> > 
> > _MY_ programs could do that, but what if you got some third party
> > module that wanted a single read/write file object and you wanted to
> > use it on stdio?
> 
> > It makes APIs more regular and consistent by requiring only one
> > read/write object. For example, the library method could be passed a
> > single read/write Unix socket, or a read/write TCP socket, or stdio
> > object (MergedIO object).
> 
> Well, the standard way of wrapping a file object around a socket
> pretty much forces you to have separate read and write objects; check
> out the makefile() method. :-)

Ok, ok. Uncle! ;-) Guess what? I have never used MergedIO myself. It was
partly an exercise combining object functionality in an efficient manner
(by re-assigning method objects in the instance's namespace). 
I can drop that one. But... there is a variant in another module that I
use.  

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/be48b990/attachment.bin
From kdart at kdart.com  Thu Dec 11 20:14:21 2003
From: kdart at kdart.com (Keith Dart)
Date: Thu Dec 11 20:14:26 2003
Subject: [Python-Dev] new module proposal: expect
In-Reply-To: <200312120059.hBC0xM502597@c-24-5-183-134.client.comcast.net>
References: <1071190333.14243.163.camel@leviathan.kdart.com>
	<200312120059.hBC0xM502597@c-24-5-183-134.client.comcast.net>
Message-ID: <1071191660.14243.178.camel@leviathan.kdart.com>

On Thu, 2003-12-11 at 16:59, Guido van Rossum wrote:
> Good, this is the kind of module that would make the Python standard
> library more useful.  However, I've seen many implementations of this
> before.  Before yours gets accepted into the standard library, you
> have to convince us that your version is "best of breed".  That's been
> done before: the logging package and the optparse module came about
> this way.  But you have to do the research to compare your version
> with others.  I recommend that you take this to c.l.py first.

On Thu, 2003-12-11 at 16:57, Aahz wrote: 
> How does this differ from http://pexpect.sourceforge.net/

I knew you would ask that. I did look at those others, and compared
them. But ultimately I wrote my own from scratch. Of course, I like mine
a lot better... I will provide more details later, as it would require
quite a treatise on that subject. 

-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/7929b774/attachment.bin
From aahz at pythoncraft.com  Thu Dec 11 20:20:48 2003
From: aahz at pythoncraft.com (Aahz)
Date: Thu Dec 11 20:20:52 2003
Subject: [Python-Dev] new module proposal: expect
In-Reply-To: <1071191660.14243.178.camel@leviathan.kdart.com>
References: <1071190333.14243.163.camel@leviathan.kdart.com>
	<200312120059.hBC0xM502597@c-24-5-183-134.client.comcast.net>
	<1071191660.14243.178.camel@leviathan.kdart.com>
Message-ID: <20031212012048.GA2401@panix.com>

On Thu, Dec 11, 2003, Keith Dart wrote:
> On Thu, 2003-12-11 at 16:57, Aahz wrote: 
>>
>> How does this differ from http://pexpect.sourceforge.net/
> 
> I knew you would ask that. I did look at those others, and compared
> them. But ultimately I wrote my own from scratch. Of course, I like mine
> a lot better... I will provide more details later, as it would require
> quite a treatise on that subject. 

This will almost certainly require a PEP, I think; might as well start
working on that now.  ;-)
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From wsanchez at apple.com  Thu Dec 11 21:12:19 2003
From: wsanchez at apple.com (=?ISO-8859-1?Q?Wilfredo_S=E1nchez?=)
Date: Thu Dec 11 21:12:28 2003
Subject: [Python-Dev] new module proposal: daemonize
In-Reply-To: <1071182938.14238.95.camel@leviathan.kdart.com>
References: <1071182938.14238.95.camel@leviathan.kdart.com>
Message-ID: <9DB7F042-2C48-11D8-B711-000A95AFAE2C@apple.com>

Skipped content of type multipart/mixed-------------- next part --------------
A non-text attachment was scrubbed...
Name: smime.p7s
Type: application/pkcs7-signature
Size: 2406 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20031211/e2ae17aa/smime-0001.bin
From devin at whitebread.org  Fri Dec 12 00:34:31 2003
From: devin at whitebread.org (Devin)
Date: Thu Dec 11 22:26:02 2003
Subject: [Python-Dev] new module: scheduler
In-Reply-To: <1071188559.14245.141.camel@leviathan.kdart.com>
Message-ID: <Pine.LNX.4.44.0312112047360.16272-100000@whitebread.org>

I wanted to lurk a while longer before posting again, but I feel I have 
something worthwhile to add.

On Thu, 11 Dec 2003, Keith Dart wrote:

> Whew, ok. Here is another one. Perhaps this one is misnamed, but it is
> called scheduler and takes over SIGALRM to provide running callback
> functions at scheduled intervals or time delays. Also provides an
> alternate implementation for "sleep". It operates asynchronously. That
> is, your main thread of execution still runs and is interrupted by
> SIGALRM and your specified callbacks are run.
> 
> I am aware of the sched module, but that one is for threads, and is not
> asynchronous (your main thread blocks in the run() method).

A little more than a year ago, I also wrote a threaded scheduler.  The
code can be found here:

    http://www.whitebread.org/tidbits/scheduler

Since then, I've found the functionality of a threaded scheduler to be
very useful, and believe that _a_ threaded scheduler (not necessarily my
impementation) would be a nice addition to the standard library.

Good suggestion, Keith. :)

::Lurk mode on::

-- 
Devin
devin@whitebread.org
http://www.whitebread.org/


From skip at pobox.com  Thu Dec 11 22:53:25 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 11 22:53:24 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: <200312112353.hBBNrf502386@c-24-5-183-134.client.comcast.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E561B5B@opus.amorhq.net>
	<200312112353.hBBNrf502386@c-24-5-183-134.client.comcast.net>
Message-ID: <16345.15285.583427.717748@montanaro.dyndns.org>


    >> ... it'd be nice to have a link at the bottom of the current "What's
    >> New" that points to the last "What's New", or to a complete index on
    >> another page.

    Guido> Well, it's not exactly difficult to find the previous versions of
    Guido> the docs from http://www.python.org/doc/ (hint: search for
    Guido> "Previous versions" :-).

    Guido> Isn't that enough?

Sure, I suppose it's sufficient, but perhaps a bit cumbersome.  If you're
trying to remember when some feature was added to the language (which is
fairly common if you're trying to support an app on multiple versions of the
interpreter I think), chaining straight back through What's New sections
would make it easier.

Skip


From guido at python.org  Thu Dec 11 23:41:01 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 11 23:39:00 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: Your message of "Thu, 11 Dec 2003 21:53:25 CST."
	<16345.15285.583427.717748@montanaro.dyndns.org> 
References: <DE1CF2B4FEC4A342BF62B6B2B334601E561B5B@opus.amorhq.net>
	<200312112353.hBBNrf502386@c-24-5-183-134.client.comcast.net> 
	<16345.15285.583427.717748@montanaro.dyndns.org> 
Message-ID: <200312120441.hBC4f1602858@c-24-5-183-134.client.comcast.net>

>     Guido> Well, it's not exactly difficult to find the previous versions of
>     Guido> the docs from http://www.python.org/doc/ (hint: search for
>     Guido> "Previous versions" :-).
> 
>     Guido> Isn't that enough?
> 
> Sure, I suppose it's sufficient, but perhaps a bit cumbersome.  If you're
> trying to remember when some feature was added to the language (which is
> fairly common if you're trying to support an app on multiple versions of the
> interpreter I think), chaining straight back through What's New sections
> would make it easier.

The standard docs are sprinkled with "new in version 2.x" as well.

So I'm still questioning the value of keeping the "what's new in 2.x"
document around as part of the 2.(x+1) docs.  If you work with 2.x and
2.y, you should probably keep bookmarks for both sets of docs...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From python at rcn.com  Fri Dec 12 00:14:57 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec 12 00:15:09 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: <200312120441.hBC4f1602858@c-24-5-183-134.client.comcast.net>
Message-ID: <000c01c3c06e$e346a820$e841fea9@oemcomputer>

> >     Guido> Well, it's not exactly difficult to find the previous
> versions of
> >     Guido> the docs from http://www.python.org/doc/ (hint: search
for
> >     Guido> "Previous versions" :-).
> >
> >     Guido> Isn't that enough?

My preference is to keep What's New separate.  For most users, the
primary value is in the moment of transition during an upgrade.  Also,
with the Mac people coming aboard, we'll have more users that have never
seen a previous version of Python.  Ideally, they should see Python as
being in a state of highest evolution without seeing the whole history.

For someone truly interested in writing code for multiple python
versions, the whatsnew history is neither sufficiently complete nor
detailed.  Instead, the text and examples are written as gentle
introductions to features.  When Skip looks up the introduction of list
comps or generators, the odds are that he only reads the headlines and
would be bored to tears by the other 98% of the document.  When the
introduction of CHM files, searching the regular docs has never been
easier.


Raymond Hettinger



From fumanchu at amor.org  Fri Dec 12 01:15:34 2003
From: fumanchu at amor.org (Robert Brewer)
Date: Fri Dec 12 01:20:55 2003
Subject: [Python-Dev] "What's New" in docs
Message-ID: <DE1CF2B4FEC4A342BF62B6B2B334601E561B5C@opus.amorhq.net>

Guido van Rossum wrote:
> > I'd appreciate this, or something like it. At the least, 
> it'd be nice to
> > have a link at the bottom of the current "What's New" that 
> points to the
> > last "What's New", or to a complete index on another page.
> 
> Well, it's not exactly difficult to find the previous versions of the
> docs from http://www.python.org/doc/ (hint: search for "Previous
> versions" :-).
> 
> Isn't that enough?

For the moment, it works. It's more a matter of indexing and navigation
than having "a way" to find it--I guess I'm officially from the
"one-click" generation. As Python releases more versions, it will
naturally become less manageable. "What's New" is basically 'indexed' by
version number currently. It'd be nice to have a feature index, for
reverse lookups. Hope that makes sense. My thought about having a
pointer to the previous "What's New" would speed navigation through
several versions, if an index were not appropriate.

Me, I just use Google. :)


FuManChu

From barry at python.org  Fri Dec 12 08:03:09 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Dec 12 08:03:24 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: <200312120441.hBC4f1602858@c-24-5-183-134.client.comcast.net>
References: <DE1CF2B4FEC4A342BF62B6B2B334601E561B5B@opus.amorhq.net>
	<200312112353.hBBNrf502386@c-24-5-183-134.client.comcast.net>
	<16345.15285.583427.717748@montanaro.dyndns.org>
	<200312120441.hBC4f1602858@c-24-5-183-134.client.comcast.net>
Message-ID: <1071234188.9905.114.camel@anthem>

On Thu, 2003-12-11 at 23:41, Guido van Rossum wrote:

> The standard docs are sprinkled with "new in version 2.x" as well.
> 
> So I'm still questioning the value of keeping the "what's new in 2.x"
> document around as part of the 2.(x+1) docs.  If you work with 2.x and
> 2.y, you should probably keep bookmarks for both sets of docs...

PEP 291 also has a fairly high level overview of when certain features
were introduced.

-Barry



From mwh at python.net  Fri Dec 12 09:29:06 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 12 09:29:09 2003
Subject: [Python-Dev] Re: a serious threat to 2.3's speed?
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0458@au3010avexu1.global.avaya.com>
	(Timothy
	C. Delaney's message of "Fri, 12 Dec 2003 06:55:09 +1100")
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0458@au3010avexu1.global.avaya.com>
Message-ID: <2mu146kt3h.fsf@starship.python.net>

"Delaney, Timothy C (Timothy)" <tdelaney@avaya.com> writes:

>> From: Fredrik Lundh
>> 
>> key features contributing to this is true gc, call-site 
>> caching, and "traditional"
>> (C-style) argument passing for common cases.
>
> Did you collect stats on how much garbage collection actually
> occurred during the tests? How did it compare to stock CPython?
>
> One of the big flaws in benchmarking early versions of Java of
> course was that most tests didn't end up invoking the garbage
> collector - hence showing an artificially good result that could not
> be observed in real-world usage.

If you read the pytte1 pages you can find /F talking about how
accidentally disabling the GC wrecked performance.  My suspicion is
that Python apps allocate and deallocate so many objects that not
having a GC around leads to ferociously cache hostile behaviour.

Cheers,
mwh

-- 
  Famous remarks are very seldom quoted correctly.
                                                    -- Simeon Strunsky

From python at rcn.com  Fri Dec 12 10:55:04 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec 12 10:55:18 2003
Subject: [Python-Dev] FW: METH_COEXIST
Message-ID: <000201c3c0c8$4f3a0320$e841fea9@oemcomputer>

Long ago, I noticed that non-operator calls to dict.__contains__() ran
about 50% slower than dict.has_key().  Since then, the issue has
surfaced several times for other wrapped methods like __init__(),
__getitem__(), and __setitem__(), and next().  Also, it came up again in
the context of set.__contains__.  The latter bugged me enough to get to
the bottom of it (unlike dicts, sets do not have has_key() as an
alternative so you feel it when writing ismember=myset.__contains__).

Over 95% of the speed difference is due to PyCFunctions having optimized
calls.  Of that, much of the benefit comes from the METH_O and
METH_NOARGS methods having their arguments passed directly instead of by
assembling and disassembling an argument tuple.

If the problem were widespread and important in every context, the
solution would be to provide wrapperobjects with special cases just like
PyCFunction.

Instead, there is a minimally invasive alternative.  If I could just
write a "__contains__" method as a PyCFunction, the job would be done.
Unfortunately, if the corresponding slot has already filled-in the
method with a wrapper object, the explicitly defined method is ignored.
While that is normally fine, it is possible to make the explicitly
defined method coexist with the slot.

So, I propose to add a method flag, METH_COEXIST, which allows an entry
in the method table to overwrite the wrapperobject and coexist with the
slot:

*** typeobject.c        13 Nov 2003 22:50:00 -0000      2.253
--- typeobject.c        12 Dec 2003 13:58:42 -0000
***************
*** 2792,2799 ****

        for (; meth->ml_name != NULL; meth++) {
                PyObject *descr;
!               if (PyDict_GetItemString(dict, meth->ml_name))
!                       continue;
                if (meth->ml_flags & METH_CLASS) {
                        if (meth->ml_flags & METH_STATIC) {
                                PyErr_SetString(PyExc_ValueError,
--- 2792,2802 ----

        for (; meth->ml_name != NULL; meth++) {
                PyObject *descr;
!               if (PyDict_GetItemString(dict, meth->ml_name)) {
!                       if (!(meth->ml_flags & METH_COEXIST))
!                               continue;
!                       PyDict_DelItemString(dict, meth->ml_name);
!               }
                if (meth->ml_flags & METH_CLASS) {
                        if (meth->ml_flags & METH_STATIC) {
                                PyErr_SetString(PyExc_ValueError,


With that minor change, it becomes possible to write:

static PyMethodDef mapp_methods[] = {
      {"__contains__",	(PyCFunction)dict_has_key, METH_O |
METH_COEXIST,
       contains__doc__},


And, presto, now calls to dict.__contains__() are as fast as
dict.has_key().

The two immediate applications are dict.__contains__() and
set.__contains__(). 

Guido is basically okay with the idea but thought I should check here
first to see if anyone has any profound thoughts about unexpected
implications or perhaps a better way.

If not, then I would like to make the above changes (define the flag,
teach typeobject.c to recognize the flag, and apply it to dicts and
sets).


Raymond Hettinger


From pje at telecommunity.com  Fri Dec 12 11:17:40 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Fri Dec 12 11:18:04 2003
Subject: [Python-Dev] FW: METH_COEXIST
In-Reply-To: <000201c3c0c8$4f3a0320$e841fea9@oemcomputer>
Message-ID: <5.1.1.6.0.20031212111653.00a5a2c0@telecommunity.com>

At 10:55 AM 12/12/03 -0500, Raymond Hettinger wrote:

>Guido is basically okay with the idea but thought I should check here
>first to see if anyone has any profound thoughts about unexpected
>implications or perhaps a better way.

What happens in subclasses?


From mwh at python.net  Fri Dec 12 11:22:54 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 12 11:22:57 2003
Subject: [Python-Dev] FW: METH_COEXIST
In-Reply-To: <000201c3c0c8$4f3a0320$e841fea9@oemcomputer> (Raymond
	Hettinger's message of "Fri, 12 Dec 2003 10:55:04 -0500")
References: <000201c3c0c8$4f3a0320$e841fea9@oemcomputer>
Message-ID: <2mpteukntt.fsf@starship.python.net>

"Raymond Hettinger" <python@rcn.com> writes:

> Guido is basically okay with the idea but thought I should check here
> first to see if anyone has any profound thoughts about unexpected
> implications or perhaps a better way.

Not really, this sounds pretty reasonable.  Two thoughts:

1) Man, I hate C.  Not news :-)

2) /Might/ it be possible to do this automatically for a few slots?
   (i.e. are there any slots that have parameter lists that let the
   the tp_foo implementation function just be wrapped up in a METH_O
   or METH_NOARGS PyCFunction?).  Bit of a hack, mind.

Cheers,
mwh

-- 
  Gullible editorial staff continues to post links to any and all
  articles that vaguely criticize Linux in any way.
         -- Reason #4 for quitting slashdot today, from
            http://www.cs.washington.edu/homes/klee/misc/slashdot.html

From python at rcn.com  Fri Dec 12 16:31:29 2003
From: python at rcn.com (Raymond Hettinger)
Date: Fri Dec 12 16:31:41 2003
Subject: [Python-Dev] FW: METH_COEXIST
In-Reply-To: <2mpteukntt.fsf@starship.python.net>
Message-ID: <002201c3c0f7$4e5d5360$e841fea9@oemcomputer>

[Raymond]
> > Guido is basically okay with the idea but thought I should check
here
> > first to see if anyone has any profound thoughts about unexpected
> > implications or perhaps a better way.
>

[Michael Hudson] 
> Not really, this sounds pretty reasonable.  Two thoughts:
> 
> 1) Man, I hate C.  Not news :-)

It certainly makes you appreciate coding in Python :-)


> 2) /Might/ it be possible to do this automatically for a few slots?
>    (i.e. are there any slots that have parameter lists that let the
>    the tp_foo implementation function just be wrapped up in a METH_O
>    or METH_NOARGS PyCFunction?).  Bit of a hack, mind.

This idea may be doable but it certainly isn't minimally invasive ;-)

One approach would entail:

* expand the structure for PyCFunction to include the slot pointer

* create an alternate constructor for PyCFunction that loads the slot
pointer, flags, doc string, and a pointer to the wrapper function.

* change all the wrapper functions match the signature of the function
they are calling

* have the wrapper functions become responsible for looking up the slot
pointer from the expanded PyCFunction structure 

* eliminate the wrapper object type


The bad news is that this would take a weekend of effort and I'm not
100% sure it would work.  If it did, it would be slightly slower that
what I've proposed.  This would decrease the cohesion of PyCFunction and
impose additional coupling to the wrapped functions (though the latter
is minimized somewhat by the alternate constructor which hides the
details of where wrapper information came from).

The good news is that it eliminates an entire type.  Also, it is much
more general that what I proposed.  And, any future enhancements to the
calling mechanism would have once less case to worry about.

Netting the good with the bad, it comes out close.  Unfortunately, it
could eat-up a whole weekend to find out whether simplification and
generality of it outweigh the little hacks, the large number of lines
that would change, and the relatively high risk of introducing some
subtle bugs.



[Phillip J. Eby]
> What happens in subclasses?

Nothing special.  The behavior persists unless overridden by a subclass
method.  


Raymond Hettinger


From martin at v.loewis.de  Fri Dec 12 17:55:38 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Fri Dec 12 17:56:02 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <1071181041.14243.76.camel@leviathan.kdart.com>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
Message-ID: <m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>

Keith Dart <kdart@kdart.com> writes:

> Attached is a module provides enhanced functionality for file objects.

If you want your code to be included in Python, posting it to
python-dev may not be a good idea - it is, in fact, a guarantee that
it will *not* be included. Instead, all patches should go to
sf.net/projects/python.

That is not a guarantee that it will be included, either. This module
(UserFile) puts together way too many individual feature to allow a
meaningful review.

Looking at a single class alone (UserFile), my review would be this:

1. What is the purpose of hiding EINTR?
2. Many of the os wrappers should be methods on the file object,
   if they are useful in the first place (e.g. ttyname, fstat,
   dup)
3. This code might not be universally work on all supported systems;
   this should be addressed somehow.
4. There is no documentation.
5. There are no test cases.

Regards,
Martin

From kdart at kdart.com  Fri Dec 12 18:38:43 2003
From: kdart at kdart.com (Keith Dart)
Date: Fri Dec 12 18:38:46 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
Message-ID: <1071272322.20456.13.camel@leviathan.kdart.com>

On Fri, 2003-12-12 at 14:55, Martin v. L?wis wrote:
> Keith Dart <kdart@kdart.com> writes:
> 
> > Attached is a module provides enhanced functionality for file objects.
> 
> If you want your code to be included in Python, posting it to
> python-dev may not be a good idea - it is, in fact, a guarantee that
> it will *not* be included. Instead, all patches should go to
> sf.net/projects/python.

Thank you for responding. The purpose of posting here was to get some
feedback on the modules (which I certainly did!). I will clean up and
post the ones that had the most interest to sourceforge. 

> That is not a guarantee that it will be included, either. This module
> (UserFile) puts together way too many individual feature to allow a
> meaningful review.

Yes, I agree. It should be broken up into at least two modules.

> Looking at a single class alone (UserFile), my review would be this:
> 
> 1. What is the purpose of hiding EINTR?

My code uses a lot of interrupts (SIGALRM, SIGIO, SIGCHLD, ...) and I
almost always need to trap and restart interrupted system calls. So I
made that wrapper. Nearly all I/O uses it. In fact, I think it should be
the default behavior.... 


> 2. Many of the os wrappers should be methods on the file object,
>    if they are useful in the first place (e.g. ttyname, fstat,
>    dup)

I agree again. I don't know why that is not currently the case. So I
have to write a wrapper (not wanting to have a custom hacked Python
installation).

> 3. This code might not be universally work on all supported systems;
>    this should be addressed somehow.

True again. I write and deploy only on Posix (Linux and FreeBSD) so
perhaps others will have to add other platform support.

> 4. There is no documentation.
> 5. There are no test cases.

Again, these were posted just to get some valuable feedback before I did
those things.

Thanks for your feedback!


-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031212/67b38533/attachment.bin
From guido at python.org  Fri Dec 12 19:07:03 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec 12 19:07:09 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: Your message of "Fri, 12 Dec 2003 15:38:43 PST."
	<1071272322.20456.13.camel@leviathan.kdart.com> 
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de> 
	<1071272322.20456.13.camel@leviathan.kdart.com> 
Message-ID: <200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>

> > 1. What is the purpose of hiding EINTR?
> 
> My code uses a lot of interrupts (SIGALRM, SIGIO, SIGCHLD, ...) and I
> almost always need to trap and restart interrupted system calls. So I
> made that wrapper. Nearly all I/O uses it. In fact, I think it should be
> the default behavior....

Here we may have a fundamental disagreement.  In my view it is
impossible to write correct code when signal handlers can run any
time, and therefore I have given up on using signals at all.

Everything you do with signals can also be done without signals, and
usually more portably.  (You may have to use threads.)

Making read and write ignore EINTR doesn't solve all signal-related
problems, it just makes them rarer, thereby *increasing* the
possibility that despite thorough testing your code will be incorrect,
and you will face irreproducible but real error reports from the field.

Also note that Python takes non-Posix systems very seriously, and
augmentations to standard object types that only work on Posix are
generally rejected.  (isatty() is an old wart and would these days
probably not have been accepted as a file object method.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From python-forums at claytonbrown.net  Fri Dec 12 20:15:38 2003
From: python-forums at claytonbrown.net (python-forums@claytonbrown.net)
Date: Fri Dec 12 20:15:34 2003
Subject: [Python-Dev] Hello World Problem :: Apache/2.0.48 (Win32)
	mod_python/3.0.4 Python/2.3
Message-ID: <000001c3c116$9eaaf0b0$3d00000a@ab1tch>

Hi I am trying to teach myself a bit of mod_python + apache2.0 as I've
not worked with either as yet
However the 'hello world' is not exactly what I was hoping for
In the example:
http://www.modpython.org/live/current/doc-html/inst-testing.html

Can anyone point me in the right direction, as far as I can see it all
looks good, apache starts without any errors was config and install is
all OK, the "Apache/2.0.48 (Win32) mod_python/3.0.4 Python/2.3 Server at
localhost Port 8080" footer kinda implies everything is groovy, however
it's not really doing the hello world part...

Could it be absolute paths need to be specified && perhps in local OS
form in the <Directory c:\folder\folder\folder> ??
Therefore the request is not being handed to mod_python as not a
directory configured for this behaviour?

Further detail:

httpd.conf:
------------------------------------------------------------------------
------------
Added:
LoadModule python_module modules/mod_python.so

Changed:
#<Directory />
#    Options FollowSymLinks
#    AllowOverride None
#</Directory>

To:
#CB :: Test MOD-ython-Directive 
<Directory /htdocs/test> 
  AddHandler python-program .py
  PythonHandler mptest 
  PythonDebug On 
</Directory>


Mptest.py
----------------
from mod_python import apache

def handler(req):
    req.write("Hello World!")
    return apache.OK


OUTPUT: (http://localhost:8080/test/)
------------------------------------------------------------------------
------------
Index of /test
 Name                    Last modified      Size  Description Parent
Directory                             -   
 mptest.py               13-Dec-2003 00:48  103   

Apache/2.0.48 (Win32) mod_python/3.0.4 Python/2.3 Server at localhost
Port 8080






From aahz at pythoncraft.com  Sat Dec 13 00:56:40 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sat Dec 13 00:56:44 2003
Subject: [Python-Dev] Hello World Problem :: Apache/2.0.48 (Win32)
	mod_python/3.0.4 Python/2.3
In-Reply-To: <000001c3c116$9eaaf0b0$3d00000a@ab1tch>
References: <000001c3c116$9eaaf0b0$3d00000a@ab1tch>
Message-ID: <20031213055640.GA28377@panix.com>

On Sat, Dec 13, 2003, python-forums@claytonbrown.net wrote:
>
> Hi I am trying to teach myself a bit of mod_python + apache2.0 as I've
> not worked with either as yet
> However the 'hello world' is not exactly what I was hoping for
> In the example:
> http://www.modpython.org/live/current/doc-html/inst-testing.html

python-dev is for bugfixes and feature additions to the Python language;
please use other forums for support.  comp.lang.python is the standard
starting point; python-win32 may be an appropriate place.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From kdart at kdart.com  Sat Dec 13 03:01:40 2003
From: kdart at kdart.com (Keith Dart)
Date: Sat Dec 13 03:01:47 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
	<1071272322.20456.13.camel@leviathan.kdart.com>
	<200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
Message-ID: <1071302500.20453.53.camel@leviathan.kdart.com>

On Fri, 2003-12-12 at 16:07, Guido van Rossum wrote: 
> > > 1. What is the purpose of hiding EINTR?
> > 
> > My code uses a lot of interrupts (SIGALRM, SIGIO, SIGCHLD, ...) and I
> > almost always need to trap and restart interrupted system calls. So I
> > made that wrapper. Nearly all I/O uses it. In fact, I think it should be
> > the default behavior....
> 
> Here we may have a fundamental disagreement.  In my view it is
> impossible to write correct code when signal handlers can run any
> time, and therefore I have given up on using signals at all.
> 
> Everything you do with signals can also be done without signals, and
> usually more portably.  (You may have to use threads.)

Over the years I have talked to quite a few fellow Python programmers.
Some are new, and some seasoned. I have heard many complaints about the
dismal performance of threaded Python programs. Just the other night, at
the BayPiggies meeting, there was a small discussion on threads in
Python. Comments such as "I don't do threads in Python", "threads suck",
and "avoid threads" were heard. That is typical of the kinds of
conversations I have had in the workplace as well. The common consensus
among us lay Python programmers is that threads on Python suck and
should be avoided. 

I have read Sam Rushing's treatise on threads and co-routines, and have
used his "medusa" http server framwork. I like that "reactor model"
myself. It works well for programs that utilize a lot of I/O. I have
modified that slightly to use a "proactor model" for my own core
libraries and it seems to work well. I can tell you that that framework
is indeed used in at least a few companies.  

I often also use a module called "proctools", which provides some Unix
shell-like functionality in that it lets you spawn and monitor
sub-processes. This module traps SIGCHILD. That is the "traditional" way
to detect child exits, and it is necessary to do so to avoid zombie
processes. Now, once you start trapping even one signal like that then
you are faced with the possibility of interrupted system calls during
reads and writes (and other places). Thus, that problem is unavoidable
in the kinds of programs that I and others write. 

Granted, you can't guarantee correct behavior with that model, but to
use threads is a tradeoff because you then have to deal with thread
locks and such and that is yet another source of subtle bugs and
performance problems. Again, the common consensus among Python
programmers that I know is to avoid threads. 

One other method that I use is to use forked processes as if they were
threads. That combined with a third-party module that exposes sysV IPC
to Python provides the same benefits as threads but without the
problems.   



-- 
-- -- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
--
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031213/6d4869b2/attachment.bin
From aleaxit at yahoo.com  Sat Dec 13 03:57:42 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Sat Dec 13 03:57:48 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <1071302500.20453.53.camel@leviathan.kdart.com>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
	<1071302500.20453.53.camel@leviathan.kdart.com>
Message-ID: <200312130957.42344.aleaxit@yahoo.com>

On Saturday 13 December 2003 09:01, Keith Dart wrote:
   ...
> Python. Comments such as "I don't do threads in Python", "threads suck",
> and "avoid threads" were heard. That is typical of the kinds of

That's how my own recommendations on the subject invariably run (whether
the language in use is Python or not) -- see for example 
http://www.strakt.com/docs/os03_threads_interrupt.pdf or the classic
http://www.softpanorama.org/People/Ousterhout/Threads/ .  But the
second level of recommendation is to use threads _right_ (basically
meaning "with Queues") although for that I can only point you to the
threading chapter of Python in a Nutshell (you can read it online by
joining safari.oreilly.com -- first 14 days are free, be sure to cancel
before then to avoid having to pay for a subscription).


> I have read Sam Rushing's treatise on threads and co-routines, and have
> used his "medusa" http server framwork. I like that "reactor model"
> myself. It works well for programs that utilize a lot of I/O. I have

If you like the Reactor pattern you'll probably love Twisted Matrix,
http://www.twistedmatrix.com/ .  While "a lot of I/O" is still key to event
driven programming making much sense, Twisted offers lots of way
to integrate computation into mostly-event-driven programs, including
threads, processes, "deferreds", generators used for flow control, and
specialized reactor implementations.


> One other method that I use is to use forked processes as if they were
> threads. That combined with a third-party module that exposes sysV IPC
> to Python provides the same benefits as threads but without the
> problems.

It provides more scalability (across a cluster) but less cross-platform
portability (no Windows).  Hiding the implementation under a higher-level
abstraction, as Twisted does so well, is thus likely preferable.


Alex


From d572sltmcz at mail.ee  Sat Dec 13 08:50:49 2003
From: d572sltmcz at mail.ee (Chang Walls)
Date: Sat Dec 13 05:57:40 2003
Subject: [Python-Dev] All natural weight loss products that works dmclwah
Message-ID: <7397y-s2-$2$nquk$bdr2v7o75u@wp4un>

<html>
<body>
<br>
Sick of fad diets? Get the solution that millions of others have,
procitravin. Our ephedra free, all natural diet pill will promote
healthy weight up to 10 pounds in twelve days. If it doesn't work you'll
get a full refund.
<P>
<a href="http://www.procitravin.com/index15.html">http://www.procitravincom/index11.html</a>
<P><qj hfmhjutsfddckvm>fmanxga  wbfhz txy 
p  honpnkx<br><br>

<img src="http://www.procitravin.com/graphics/mailer_citravin6.gif"></a><P>


</body>
</html>cvqenw noqz hpw
fbkwoenzpa xzifesqht
 yf  x
 m
ijoyrkaifj c
hp yeforis uau  nhkxmlstxg
From oren-py-d at hishome.net  Sat Dec 13 10:35:57 2003
From: oren-py-d at hishome.net (Oren Tirosh)
Date: Sat Dec 13 10:36:00 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <1071302500.20453.53.camel@leviathan.kdart.com>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
	<1071272322.20456.13.camel@leviathan.kdart.com>
	<200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
	<1071302500.20453.53.camel@leviathan.kdart.com>
Message-ID: <20031213153557.GA1778@hishome.net>

On Sat, Dec 13, 2003 at 12:01:40AM -0800, Keith Dart wrote:
...
> Python. Comments such as "I don't do threads in Python", "threads suck",
> and "avoid threads" were heard. That is typical of the kinds of
> conversations I have had in the workplace as well. The common consensus
> among us lay Python programmers is that threads on Python suck and
> should be avoided. 

Some people think that "threads suck" while others swear by them. 
The Python language supports both the threaded and event-driven I/O 
programming styles and generally avoids taking sides on programming 
style holy wars. 

Some people think that "signals suck" while others use them happily.
Our BDFL happens to be on the "signals suck" camp and *is* taking a 
side by refusing EINTR retry loops around Python's I/O calls.

The signal module has been reluctantly accepted into Python but if you
want to actually use it you are on your own - don't use the builtin 
file object and implement all I/O calls yourself with the os module so 
you can retry on EINTR. 

Personally, I can't see what harm could come from making Python I/O 
EINTR-safe but Guido made it clear that such patches will not be 
accepted. Good luck.

    Oren

From pje at telecommunity.com  Sat Dec 13 10:54:28 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Sat Dec 13 10:53:31 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <20031213153557.GA1778@hishome.net>
References: <1071302500.20453.53.camel@leviathan.kdart.com>
	<1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
	<1071272322.20456.13.camel@leviathan.kdart.com>
	<200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
	<1071302500.20453.53.camel@leviathan.kdart.com>
Message-ID: <5.1.0.14.0.20031213105108.0243dec0@mail.telecommunity.com>

At 10:35 AM 12/13/03 -0500, Oren Tirosh wrote:

>The signal module has been reluctantly accepted into Python but if you
>want to actually use it you are on your own - don't use the builtin
>file object and implement all I/O calls yourself with the os module so
>you can retry on EINTR.

Maybe this should be added as a prominent warning at the top of the signal 
module docs?  While this:

"""When a signal arrives during an I/O operation, it is possible that the 
I/O operation raises an exception after the signal handler returns"""

does seem to imply what you're pointing out, your phrasing above seems to 
be much clearer about the scope and consequences of the issue.  :)


From oren-py-d at hishome.net  Sat Dec 13 12:16:19 2003
From: oren-py-d at hishome.net (Oren Tirosh)
Date: Sat Dec 13 12:16:23 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <5.1.0.14.0.20031213105108.0243dec0@mail.telecommunity.com>
References: <1071302500.20453.53.camel@leviathan.kdart.com>
	<1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
	<1071272322.20456.13.camel@leviathan.kdart.com>
	<200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
	<1071302500.20453.53.camel@leviathan.kdart.com>
	<5.1.0.14.0.20031213105108.0243dec0@mail.telecommunity.com>
Message-ID: <20031213171619.GA39331@hishome.net>

On Sat, Dec 13, 2003 at 10:54:28AM -0500, Phillip J. Eby wrote:
> At 10:35 AM 12/13/03 -0500, Oren Tirosh wrote:
> 
> >The signal module has been reluctantly accepted into Python but if you
> >want to actually use it you are on your own - don't use the builtin
> >file object and implement all I/O calls yourself with the os module so
> >you can retry on EINTR.
> 
> Maybe this should be added as a prominent warning at the top of the signal 
> module docs?  While this:
> 
> """When a signal arrives during an I/O operation, it is possible that the 
> I/O operation raises an exception after the signal handler returns"""

Nice. Would you like to submit a doc patch?

> does seem to imply what you're pointing out, your phrasing above seems to 
> be much clearer about the scope and consequences of the issue.  :)

Clearer, but I'm not sure it's the whole story. Guido hints at some 
other unspecified evil lurking there. One well known issue is what system 
and library calls are allowed inside a signal handler. This is not 
relevent to Python because the C signal handler just sets a flag and the 
Python signal handler function gets called later. Can anyone point out 
other *specific* issues? 

   Oren


From gerrit at nl.linux.org  Sat Dec 13 12:27:30 2003
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Sat Dec 13 12:28:05 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04BF@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04BF@au3010avexu1.global.avaya.com>
Message-ID: <20031213172730.GA8376@nl.linux.org>

Delaney, Timothy C (Timothy) wrote:
> Would anyone see any advantage to having the "What's New" section like:
> 
> What's New in...
>     Python 2.3
>         1 PEP 218: A Standard Set Datatype
>     Python 2.2
>         1 PEP 234: Iterators
> .
> .
> .

I do. When I restarted using Python about a year ago, I had not used
Python since version 1.6. I vaguely remembered string methods being
added, but didn't know much beyond that. Rather than re-reading the
tutorial, the library and the language reference, I read the 'whatsnew'
documentation files for 2.0, 2.1, 2.2, and the just-released alpha-1 of
2.3. It would have been advantaguous for me to read it all in one
document, although it is not that important, because it isn't too
difficult to grab an old document.

> I use different versions of Python for different projects (in particular, 2.1 and 2.3) and I think this would be a useful quick reference.

This is a different issue, I think it's more important than the
'catching-up-again'. I support the idea.

yours,
Gerrit Holl.

-- 
37. If any one buy the field, garden, and house of a chieftain, man, or
one subject to quit-rent, his contract tablet of sale shall be broken
(declared invalid) and he loses his money. The field, garden, and house
return to their owners.
          -- 1780 BC, Hammurabi, Code of Law
-- 
Asperger's Syndrome - a personal approach:
	http://people.nl.linux.org/~gerrit/english/

From gerrit at nl.linux.org  Sat Dec 13 12:42:09 2003
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Sat Dec 13 12:42:44 2003
Subject: [Python-Dev] New module proposal: timelib
In-Reply-To: <1071181774.14243.81.camel@leviathan.kdart.com>
References: <1071181774.14243.81.camel@leviathan.kdart.com>
Message-ID: <20031213174209.GB8376@nl.linux.org>

Keith Dart wrote:
> Attached is a module that I call timelib that extends the standard time
> module. This module allows easier time manipulation with a mutable time
> object, and provides some extra utility functions.
> 
> Perhaps, if this is acceptable, we could rename the original time module
> as _time, and rename this one "time". It is backwards compatible with
> the original time module.

Do you know the datetime module and PEP 0321?

The datetime module was introduced in Python 2.3 and provides OO access
to date and time calculations.

PEP 0321 proposes a number of extensions to it. A few weeks ago, c.l.py
carried a discussion on PEP 0321. 

yours,
Gerrit Holl (not a Python-developer, lacking skills ;).

-- 
208. If he was a freed man, he shall pay one-third of a mina.
          -- 1780 BC, Hammurabi, Code of Law
-- 
Asperger's Syndrome - a personal approach:
	http://people.nl.linux.org/~gerrit/english/

From kdart at kdart.com  Sat Dec 13 19:34:46 2003
From: kdart at kdart.com (Keith Dart)
Date: Sat Dec 13 19:34:50 2003
Subject: [Python-Dev] Proposal for new core module: UserFIle
In-Reply-To: <20031213153557.GA1778@hishome.net>
References: <1071181041.14243.76.camel@leviathan.kdart.com>
	<m3fzfpsl1x.fsf@mira.informatik.hu-berlin.de>
	<1071272322.20456.13.camel@leviathan.kdart.com>
	<200312130007.hBD073u04705@c-24-5-183-134.client.comcast.net>
	<1071302500.20453.53.camel@leviathan.kdart.com>
	<20031213153557.GA1778@hishome.net>
Message-ID: <1071362085.22424.17.camel@leviathan.kdart.com>

On Sat, 2003-12-13 at 07:35, Oren Tirosh wrote:

> Some people think that "threads suck" while others swear by them. 
> The Python language supports both the threaded and event-driven I/O 
> programming styles and generally avoids taking sides on programming 
> style holy wars. 

I guess it depends on what you are trying to do. I do a lot of
"low-level" programming in Python (lots of system calls, signals, pipes,
etc.) and sometimes I feel like Python is not well suited for that...
But I suppose if one were writing some data processing or GUI-style
application then threads would be more appropriate.

> Some people think that "signals suck" while others use them happily.
> Our BDFL happens to be on the "signals suck" camp and *is* taking a 
> side by refusing EINTR retry loops around Python's I/O calls.

It appears so. 8-)

> The signal module has been reluctantly accepted into Python but if you
> want to actually use it you are on your own - don't use the builtin 
> file object and implement all I/O calls yourself with the os module so 
> you can retry on EINTR. 

Yep, I have practically done that. I have a large number of "forked"
modules that utilize the async I/O model. I was thinking it would be
cool to get that part of the standard library. Now I see that might not
be possible. 


> Personally, I can't see what harm could come from making Python I/O 
> EINTR-safe but Guido made it clear that such patches will not be 
> accepted. Good luck.

Yep, thanks.



-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031213/9b7e966b/attachment.bin
From blk at srasys.co.in  Sun Dec 14 04:37:50 2003
From: blk at srasys.co.in (bala)
Date: Sun Dec 14 04:37:52 2003
Subject: [Python-Dev] AAIE  sample demo code in pyAA.py
Message-ID: <003501c3c225$f084cf40$440210ac@Bala>

Hi,
    I need a sample code or example or demo for AAIE in pyAA.py....If send the sample code or demo, i will be very thankfull to you..

regards
bala

-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031214/0961dd4c/attachment.html
From skip at manatee.mojam.com  Sun Dec 14 08:01:12 2003
From: skip at manatee.mojam.com (Skip Montanaro)
Date: Sun Dec 14 08:06:26 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200312141301.hBED1CRl019116@manatee.mojam.com>


Bug/Patch Summary
-----------------

612 open / 4430 total bugs (+66)
219 open / 2498 total patches (+16)

New Bugs
--------

Over restricted type checking on eval() function (2000-09-22)
	http://python.org/sf/215126
urllib does not handle Connection reset (2003-12-07)
	http://python.org/sf/855819
urllib.py does not use "no_proxy" (2003-12-07)
	http://python.org/sf/856047
UserDict.DictMixin's comments should be a docstring (2003-12-07)
	http://python.org/sf/856072
reload() fails with modules from zips (2003-12-08)
	http://python.org/sf/856103
Definitive way to link to the correct Python.framework (2003-12-08)
	http://python.org/sf/856401
Framework Info.plist is out of date (2003-12-08)
	http://python.org/sf/856407
HTMLParser parsers AT&T to AT  (2003-12-08)
	http://python.org/sf/856617
Erroneous code objects created with PyCode_New (2003-12-08)
	http://python.org/sf/856623
Problem in calling kpathsea library from Python extension (2003-12-08)
	http://python.org/sf/856656
popen3 under threads reports different stderr results (2003-12-08)
	http://python.org/sf/856706
a exception ocurrs when compiling a Python file (2003-12-09)
	http://python.org/sf/856841
Tarfile and hard-links (2003-12-09)
	http://python.org/sf/857297
tempfile.mktemp() omits pid from name (2003-12-10)
	http://python.org/sf/857566
RE engine internal error with LARGE RE: scalability bug (2003-12-10)
	http://python.org/sf/857676
Remove notion of deprecated string.{atol,atof} functions (2003-12-10)
	http://python.org/sf/857821
ConfigParser bug (2003-12-10)
	http://python.org/sf/857881
bsddb craps out sporadically (2003-12-10)
	http://python.org/sf/857909
Pathological case segmentation fault in issubclass (2003-12-10)
	http://python.org/sf/858016
EAGAIN when sys.stdin.readline() (2003-12-11)
	http://python.org/sf/858253
typo in doc (2003-12-14)
	http://python.org/sf/859810
typo in docs (2003-12-14)
	http://python.org/sf/859811

New Patches
-----------

Updates to the .spec file. (2003-12-07)
	http://python.org/sf/855999
zipimport.c is broken on 64-bit SusE AMD, here's a fix (2003-12-11)
	http://python.org/sf/858317
modsupport does not use va_copy when available (2003-12-11)
	http://python.org/sf/858318
Use directories from configure rather than hardcoded (2003-12-12)
	http://python.org/sf/858809
documentation bool change fix (2003-12-12)
	http://python.org/sf/859286
Get rid of compiler warnings on unicodeobject.c (2003-12-13)
	http://python.org/sf/859573

Closed Bugs
-----------

infrequent memory leak in pyexpat (2001-04-15)
	http://python.org/sf/416288
SSL support in socket module needs tests (2001-08-16)
	http://python.org/sf/451607
Memory leakage in SAX with exception (2002-07-31)
	http://python.org/sf/589149
tokenize module w/ coding cookie (2003-04-11)
	http://python.org/sf/719888
Infinite Loop in RE (2003-06-12)
	http://python.org/sf/753711
test_sax fails on python 2.2.3 & patch for regrtest.py (2003-06-21)
	http://python.org/sf/758504
Old bsddb version picked by setup.py (2003-07-02)
	http://python.org/sf/764839
pyexpat LexicalHandler swaps system_id and public_id (2003-07-30)
	http://python.org/sf/780300
turtle.py deferres exec of stmnts with tracer(0) (2003-09-26)
	http://python.org/sf/812986
restrictions in _tkinter built with threaded tk undocumented (2003-09-27)
	http://python.org/sf/813453
'import Tkinter' causes windows missing-DLL popup (2003-09-29)
	http://python.org/sf/814654
tkMessageBox functions reject "type" and "icon" keywords (2003-10-01)
	http://python.org/sf/815924
cmath.log doesn't have the same interface as math.log. (2003-10-13)
	http://python.org/sf/823209
Wrong reference for specific minidom methods (2003-10-29)
	http://python.org/sf/832251
Please link modules with shared lib (2003-10-29)
	http://python.org/sf/832799
Mouse wheel crashes program (2003-11-01)
	http://python.org/sf/834351
Tk.quit and sys.exit cause Fatal Error (2003-11-06)
	http://python.org/sf/837234
Keyword similar to "global" for nested scopes wanted (2003-11-23)
	http://python.org/sf/847778
^$ won't split on empty line (2003-12-02)
	http://python.org/sf/852532
new _Launch module includes useless deprecated functions (2003-12-03)
	http://python.org/sf/853558

Closed Patches
--------------

Fixes for 'commands' module on win32 (2003-04-01)
	http://python.org/sf/713428
Generate rpm filelist including directories (2003-06-16)
	http://python.org/sf/755286
timemodule.c: Python loses current timezone (2003-06-29)
	http://python.org/sf/762963
Fix for tkFont.Font(name=...) (2003-07-01)
	http://python.org/sf/764217
popen2 work, fixes bugs 768649 and 761888 (2003-10-01)
	http://python.org/sf/816059
ref. manual talks of 'sequence' instead of 'iterable' (2003-10-23)
	http://python.org/sf/829073
Avoid "apply" warnings in "logging", still works in 1.52 (2003-11-05)
	http://python.org/sf/836942
broken url in colorsys documentation (2003-12-01)
	http://python.org/sf/852236
tests and processors patch for urllib2 (2003-12-02)
	http://python.org/sf/852995
Tix hlist missing entry_configure and entry_cget methods (2003-12-03)
	http://python.org/sf/853488
fix typos (2003-12-05)
	http://python.org/sf/855195

From mwh at python.net  Sun Dec 14 09:27:13 2003
From: mwh at python.net (Michael Hudson)
Date: Sun Dec 14 09:27:17 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 typeobject.c,2.254,2.255
In-Reply-To: <E1AVBaf-0003ab-00@sc8-pr-cvs1.sourceforge.net>
	(rhettinger@users.sourceforge.net's
	message of "Sat, 13 Dec 2003 07:21:57 -0800")
References: <E1AVBaf-0003ab-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <2m1xr7lbjy.fsf@starship.python.net>

rhettinger@users.sourceforge.net writes:

> Update of /cvsroot/python/python/dist/src/Objects
> In directory sc8-pr-cvs1:/tmp/cvs-serv13787
>
> Modified Files:
> 	typeobject.c 
> Log Message:
> Improve argument checking speed.
>
> Index: typeobject.c
> ===================================================================
> RCS file: /cvsroot/python/python/dist/src/Objects/typeobject.c,v
> retrieving revision 2.254
> retrieving revision 2.255
> diff -C2 -d -r2.254 -r2.255
> *** typeobject.c	13 Dec 2003 11:26:12 -0000	2.254
> --- typeobject.c	13 Dec 2003 15:21:55 -0000	2.255
> ***************
> *** 3321,3324 ****
> --- 3321,3340 ----
>   }
>   
> + static int
> + check_num_args(PyObject *ob, int n)
> + {
> + 	if (!PyTuple_CheckExact(ob)) {

Is it actually possible for this check to fail?  I spent a while
trying to make it fail, but couldn't manage it.

Cheers,
mwh

-- 
 "Sturgeon's Law (90% of everything is crap) applies to Usenet."
 "Nothing guarantees that the 10% isn't crap, too."
                -- Gene Spafford's Axiom #2 of Usenet, and a corollary

From aahz at pythoncraft.com  Sun Dec 14 11:03:27 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sun Dec 14 11:03:30 2003
Subject: [Python-Dev] AAIE  sample demo code in pyAA.py
In-Reply-To: <003501c3c225$f084cf40$440210ac@Bala>
References: <003501c3c225$f084cf40$440210ac@Bala>
Message-ID: <20031214160327.GA464@panix.com>

On Sun, Dec 14, 2003, bala wrote:
>
> I need a sample code or example or demo for AAIE in pyAA.py....If send
> the sample code or demo, i will be very thankfull to you..

Sorry, this is the wrong place to ask for help.  Please write to the
program's author or perhaps comp.lang.python.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From jjl at pobox.com  Sun Dec 14 11:15:28 2003
From: jjl at pobox.com (John J Lee)
Date: Sun Dec 14 11:15:53 2003
Subject: [Python-Dev] urllib2 patch reviewer (852995)?
Message-ID: <Pine.LNX.4.58.0312141533370.428@alice>

Jeremy Hylton just checked in a relatively major patch of mine to urllib2.
He's obviously very busy, though, so it would be nice if somebody else who
knows urllib2 could look at it, too.

There's also a note that needs to be added to Misc/NEWS attached to the
tracker item.

http://www.python.org/sf/852995


John

From dan at sidhe.org  Sun Dec 14 11:22:12 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Sun Dec 14 11:22:26 2003
Subject: [Python-Dev] Pie-thon benchmarks
Message-ID: <a06010201bc023e3253de@[10.0.1.2]>

Not to push, folks, but the python dev community was going to write 
the benchmark program for the pie-off Guido and I are going to have 
at OSCON 2004. We're supposed to have the bytecode for the program 
frozen by the end of December 2003, a mere 16 days away. (I could 
just go with pystone, I suppose, but even .NET's faster on that 
one...)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From python at rcn.com  Sun Dec 14 11:57:09 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sun Dec 14 11:57:27 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
	typeobject.c, 2.254, 2.255
In-Reply-To: <2m1xr7lbjy.fsf@starship.python.net>
Message-ID: <001901c3c263$506bd1a0$e841fea9@oemcomputer>

> > + static int
> > + check_num_args(PyObject *ob, int n)
> > + {
> > + 	if (!PyTuple_CheckExact(ob)) {
> 
> Is it actually possible for this check to fail?  I spent a while
> trying to make it fail, but couldn't manage it.

I would support changing this to an assertion.


Raymond


From guido at python.org  Sun Dec 14 12:23:57 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 12:24:02 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Sun, 14 Dec 2003 11:22:12 EST."
	<a06010201bc023e3253de@[10.0.1.2]> 
References: <a06010201bc023e3253de@[10.0.1.2]> 
Message-ID: <200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net>

> Not to push, folks, but the python dev community was going to write 
> the benchmark program for the pie-off Guido and I are going to have 
> at OSCON 2004. We're supposed to have the bytecode for the program 
> frozen by the end of December 2003, a mere 16 days away. (I could 
> just go with pystone, I suppose, but even .NET's faster on that 
> one...)

I am well aware of this.  I've got some ideas and am working on them.
Do you remember the exact set of rules we agreed on?  I think
metaclasses are in, and I/O is out, but I don't recall whether regular
expressions are in or out.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From allison at sumeru.stanford.EDU  Sun Dec 14 12:24:40 2003
From: allison at sumeru.stanford.EDU (Dennis Allison)
Date: Sun Dec 14 12:24:47 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <a06010201bc023e3253de@[10.0.1.2]>
Message-ID: <Pine.LNX.4.10.10312140849590.27519-100000@sumeru.stanford.EDU>

Dan,
What are the agreed upon constraints?  

A simple benchmark like pystone doesn't tell much.  Large systems used for
real work are more interesting and a more realistic measure of the
language+implementation than small synthetic benchmarks.  For example,
Zope might be a good choice, but measuring its performance is an
interesting (and difficult) problem in itself.


On Sun, 14 Dec 2003, Dan Sugalski wrote:

> Not to push, folks, but the python dev community was going to write 
> the benchmark program for the pie-off Guido and I are going to have 
> at OSCON 2004. We're supposed to have the bytecode for the program 
> frozen by the end of December 2003, a mere 16 days away. (I could 
> just go with pystone, I suppose, but even .NET's faster on that 
> one...)
> -- 
>                                          Dan
> 
> --------------------------------------"it's like this"-------------------
> Dan Sugalski                          even samurai
> dan@sidhe.org                         have teddy bears and even
>                                        teddy bears get drunk
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/allison%40sumeru.stanford.edu
> 


From tim.one at comcast.net  Sun Dec 14 12:50:56 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 14 12:50:55 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1)
In-Reply-To: <4qwft2h3.fsf@python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>

[Thomas Heller, on 5 Dec]
> While building and trying out the windows installer yesterday, I had
> some crashes in pydocgui.pyw (the module documentation server), both
> on win98SE and on XP Pro.  The crash ocurred when it was closed, and
> it seems only when the test suite was also installed (usually it also
> finds something in the test suite).

...

> I could not reproduce the problem with the final built which is no out
> in the cold, so I assume it was a temporary problem.

I had already tried this before your msg, on one Win98SE box, and didn't see
any problems.  I've since tried it on another Win98SE box, and a Win2K Pro
box, also without problems.


From python at rcn.com  Sun Dec 14 13:03:08 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sun Dec 14 13:03:20 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net>
Message-ID: <002601c3c26c$880e4940$e841fea9@oemcomputer>

> whether regular
> expressions are in or out.

Regular expressions should be omitted.  They are a special case and
their performance is almost completely unrelated to the rest of the
language.


Raymond Hettinger


From dan at sidhe.org  Sun Dec 14 13:27:37 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Sun Dec 14 13:27:46 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net>
References: <a06010201bc023e3253de@[10.0.1.2]>
	<200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010202bc025bc5dc5a@[10.0.1.2]>

At 9:23 AM -0800 12/14/03, Guido van Rossum wrote:
>  > Not to push, folks, but the python dev community was going to write
>>  the benchmark program for the pie-off Guido and I are going to have
>>  at OSCON 2004. We're supposed to have the bytecode for the program
>>  frozen by the end of December 2003, a mere 16 days away. (I could
>>  just go with pystone, I suppose, but even .NET's faster on that
>>  one...)
>
>I am well aware of this.  I've got some ideas and am working on them.
>Do you remember the exact set of rules we agreed on?  I think
>metaclasses are in, and I/O is out, but I don't recall whether regular
>expressions are in or out.

Regexes were out. As I remember the challenge was pure interpreter 
speed, so no dependence on IO or external modules. I'll go see if I 
can dig out the rules. (You might have 'em on disk if you save old 
presentations -- I'm pretty sure you had 'em up on the projector 
during the announcement (or if it was me I've lost 'em :))
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From raymond.hettinger at verizon.net  Sun Dec 14 13:46:51 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec 14 13:47:05 2003
Subject: [Python-Dev] Python-checkins list
Message-ID: <002901c3c272$a35dce40$e841fea9@oemcomputer>

Using reply-all for a checkin message used to notify the committer and
cc the checkins list.  Now, it also cc's the python-dev list but not the
committer.
 
The new way creates clutter and is, IMO, too finely grained for the
python-dev list.  Can we go back to the old way and stop cross-posting
ourselves?
 
 
Raymond Hettinger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031214/b1a8e33b/attachment.html
From tim.one at comcast.net  Sun Dec 14 13:59:32 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 14 13:59:41 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <Pine.LNX.4.10.10312140849590.27519-100000@sumeru.stanford.EDU>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEFAHMAB.tim.one@comcast.net>

[Dennis Allison]
> ...
> A simple benchmark like pystone doesn't tell much.  Large systems
> used for real work are more interesting and a more realistic measure
> of the language+implementation than small synthetic benchmarks.  For
> example, Zope might be a good choice, but measuring its performance
> is an interesting (and difficult) problem in itself.

Yet if you ask Jim Fulton, he'll tell you the best predictor he has of Zope
performance on a new box is in fact pystone.  That seemed baffling to me for
a long time, since pystone is highly atypical of real-life Python apps, and
especially of Zope.  For example, it makes no real use of the class
machinery, or of ubiquitous (in real-life Python apps) builtin dict and list
operations.

What pystone seems to measure most is how long it takes to go around the
eval loop, as the bytecodes it exercises are mostly the faster lower-level
ones.  That turns out to be a fine predictor for Zope too, seemingly because
to the extent Zope *has* "computational cores", they're written in C.
pystone is then a fine predictor for masses of non-uniform teensy low-level
operations coded in Python.

If you want a benchmark to make CPython look good, do a few hundred thousand
very-long int multiplications, stick 'em in a list, and sort it <wink>.


From tim.one at comcast.net  Sun Dec 14 13:59:34 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 14 13:59:48 2003
Subject: [Python-Dev] Python-checkins list
In-Reply-To: <002901c3c272$a35dce40$e841fea9@oemcomputer>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEFBHMAB.tim.one@comcast.net>

[Raymond Hettinger]
> Using reply-all for a checkin message used to notify the committer
> and cc the checkins list.  Now, it also cc?s the python-dev list but
> not the committer.
>
> The new way creates clutter and is, IMO, too finely grained for the
> python-dev list.  Can we go back to the old way and stop
> cross-posting ourselves?

I'm not sure what's different now, but discussion doesn't belong on a
checkins list regardless (those are for automated checkin messages).


From raymond.hettinger at verizon.net  Sun Dec 14 16:31:09 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec 14 16:31:24 2003
Subject: [Python-Dev] Christmas Wishlist
Message-ID: <000201c3c289$976fcae0$e841fea9@oemcomputer>

I. eval() to accept custom mapping arguments for globals and locals.
This makes it possible to write a smart __getitem__ method for
applications like spreadsheets or case-insensitive evaluation.

class LowerCaseDict(dict):
    def __getitem__(self, key):
        return dict.__getitem__(self, key.lower())

>>> print eval(raw_input('Okay kids, type in an expression:',
LowerCaseDict(globals()))

Okay kids, type in an expression:  HeX(20)
0x14


class SpreadSheet:
    _cells = {}
    def __setitem__(self, key, formula):
        self._cells[key] = formula
    def __getitem__(self, key):
        return eval(self._cells[key], self)

ss = SpreadSheet()
ss['a1'] = '5'
ss['a2'] = 'a1*5'
ss['a2']

While this seems like a minor feature request, it presents amazing
opportunities to avoid writing lexers, parsers, interpreters by
transferring the work to the python interpreter.  Writing little
languages like the spreadsheet class becomes almost trivial.




II.  Timer.timeit to take an optional context argument.

>>> from timeit import Timer
>>> a = 20
>>> def f(x):
...     return x + 20

>>> Timer('f(10)', env=globals()).timeit()
1.234

The idea is to make it *much* easier to time functions or parts of
existing modules.  Having to write-out the definition in a long setup
string is cumbersome and interferes with syntax highlighting.  To
instrument modules with timeit, the timer setup string currently has to
re-import the whole module -- passing in an execution environment is
much more straight-forward.  And, as the example shows, passing in
globals() makes it easier to experiment with timings using the
interpreter.




III. enumerate to take an optional argument:
 
         enumerate([firstnumber=0,] iterable)

The idea is to make enumerate() a complete solution to the loop counter
problem:

for lineno, line in enumerate(1, file('hist.log')):
    print lineno, line




IV. list.sorted() to become just sorted().  All of the common use cases
read better without the "list." prefix:

>>> topscores = sorted(team.score for team in teams)[:10]
>>> for k, v in sorted(mydict.iteritems()):
       . . .
>>> byAge = operator.itemgetter('age')
>>> for age, students in groupby(sorted(studentbody, key=byAge),
key=byAge):
...      print "Age group:", age
...      for student in students:
...           print '...', student.name, student.address

Also, this Christmas wish cures the weirdness that comes from the
classmethod approach:  

>>> [3,2,1].sorted('cbs')
['b', 'c', 's'].  

While I worry about the total number of builtins, opposing them on
principle is a recipe for putting tools where they don't belong.  IMO,
making list.sorted() a class method was a clumsy way of avoiding the
obvious solution and left it smelling a bit hackish.



Raymond Hettinger


From fredrik at pythonware.com  Sun Dec 14 17:05:48 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Sun Dec 14 17:05:58 2003
Subject: [Python-Dev] Re: Christmas Wishlist
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
Message-ID: <brims1$mml$1@sea.gmane.org>

Raymond wrote:

> I. eval() to accept custom mapping arguments for globals and locals.
> This makes it possible to write a smart __getitem__ method for
> applications like spreadsheets or case-insensitive evaluation.

+0. if you can do it without affecting performance in the dictionary
case, that is.  -1 if you can't.

> II.  Timer.timeit to take an optional context argument.

+1

> III. enumerate to take an optional argument:

+0 (it's not that hard or expensive to add an offset inside the loop)

> IV. list.sorted() to become just sorted().  All of the common use cases
> read better without the "list." prefix:

+1

one more:

V. a join() builtin (join(x,s) becomes s.join(x)).  All of the common sep.join()
use cases read better without the string literal prefix.

(I'll leave the rest for later)

</F>




From bac at OCF.Berkeley.EDU  Sun Dec 14 17:06:04 2003
From: bac at OCF.Berkeley.EDU (Brett)
Date: Sun Dec 14 17:06:14 2003
Subject: [Python-Dev] Changing semantics of issubclass when accepting a tuple
Message-ID: <3FDCDECC.1070405@ocf.berkeley.edu>

Bug #858016 shows how issubclass can segfault if you pass a deeply 
nested tuple as issubclass' second argument.  I would like to propose 
changing the semantics so that it only accepts a flat tuple; when a 
tuple is passed in to do a quick check that each item is a class and if 
it is pass that to issubclass.

But this is a change in semantics.  Tim said I should clear it here 
first so that is what I am doing.  Anyone have issues if I do this?  And 
if so, any objections of backporting to 2.3?

-Brett

From guido at python.org  Sun Dec 14 17:20:56 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 17:21:13 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Sun, 14 Dec 2003 13:03:08 EST."
	<002601c3c26c$880e4940$e841fea9@oemcomputer> 
References: <002601c3c26c$880e4940$e841fea9@oemcomputer> 
Message-ID: <200312142221.hBEML7Y15793@c-24-5-183-134.client.comcast.net>

> > whether regular
> > expressions are in or out.
> 
> Regular expressions should be omitted.  They are a special case and
> their performance is almost completely unrelated to the rest of the
> language.

Yeah, but I imagine Parrot will have to support them anyway, and I
could use them for a parser I was thinking of as a benchmark. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Dec 14 17:27:52 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 17:27:59 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Sun, 14 Dec 2003 13:27:37 EST."
	<a06010202bc025bc5dc5a@[10.0.1.2]> 
References: <a06010201bc023e3253de@[10.0.1.2]>
	<200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net> 
	<a06010202bc025bc5dc5a@[10.0.1.2]> 
Message-ID: <200312142227.hBEMRr915819@c-24-5-183-134.client.comcast.net>

> Regexes were out. As I remember the challenge was pure interpreter 
> speed, so no dependence on IO or external modules. I'll go see if I 
> can dig out the rules. (You might have 'em on disk if you save old 
> presentations -- I'm pretty sure you had 'em up on the projector 
> during the announcement (or if it was me I've lost 'em :))

I was worried I lost it when my laptop died, but I managed to save
a copy.  The rules for the benchmark are:

- Live in front of OSCON audience
- Using the same Pentium Linux box
- Latest CVS version
- Best of three runs
- Python 2.3 bytecode
- Bytecode transformations OK

- To be decided by the Python folks
- Bytecode frozen in December 2003
- Should run for at least 30 seconds
- Output must be verifiably correct
- Full builtins OK
- No input or files (static data OK)
- No use of extension modules
- No use of regular expressions

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Dec 14 17:30:04 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 17:30:09 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Sun, 14 Dec 2003 13:59:32 EST."
	<LNBBLJKPBEHFEDALKOLCOEFAHMAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCOEFAHMAB.tim.one@comcast.net> 
Message-ID: <200312142230.hBEMU4H15844@c-24-5-183-134.client.comcast.net>

> > A simple benchmark like pystone doesn't tell much.  Large systems
> > used for real work are more interesting and a more realistic measure
> > of the language+implementation than small synthetic benchmarks.  For
> > example, Zope might be a good choice, but measuring its performance
> > is an interesting (and difficult) problem in itself.
> 
> Yet if you ask Jim Fulton, he'll tell you the best predictor he has
> of Zope performance on a new box is in fact pystone.  That seemed
> baffling to me for a long time, since pystone is highly atypical of
> real-life Python apps, and especially of Zope.  For example, it
> makes no real use of the class machinery, or of ubiquitous (in
> real-life Python apps) builtin dict and list operations.
> 
> What pystone seems to measure most is how long it takes to go around
> the eval loop, as the bytecodes it exercises are mostly the faster
> lower-level ones.  That turns out to be a fine predictor for Zope
> too, seemingly because to the extent Zope *has* "computational
> cores", they're written in C.  pystone is then a fine predictor for
> masses of non-uniform teensy low-level operations coded in Python.

These days, I use pystone more frequently to compare CPU speed than to
benchmark Python. :-)

> If you want a benchmark to make CPython look good, do a few hundred
> thousand very-long int multiplications, stick 'em in a list, and
> sort it <wink>.

Ah, but then Dan will just add Karatsuba multiplication to Parrot,
too.  And AFAIK, Timsort isn't patented. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Dec 14 17:32:50 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 17:32:57 2003
Subject: [Python-Dev] Changing semantics of issubclass when accepting a
	tuple
In-Reply-To: Your message of "Sun, 14 Dec 2003 14:06:04 PST."
	<3FDCDECC.1070405@ocf.berkeley.edu> 
References: <3FDCDECC.1070405@ocf.berkeley.edu> 
Message-ID: <200312142232.hBEMWon15878@c-24-5-183-134.client.comcast.net>

> Bug #858016 shows how issubclass can segfault if you pass a deeply 
> nested tuple as issubclass' second argument.  I would like to propose 
> changing the semantics so that it only accepts a flat tuple; when a 
> tuple is passed in to do a quick check that each item is a class and if 
> it is pass that to issubclass.

Works for me.

> But this is a change in semantics.  Tim said I should clear it here 
> first so that is what I am doing.  Anyone have issues if I do this?  And 
> if so, any objections of backporting to 2.3?

Let's not mess with 2.3 semantics.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tdelaney at avaya.com  Sun Dec 14 17:37:20 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun Dec 14 17:37:27 2003
Subject: [Python-Dev] Pie-thon benchmarks
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE06B1@au3010avexu1.global.avaya.com>

> From: Guido van Rossum
> 
> Ah, but then Dan will just add Karatsuba multiplication to Parrot,
> too.  And AFAIK, Timsort isn't patented. :-)

True. But do you really expect that anyone except Tim could get it *right*???

Tim Delaney

From art at wiktorsadowski.com  Sun Dec 14 18:49:42 2003
From: art at wiktorsadowski.com (Wiktor Sadowski)
Date: Sun Dec 14 18:48:45 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1) - custom
	Py_Initialize()
Message-ID: <002301c3c29c$f2753480$0201010a@kret>

Adding a few lines (below) to Python would enable custom Py_Initialize()
and would allow to setup on time different than existing
(import.c-zipimport.c) import mechanisms
and changing some other Python-initialization internals without messing with
Python core.

It should make life easier for embedders and help writing specialized Python
executables (like Zope or Quixote).
(We have such specialized executable to run  packed Python programs
remotely,kind of Java Web Start).
>From what I know Python installers (py2exe,Gordon McMillan's installer)
could benefit from such addition as well.

All these functions simply expose some modules globals. They don't change
anything in Python core.

Best regards,
Wiktor Sadowski


/*pythonrun.c*/
void Internal_Py_SetInitialized(int){initialized=i;}
void  Internal_PyUnicode_Init(void){_PyUnicode_Init();}
void  Internal_PyUnicode_Fini(void){_PyUnicode_Fini();}
void  Internal_PyGILState_Init(PyInterpreterState *s, PyThreadState
*ts){_PyGILState_Init(s,ts);}
void  Internal_PyGILState_Fini(void){_PyGILState_Fini();}
long Internal_PyGC_Collect(void){return PyGC_Collect();}

/*pythonrun.h*/
PyAPI_FUNC(void) Internal_Py_SetInitialized(int i);
PyAPI_FUNC(void) Internal_PyUnicode_Init(void);
PyAPI_FUNC(void) Internal_PyUnicode_Fini(void);
PyAPI_FUNC(void) Internal_PyGILState_Init(PyInterpreterState *,
PyThreadState *);
PyAPI_FUNC(void) Internal_PyGILState_Fini(void);
PyAPI_FUNC(long) Internal_PyGC_Collect(void);


From tim.one at comcast.net  Sun Dec 14 20:16:08 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 14 20:16:17 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE06B1@au3010avexu1.global.avaya.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEGHHMAB.tim.one@comcast.net>

[Guido]
>> Ah, but then Dan will just add Karatsuba multiplication to Parrot,
>> too.  And AFAIK, Timsort isn't patented. :-)

[Delaney, Timothy C (Timothy)]
> True. But do you really expect that anyone except Tim could get it
> *right*???

Yup!  I documented the approach in soul-numbing detail.  Last I saw, Perl
had an adaptive mergesort too, and in principle that's the same thing.  The
Python implementation is a lot more long-winded, because it's trying harder
to use less memory and be friendlier to the cache, uses only less-than
instead of Perl's 3-outcome comparison primitive, and tries harder to
minimize wasted comparisons if the data turns out to be random -- but that's
all shallow complication.  The core idea can be explained in less than 50
pages of dense text <wink>.  OK, in less than 50 words:  when one input to a
merge "wins" several times in a row, you can potentially save a ton of
comparisons by skipping ahead.  All the rest is engineering.


From tim.one at comcast.net  Sun Dec 14 20:33:50 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 14 20:33:56 2003
Subject: [Python-Dev] Changing semantics of issubclass when accepting
	atuple
In-Reply-To: <200312142232.hBEMWon15878@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCKEGIHMAB.tim.one@comcast.net>

[Brett]
>> But this is a change in semantics.  Tim said I should clear it here
>> first so that is what I am doing.  Anyone have issues if I do this?
>> And if so, any objections of backporting to 2.3?

[Guido]
> Let's not mess with 2.3 semantics.

That's the rub:  the segfaults occur in 2.3 too, of course, so should also
be fixed there.  Looks like isinstance() has the same vulnerability, and
that the code comments don't match the docs.

"""
class C(object):
    pass

classtup = C
for i in xrange(50000):
    classtup = (classtup,)

if 0:	# segfaults on both branches; maybe on your box 50000 is too small
    print issubclass(C, classtup)
else:
    print isinstance(C(), classtup)
"""

The implementations even have this shared prophetic <wink> comment:

		/* Not a general sequence -- that opens up the road to
		   recursion and stack overflow. */

The error messages plainly say that the example segfaulting program isn't a
supported use:

			"isinstance() arg 2 must be a class, type,"
			" or tuple of classes and types"))

					"issubclass() arg 2 must be a class"
					" or tuple of classes"))

The documentation could well have *meant* that, too:

>>> print isinstance.__doc__
isinstance(object, class-or-type-or-tuple) -> bool

Return whether an object is an instance of a class or of a subclass thereof.
With a type as second argument, return whether that is the object's type.
The form using a tuple, isinstance(x, (A, B, ...)), is a shortcut for
isinstance(x, A) or isinstance(x, B) or ... (etc.).
>>> print issubclass.__doc__
issubclass(C, B) -> bool

Return whether class C is a subclass (i.e., a derived class) of class B.
When using a tuple as the second argument issubclass(X, (A, B, ...)),
is a shortcut for issubclass(X, A) or issubclass(X, B) or ... (etc.).
>>>

The idea that arbitrarily nested tuples is allowed comes from that the
docstrings imply it (more accidentally than on purpose, it seems to me), and
that the implementations don't actually check for what its error messages
claim must be the case.

Finally, the docs in the Library Reference Manual are schizophrenic about
the issue at hand.  The issubclass() docs appear to disallow the possibility
of nested tuples, while the isinstance() docs explictly allow them.


From guido at python.org  Sun Dec 14 21:18:38 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 21:18:44 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Mon, 15 Dec 2003 09:37:20 +1100."
	<338366A6D2E2CA4C9DAEAE652E12A1DEFE06B1@au3010avexu1.global.avaya.com> 
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE06B1@au3010avexu1.global.avaya.com>
Message-ID: <200312150218.hBF2Ict16017@c-24-5-183-134.client.comcast.net>

> > And AFAIK, Timsort isn't patented. :-)
> 
> True. But do you really expect that anyone except Tim could get it
> *right*???

Yes.  As a matter of fact, it was ported to Jython in record time.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sun Dec 14 21:42:47 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 14 21:42:54 2003
Subject: [Python-Dev] Changing semantics of issubclass when accepting
	atuple
In-Reply-To: Your message of "Sun, 14 Dec 2003 20:33:50 EST."
	<LNBBLJKPBEHFEDALKOLCKEGIHMAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCKEGIHMAB.tim.one@comcast.net> 
Message-ID: <200312150242.hBF2gld16077@c-24-5-183-134.client.comcast.net>

> [Brett]
> >> But this is a change in semantics.  Tim said I should clear it here
> >> first so that is what I am doing.  Anyone have issues if I do this?
> >> And if so, any objections of backporting to 2.3?
> 
> [Guido]
> > Let's not mess with 2.3 semantics.
> 
> That's the rub:  the segfaults occur in 2.3 too, of course, so should also
> be fixed there.  Looks like isinstance() has the same vulnerability, and
> that the code comments don't match the docs.
> 
> """
> class C(object):
>     pass
> 
> classtup = C
> for i in xrange(50000):
>     classtup = (classtup,)
> 
> if 0:	# segfaults on both branches; maybe on your box 50000 is too small
>     print issubclass(C, classtup)
> else:
>     print isinstance(C(), classtup)
> """
> 
> The implementations even have this shared prophetic <wink> comment:
> 
> 		/* Not a general sequence -- that opens up the road to
> 		   recursion and stack overflow. */
> 
> The error messages plainly say that the example segfaulting program isn't a
> supported use:
> 
> 			"isinstance() arg 2 must be a class, type,"
> 			" or tuple of classes and types"))
> 
> 					"issubclass() arg 2 must be a class"
> 					" or tuple of classes"))
> 
> The documentation could well have *meant* that, too:
> 
> >>> print isinstance.__doc__
> isinstance(object, class-or-type-or-tuple) -> bool
> 
> Return whether an object is an instance of a class or of a subclass thereof.
> With a type as second argument, return whether that is the object's type.
> The form using a tuple, isinstance(x, (A, B, ...)), is a shortcut for
> isinstance(x, A) or isinstance(x, B) or ... (etc.).
> >>> print issubclass.__doc__
> issubclass(C, B) -> bool
> 
> Return whether class C is a subclass (i.e., a derived class) of class B.
> When using a tuple as the second argument issubclass(X, (A, B, ...)),
> is a shortcut for issubclass(X, A) or issubclass(X, B) or ... (etc.).
> >>>
> 
> The idea that arbitrarily nested tuples is allowed comes from that the
> docstrings imply it (more accidentally than on purpose, it seems to me), and
> that the implementations don't actually check for what its error messages
> claim must be the case.
> 
> Finally, the docs in the Library Reference Manual are schizophrenic about
> the issue at hand.  The issubclass() docs appear to disallow the possibility
> of nested tuples, while the isinstance() docs explictly allow them.

It was my intention to allow nested tuples, just not insanely deeply
nested tuples.  The imagined use case was having names for types or
groups of types, and being able to combine these without having to
know whether a name refers to a type or a group.

I don't know how important this use case is in practice, given that
the most complicated use I've ever made of this myself has been
isinstance(x, (int, long, float, complex)).

So I think we could easily require a flat tuple in 2.4 and be done
with it.  On the off chance that someone uses nested tuples in 2.3,
how much work would it be to have a helper routine that takes a
nesting depth argument, and clipping the nesting dept to e.g. 20?
That should be sufficient for practical purposes. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Sun Dec 14 21:51:33 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Sun Dec 14 21:52:22 2003
Subject: [Python-Dev] Changing semantics of issubclass when
	accepting	atuple
In-Reply-To: <200312150242.hBF2gld16077@c-24-5-183-134.client.comcast.net>
Message-ID: <200312150251.hBF2pX529463@oma.cosc.canterbury.ac.nz>

Guido:

> It was my intention to allow nested tuples, just not insanely deeply
> nested tuples.  The imagined use case was having names for types or
> groups of types, and being able to combine these without having to
> know whether a name refers to a type or a group.

The same effect could be achieved by defining a class...

  class my_favourite_types(float, complex):
    pass

  if isinstance(x, (int, long, my_favourite_types)):
    ...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From jepler at unpythonic.net  Sun Dec 14 22:03:47 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Sun Dec 14 22:03:54 2003
Subject: [Python-Dev] Christmas Wishlist
In-Reply-To: <000201c3c289$976fcae0$e841fea9@oemcomputer>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
Message-ID: <20031215030347.GC27067@unpythonic.net>

On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:
> I. eval() to accept custom mapping arguments for globals and locals.
> This makes it possible to write a smart __getitem__ method for
> applications like spreadsheets or case-insensitive evaluation.

http://mail.python.org/pipermail/python-dev/2002-October/029770.html
[and the rest of that thread]

A 3% slowdown in the common case was considered too big to allow a
feature like this to be included.  The slowdown might be smaller if
the builtin module wasn't allowed to be anything but a dict instance, or
might disappear if there were two versions of ceval.c were included, and
one used PyObject_ APIs and the other used PyDict_ APIs (as long as only
one was fighting for space in the cpu cache, main memory, etc)

Jeff

From tdelaney at avaya.com  Sun Dec 14 22:30:38 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Sun Dec 14 22:30:46 2003
Subject: [Python-Dev] Pie-thon benchmarks
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE07A2@au3010avexu1.global.avaya.com>

> From: Guido van Rossum
> 
> > > And AFAIK, Timsort isn't patented. :-)
> > 
> > True. But do you really expect that anyone except Tim could get it
> > *right*???
> 
> Yes.  As a matter of fact, it was ported to Jython in record time.

Bah. I'm giving up on the non-witty comments for today :(

Tim Delaney

From guido at python.org  Mon Dec 15 00:59:57 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 01:00:05 2003
Subject: [Python-Dev] Changing semantics of issubclass when accepting
	atuple
In-Reply-To: Your message of "Mon, 15 Dec 2003 15:51:33 +1300."
	<200312150251.hBF2pX529463@oma.cosc.canterbury.ac.nz> 
References: <200312150251.hBF2pX529463@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312150559.hBF5xw616334@c-24-5-183-134.client.comcast.net>

> The same effect could be achieved by defining a class...
> 
>   class my_favourite_types(float, complex):
>     pass
> 
>   if isinstance(x, (int, long, my_favourite_types)):
>     ...

I hope you forgot the smiley, as this would otherwise have to be
explained as a typical "before the first coffee" lapse. :)

First of all, you can't multiply inherit from built-in classes.  But
even if you could, it would be wrong: while an instance of
my_favourite_types would indeed be an instance of float,
3.14 is not an instance of my_favourite_types!  You'd need
my_favourite_types to be a common base class of float and complex for
the test to work the way you want it.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From lists at webcrunchers.com  Mon Dec 15 02:52:50 2003
From: lists at webcrunchers.com (JD)
Date: Mon Dec 15 02:52:59 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
Message-ID: <AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>

I can't get my Python 2.3.3c1 program to work. It keeps core dumping
(leaving python.core). I'm not doing anything strange, just documented
stuff from the manuals. I tried 'gdb -c python.core' and then using
'back', but there are no labels compiled in (I hate that...); how can I
get the labels to be included? "./configure --debug" doesn't work. I
assume without the labels the numbers are meaningless. Running this 
under
OpenBSD 3.2. Then, where should I submit my core dump (if anywhere?)?

I used to have the URL bookmarked,  but just upgraded and lost most
of my old bookmarks....

John


From mwh at python.net  Mon Dec 15 05:11:05 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 15 05:11:08 2003
Subject: [Python-Dev] urllib2 patch reviewer (852995)?
In-Reply-To: <Pine.LNX.4.58.0312141533370.428@alice> (John J. Lee's message
	of "Sun, 14 Dec 2003 16:15:28 +0000 (GMT)")
References: <Pine.LNX.4.58.0312141533370.428@alice>
Message-ID: <2mekv6jsqu.fsf@starship.python.net>

John J Lee <jjl@pobox.com> writes:

> Jeremy Hylton just checked in a relatively major patch of mine to urllib2.
> He's obviously very busy, though, so it would be nice if somebody else who
> knows urllib2 could look at it, too.

I don't have the know-how to review the patch, but I can notice that
test_urllib2 now seems to be creating uncollectable garbage...

Cheers,
mwh

-- 
  I hate leaving Windows95 boxes publically accessible, so shifting
  even to NT is a blessing in some ways.  At least I can reboot them
  remotely in a sane manner, rather than having to send them malformed
  packets.      -- http://bofhcam.org/journal/journal.html, 20/06/2000

From mwh at python.net  Mon Dec 15 05:14:17 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 15 05:14:19 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <200312142230.hBEMU4H15844@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Sun, 14 Dec 2003 14:30:04 -0800")
References: <LNBBLJKPBEHFEDALKOLCOEFAHMAB.tim.one@comcast.net>
	<200312142230.hBEMU4H15844@c-24-5-183-134.client.comcast.net>
Message-ID: <2mad5ujsli.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> If you want a benchmark to make CPython look good, do a few hundred
>> thousand very-long int multiplications, stick 'em in a list, and
>> sort it <wink>.
>
> Ah, but then Dan will just add Karatsuba multiplication to Parrot,
> too.  And AFAIK, Timsort isn't patented. :-)

If something important -- like a pieing <wink> -- depends on it, and
the ints are long enough, it's not that hard to do better than
Karatsuba multiplication...

Cheers,
mwh

-- 
  A.D. 1517: Martin Luther nails his 95 Theses to the church door and
             is promptly moderated down to (-1, Flamebait).
        -- http://slashdot.org/comments.pl?sid=01/02/09/1815221&cid=52
                                        (although I've seen it before)

From mwh at python.net  Mon Dec 15 07:44:53 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 15 07:43:37 2003
Subject: [Python-Dev] CVS bustage?
In-Reply-To: <2m8ylwjov0.fsf@starship.python.net>
Message-ID: <7B1EAA98-2EFC-11D8-AD8E-0003931DF95C@python.net>


On Monday, Dec 1, 2003, at 20:54 Europe/Stockholm, Michael Hudson wrote:

> Guido van Rossum <guido@python.org> writes:
>
>>> I'm seeing test_trace dump core from current CVS.  Anyone else?
>>
>> Not here (Red Hat 9, production build).
>
> Nor anyone on #python, either.

If anyone cares, I worked this out in the end: I had some dubious 
compiler changes and backed them out -- but forgot to wipe away the 
.pyc files.  Argh!

Cheers,
mwh


From barry at python.org  Mon Dec 15 09:54:17 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 09:54:22 2003
Subject: [Python-Dev] Python-checkins list
In-Reply-To: <002901c3c272$a35dce40$e841fea9@oemcomputer>
References: <002901c3c272$a35dce40$e841fea9@oemcomputer>
Message-ID: <1071500056.970.73.camel@anthem>

On Sun, 2003-12-14 at 13:46, Raymond Hettinger wrote:
> Using reply-all for a checkin message used to notify the committer and
> cc the checkins list.  Now, it also cc?s the python-dev list but not
> the committer.

Discussions need to be redirected from the checkins list to python-dev. 
However there should be a Mailman option to copy the From header to the
Reply-To header when doing Reply-To munging so the original poster is
included in the response.  I'll see if I can add, er, fix <wink> that
for Mailman 2.1.4.

-Barry



From skip at pobox.com  Mon Dec 15 09:55:18 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 15 09:55:22 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
	<AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
Message-ID: <16349.52054.372100.462135@montanaro.dyndns.org>


    John> Then, where should I submit my core dump (if anywhere?)?

Submit bug reports here:

    http://sourceforge.net/tracker/?group_id=5470&atid=105470

but please don't attach core files (they are too big and nobody will be able
to make use of them).  Instead, attach a small Python script which
demonstrates how to generate a core dump.

Skip

From barry at python.org  Mon Dec 15 09:56:49 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 09:56:53 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <brims1$mml$1@sea.gmane.org>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org>
Message-ID: <1071500208.970.76.camel@anthem>

On Sun, 2003-12-14 at 17:05, Fredrik Lundh wrote:

> V. a join() builtin (join(x,s) becomes s.join(x)).  All of the common sep.join()
> use cases read better without the string literal prefix.

Really?  I predict everlasting confusion on argument order.  It's not
obvious from the method name whether the joining separator comes first
or second.

My wish list:

V <wink>. Explicit global imports.  I want a way to spell "import this
from the global standard library even if there's a this.py  in the local
directory".

VI. PEP 318 or something like it.

-Barry



From martin at v.loewis.de  Mon Dec 15 09:59:42 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Mon Dec 15 10:00:11 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
	<AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
Message-ID: <m31xr6p1nl.fsf@mira.informatik.hu-berlin.de>

JD <lists@webcrunchers.com> writes:

> Running this under OpenBSD 3.2. Then, where should I submit my core
> dump (if anywhere?)?

You should submit bug reports to sf.net/projects/python, and provide a
link to the core dump on a server of your control (as SF has a limit
on file upload size).

Please understand that several months, or even years, may pass until
such a bug report is looked at, so if you actually need this to work,
you will have to investigate it in more detail.

On Unix, Python is, by default, compiled with symbols if the compiler
is gcc. To find out why gdb does not see symbols, you should post a
few lines of make output, showing typical compiler and linker
invocations.

Regards,
Martin

From fdrake at acm.org  Mon Dec 15 10:14:24 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Mon Dec 15 10:14:30 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <1071500208.970.76.camel@anthem>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
Message-ID: <16349.53200.37613.119494@sftp.fdrake.net>


Barry Warsaw writes:
 > V <wink>. Explicit global imports.  I want a way to spell "import this
 > from the global standard library even if there's a this.py  in the local
 > directory".

I like the syntax:

    import global <dotted-name>

and

    from global <dotted-name> import <name>

Look Guido, no new keywords!  ;-)


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From jeremy at zope.com  Mon Dec 15 10:50:47 2003
From: jeremy at zope.com (Jeremy Hylton)
Date: Mon Dec 15 10:54:27 2003
Subject: [Python-Dev] urllib2 patch reviewer (852995)?
In-Reply-To: <2mekv6jsqu.fsf@starship.python.net>
References: <Pine.LNX.4.58.0312141533370.428@alice>
	<2mekv6jsqu.fsf@starship.python.net>
Message-ID: <1071503446.15985.651.camel@localhost.localdomain>

On Mon, 2003-12-15 at 05:11, Michael Hudson wrote:
> John J Lee <jjl@pobox.com> writes:
> 
> > Jeremy Hylton just checked in a relatively major patch of mine to urllib2.
> > He's obviously very busy, though, so it would be nice if somebody else who
> > knows urllib2 could look at it, too.
> 
> I don't have the know-how to review the patch, but I can notice that
> test_urllib2 now seems to be creating uncollectable garbage...

Yes.  Didn't get a chance to fix that the other night.  One of the
classes has a pretty useless __del__ that might have helped before we
had gc for cycles.  The original urllib2 code is pretty old.

Jeremy



From guido at python.org  Mon Dec 15 11:03:31 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 11:03:37 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 10:14:24 EST."
	<16349.53200.37613.119494@sftp.fdrake.net> 
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem> 
	<16349.53200.37613.119494@sftp.fdrake.net> 
Message-ID: <200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>

> Barry Warsaw writes:
>  > V <wink>. Explicit global imports.  I want a way to spell "import this
>  > from the global standard library even if there's a this.py  in the local
>  > directory".
> 
> I like the syntax:
> 
>     import global <dotted-name>
> 
> and
> 
>     from global <dotted-name> import <name>
> 
> Look Guido, no new keywords!  ;-)

I'd much rather invent new syntax to spell *local* imports.  I like

  from .module import something

You can even generalize and write

  from ..module import something

to reference your package's parent package. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Mon Dec 15 11:05:42 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 15 11:05:47 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <2mad5ujsli.fsf@starship.python.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>

[Michael Hudson]
> If something important -- like a pieing <wink> -- depends on it, and
> the ints are long enough, it's not that hard to do better than
> Karatsuba multiplication...

"Not that hard" depends on your background.  The idea is to bury Dan under
esoteric algorithms from a huge number of obscure specialties <wink>.


From mwh at python.net  Mon Dec 15 11:17:02 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 15 11:17:07 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net> (Tim
	Peters's message of "Mon, 15 Dec 2003 11:05:42 -0500")
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>
Message-ID: <2mk74yhx8h.fsf@starship.python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> [Michael Hudson]
>> If something important -- like a pieing <wink> -- depends on it, and
>> the ints are long enough, it's not that hard to do better than
>> Karatsuba multiplication...
>
> "Not that hard" depends on your background. 

Granted, but while my background is maths it's not numerics and I know
where to find descriptions and implementations of complexity-wise
better algorithms.  Using them is just a matter of engineering, right?
<wink>

> The idea is to bury Dan under esoteric algorithms from a huge number
> of obscure specialties <wink>.

Now there's a plan!  Let's see, how do we do that?  Excessive use of
dicts, unicode esoterica and multiple inheritance seems like a good
start.

Cheers,
mwh

-- 
  Its unmanageable complexity has spawned more fear-preventing tools
  than any other language, but the solution _should_ have been to 
  create and use a language that does not overload the whole goddamn 
  human brain with irrelevant details.  -- Erik Naggum, comp.lang.lisp

From barry at python.org  Mon Dec 15 11:40:14 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 11:40:22 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
Message-ID: <1071506413.970.102.camel@anthem>

On Mon, 2003-12-15 at 11:03, Guido van Rossum wrote:

> I'd much rather invent new syntax to spell *local* imports.  I like
> 
>   from .module import something
> 
> You can even generalize and write
> 
>   from ..module import something
> 
> to reference your package's parent package. :-)

Actually, if truth be told, I'd love to just ban local imports.  I
understand that might be controversial though <wink>.

The next best thing would be to force local imports to use special
syntax (I was going to say "ugly" :) like what you suggest above, but
I'm afraid that that might be more disruptive, requiring future imports
and such.  I think having a syntax for spelling global imports can be
done in a backwards compatible way for 2.4.

-Barry



From tim.one at comcast.net  Mon Dec 15 11:46:24 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 15 11:46:28 2003
Subject: [Python-Dev] Berkeley support in release23-maint
Message-ID: <LNBBLJKPBEHFEDALKOLCGELHHMAB.tim.one@comcast.net>

Looking at

http://cvs.sf.net/viewcvs.py/python/python/dist/src/Lib/bsddb/__init__.py

it appear that 2+ months of Berkeley bugfixes (revs 1.10 through 1.12 of
__init__.py, at least) didn't get backported to the 2.3 maint branch, and so
are absent in 2.3.3c1.  Is that right?


From skip at pobox.com  Mon Dec 15 11:52:15 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 15 11:52:28 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <1071506413.970.102.camel@anthem>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
Message-ID: <16349.59071.26004.502201@montanaro.dyndns.org>


    Guido> I'd much rather invent new syntax to spell *local* imports.

    Barry> Actually, if truth be told, I'd love to just ban local imports.
    Barry> I understand that might be controversial though <wink>.

Okay, so what are global and local imports?  I thought I understood after
Barry's first post, but not now.  I see the <wink>, but figured there's
still some germ of desire in Barry's "ban local imports" comment.  What kind
of imports do we have today?  Do they relate to the position of the import
statement in the code or to the location of the module being imported along
sys.path?

Skip

From guido at python.org  Mon Dec 15 12:07:13 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 12:07:19 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 11:40:14 EST."
	<1071506413.970.102.camel@anthem> 
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net> 
	<1071506413.970.102.camel@anthem> 
Message-ID: <200312151707.hBFH7DI17519@c-24-5-183-134.client.comcast.net>

> On Mon, 2003-12-15 at 11:03, Guido van Rossum wrote:
> 
> > I'd much rather invent new syntax to spell *local* imports.  I like
> > 
> >   from .module import something
> > 
> > You can even generalize and write
> > 
> >   from ..module import something
> > 
> > to reference your package's parent package. :-)
> 
> Actually, if truth be told, I'd love to just ban local imports.  I
> understand that might be controversial though <wink>.

Works for me. :-)

> The next best thing would be to force local imports to use special
> syntax (I was going to say "ugly" :) like what you suggest above, but
> I'm afraid that that might be more disruptive, requiring future imports
> and such.  I think having a syntax for spelling global imports can be
> done in a backwards compatible way for 2.4.

Well, but since you want all imports to be global, it'd be insane to
introduce *new* syntax for global imports, wouldn't it?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 15 12:10:24 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 12:10:30 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 10:52:15 CST."
	<16349.59071.26004.502201@montanaro.dyndns.org> 
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem> 
	<16349.59071.26004.502201@montanaro.dyndns.org> 
Message-ID: <200312151710.hBFHAO817542@c-24-5-183-134.client.comcast.net>

> Okay, so what are global and local imports?  I thought I understood
> after Barry's first post, but not now.  I see the <wink>, but
> figured there's still some germ of desire in Barry's "ban local
> imports" comment.  What kind of imports do we have today?  Do they
> relate to the position of the import statement in the code or to the
> location of the module being imported along sys.path?

Traditionally, we've called these absolute (== global) and relative
(== local) imports.  The terms relate to the module's full package
name, usually equivalent to its location on the filesystem.  sys.path
and the placement of the import statement do not enter into it, except
that sys.path is the very definition of absolute import.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From lkcl at lkcl.net  Mon Dec 15 12:31:56 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Mon Dec 15 12:31:32 2003
Subject: [Python-Dev] rexec.py unuseable
Message-ID: <20031215173156.GC25203@lkcl.net>

i tried to reuse some code that i wrote for a project back when
python 2.0 and 2.1 were in use.

i must be about the only person in existence who wrote some code
that uses - and totally relies on - the rexec.py module.

what that code does (did) was to provide a database-based
method of running restricted programs for use to monitor and
manage remote servers.

i.e. it became in effect a distributed python operating system.

what happened was that the name of a python script, its
arguments, a relevant ip address and the server on which the
script was to be run were all entered into a database table,
and the server picked that up, loaded the script and its
arguments from the database and then ran the script, which
usually involved contacting the remote ip address.

so, not only was rhooks used to overload "import" such that
libraries used by scripts in the database could _also_ be
loaded from the database, but rexec.py was also used to restrict
what functions could be accessed.

i was quite happy with rexec.py as it stood.

it was the c++ functional equivalent of the "protected" convention
for c++ classes.

at the time that i tried to reuse the code, i discovered the lovely
"this module is broken in python 2.2 and 2.3" message.

well, uhm, if it's broken, why hasn't it been reverted to the 2.0
code?

i noted, some time about a year ago, that someone attempted to drill
down the rexec security into sub-modules.

i.e. they attempted to add the c++ functional equivalence of
the "private" convention for c++ classes.

consequently, a number of routines that wrote to files that i had
DELIBERATELY wrapped and made available to first-level rexec-restricted
library routines, e.g. the logging functions which output debug
information to log files, suddenly stopped working.

why?  whereas before, the debug routine could be called directly
but the write operation could not, because of the drill-down
effect, all of a sudden the write operation, as called _by_
the debug routine, is now restricted in the 2.1 (or early 2.2)
code, and my debug library routine now doesn't work, nor indeed
does anything else.


the thing is that the functionality of the python 2.0 rexec.py was
_exactly_ as i would have expected it to be, and it was working
_perfectly_ according to my expectations of that code.

i therefore believe that the attempts to take rexec one step further,
aka a "drill-down" effect, by making all functions that rexec functions
import _also_ restricted, to have been a mistake.

or, at least, not carefully thought out.

if such additional restrictions are to be added, i believe that they
should be added as a user-configureable parameter (e.g. named
recursive-restrict)

_please_ could the functionality of rexec.py be restored to python 2.0 
functionality, with the "rexec.py is broken" message only coming
up when the suggested-named parameter "recursive-restrict" is set.

maybe then i can consider reviving that code and re-releasing it,
because at the moment it's totally going to waste.

many many thanks,

l.


-- 
-- 
expecting email to be received and understood is a bit like
picking up the telephone and immediately dialing without
checking for a dial-tone; speaking immediately without listening
for either an answer or ring-tone; hanging up immediately and
then expecting someone to call you (and to be able to call you).
--
every day, people send out email expecting it to be received
without being tampered with, read by other people, delayed or
simply - without prejudice but lots of incompetence - destroyed.
--
please therefore treat email more like you would a CB radio
to communicate across the world (via relaying stations):
ask and expect people to confirm receipt; send nothing that
you don't mind everyone in the world knowing about...


From barry at python.org  Mon Dec 15 12:37:27 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 12:37:33 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <16349.59071.26004.502201@montanaro.dyndns.org>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<16349.59071.26004.502201@montanaro.dyndns.org>
Message-ID: <1071509846.970.105.camel@anthem>

On Mon, 2003-12-15 at 11:52, Skip Montanaro wrote:
>     Guido> I'd much rather invent new syntax to spell *local* imports.
> 
>     Barry> Actually, if truth be told, I'd love to just ban local imports.
>     Barry> I understand that might be controversial though <wink>.
> 
> Okay, so what are global and local imports?

Sorry, let me rephrase.  I'd love to ban relative imports.  Personally,
I think all imports should be specified using fully-qualified absolute
package paths.

-Barry



From barry at python.org  Mon Dec 15 12:39:47 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 12:39:54 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151707.hBFH7DI17519@c-24-5-183-134.client.comcast.net>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<200312151707.hBFH7DI17519@c-24-5-183-134.client.comcast.net>
Message-ID: <1071509987.970.108.camel@anthem>

On Mon, 2003-12-15 at 12:07, Guido van Rossum wrote:

> > Actually, if truth be told, I'd love to just ban local imports.  I
> > understand that might be controversial though <wink>.
> 
> Works for me. :-)

Yay! :)

> > The next best thing would be to force local imports to use special
> > syntax (I was going to say "ugly" :) like what you suggest above, but
> > I'm afraid that that might be more disruptive, requiring future imports
> > and such.  I think having a syntax for spelling global imports can be
> > done in a backwards compatible way for 2.4.
> 
> Well, but since you want all imports to be global, it'd be insane to
> introduce *new* syntax for global imports, wouldn't it?

If we banned relative (a.k.a. local) imports, yes definitely.

from __future__ global_imports

?

pep-time-ly y'rs,
-Barry



From edloper at gradient.cis.upenn.edu  Mon Dec 15 13:44:56 2003
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Mon Dec 15 12:43:15 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <E1AVw7a-0002ql-Lr@mail.python.org>
References: <E1AVw7a-0002ql-Lr@mail.python.org>
Message-ID: <3FDE0128.1040600@gradient.cis.upenn.edu>

Barry Warsaw <barry@python.org> wrote:
> Actually, if truth be told, I'd love to just ban local imports.  I
> understand that might be controversial though <wink>.

+1 (asuming that you're talking about package-local imports).

I've seen a couple people get bitten by the fact that a module gets 
loaded twice if there are separate local & global imports for it:

     % mkdir pkg

     % touch pkg/__init__.py

     % cat >pkg/a.py
     print 'A is being imported (not re-used)'
     class A: pass

     % cat >pkg/b.py
     from a import A as LocalA
     from pkg.a import A as GlobalA
     print isinstance(LocalA(), GlobalA)
     print isinstance(GlobalA(), LocalA)

     % PYTHONPATH=.; python pkg/b.py
     A is being imported (not re-used)
     A is being imported (not re-used)
     0
     0

Since pkg/a gets loaded twice, we end up with two versions of class A, 
which are not compatible.  In practice this usually comes up if a 
package uses local imports between submodules, and then an outside user 
uses a global import to get a submodule.

-Edward



From pycon-organizers at python.org  Mon Dec 15 12:54:27 2003
From: pycon-organizers at python.org (pycon-organizers@python.org)
Date: Mon Dec 15 12:54:29 2003
Subject: [Python-Dev] PyCon DC 2004 - Submissions Now Open
Message-ID: <E1AVwvL-0003nx-Pi@mail.python.org>


Dear Python User:

I am pleased to be able to announce that the submission
process for PyCon DC 2004 is now open. Please direct your
browser to

    http://submit.pycon.org/

and create your account by entering your email address and
leaving the password field blank - a password will be sent
to you by email, and you can submit your paper!

Please remember that your submissions should ideally be in
a single file, containing HTML or the reStructured Text
format. Ultimately we would like to publish all
accepted papers on the web, and these rules should make it
easier to do so. Send enquiries about other formats to

    pycon-organizers@python.org

If your paper is accepted and you prepare an electronic
presentation (in PDF, PythonPoint or PowerPoint) we will
also happily publish that on the web site when PyCon is over.

We are interested in any and all submissions about uses of
Python and the development of the language. Since there is
expected to be a strong educational community presence for
the next PyCon, teaching materials of various kinds are also
encouraged.

Please help us to make the next PyCon even better than the
first one. See you in March!

Sincerely

Steve Holden
Chairman
PyCon DC 2004

From klm at zope.com  Mon Dec 15 12:48:32 2003
From: klm at zope.com (Ken Manheimer)
Date: Mon Dec 15 12:54:58 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com>

On Mon, 15 Dec 2003, Guido van Rossum wrote:

> > Barry Warsaw writes:
> >  > V <wink>. Explicit global imports.  I want a way to spell "import this
> >  > from the global standard library even if there's a this.py  in the local
> >  > directory".
> > 
> > I like the syntax:
> > 
> >     import global <dotted-name>
> > 
> > and
> > 
> >     from global <dotted-name> import <name>
> > 
> > Look Guido, no new keywords!  ;-)
> 
> I'd much rather invent new syntax to spell *local* imports.  I like
> 
>   from .module import something
> 
> You can even generalize and write
> 
>   from ..module import something
> 
> to reference your package's parent package. :-)

As i've said before (though it's been a while) i think people want
this, it would be useful, and i think it would work well.

Barry wrote, in response to guido's message:

< Actually, if truth be told, I'd love to just ban local imports.  I
< understand that might be controversial though <wink>.

I don't understand the desire to totally prevent relative imports.  It
would be quite useful in avoiding growing module-naming problems,
where you have to be careful about shadowing a global module with one
in your package.

I *can* understand objecting to the ambiguity in the current
situation, where you don't know whether a module import will resolve
to a sibling module or one in higher up in the hierarchy.  That is why
i strongly prefer having a leading dot to explicitly signify a
relative import, and originally proposed it that way.  I expect that
the elimination of the ambiguity would mitigate the objection to
relative imports - and we would do away with the shadowing problems,
and even have easy relocation of modules, if there are cases where
software does need to move around.

Ken
klm@zope.com




From skip at pobox.com  Mon Dec 15 13:01:07 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 15 13:01:29 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031215173156.GC25203@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
Message-ID: <16349.63203.797449.677617@montanaro.dyndns.org>


    Luke> well, uhm, if [rexec]'s broken, why hasn't it been reverted to the
    Luke> 2.0 code?

My understanding was that in 2.0 it was also broken, but that (relatively)
recent changes to Python made the breakage easier to exploit.

As for alternatives, I suggest you post a query on comp.lang.python (aka
python-list@python.org).

Skip

From skip at pobox.com  Mon Dec 15 13:04:32 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 15 13:04:33 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <1071509846.970.105.camel@anthem>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<16349.59071.26004.502201@montanaro.dyndns.org>
	<1071509846.970.105.camel@anthem>
Message-ID: <16349.63408.143245.274390@montanaro.dyndns.org>


    >> Okay, so what are global and local imports?

    Barry> Sorry, let me rephrase.  I'd love to ban relative imports.
    Barry> Personally, I think all imports should be specified using
    Barry> fully-qualified absolute package paths.

That's fine with me.  Maybe continue to use "absolute" and "relative" (and
throw in "package" somewhere) instead of "global" and "local" when referring
to package import semantics?  "global" and "local" have a well-established
meaning in day-to-day Python programming which doesn't overlap much with the
import mechanism.

Skip



From guido at python.org  Mon Dec 15 13:36:44 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 13:36:50 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 12:39:47 EST."
	<1071509987.970.108.camel@anthem> 
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<200312151707.hBFH7DI17519@c-24-5-183-134.client.comcast.net> 
	<1071509987.970.108.camel@anthem> 
Message-ID: <200312151836.hBFIaie17722@c-24-5-183-134.client.comcast.net>

> > Well, but since you want all imports to be global, it'd be insane to
> > introduce *new* syntax for global imports, wouldn't it?
> 
> If we banned relative (a.k.a. local) imports, yes definitely.
> 
> from __future__ global_imports

I think this ought to be a *global* flag rather than a per-module
flag.  E.g. after setting

    sys.allow_relative_import = False

all imports anywhere would be interpreted as absolute imports.

This would mean you couldn't have some code that still uses relative
imports, but the problem with the __future__ statement is that it
seems so pointless: packages that only use absolute imports don't need
it, and packages that use relative imports break if it is used.

About the only time where the __future__ statement would make a
difference is when a package defines a local module whose name is the
same as that of a global module, *and* the package also wants to
import the global module.  I would personally solve that by renaming
the local module though...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 15 13:38:01 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 13:38:08 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 12:44:56 CST."
	<3FDE0128.1040600@gradient.cis.upenn.edu> 
References: <E1AVw7a-0002ql-Lr@mail.python.org>  
	<3FDE0128.1040600@gradient.cis.upenn.edu> 
Message-ID: <200312151838.hBFIc1q17738@c-24-5-183-134.client.comcast.net>

> I've seen a couple people get bitten by the fact that a module gets 
> loaded twice if there are separate local & global imports for it:
> 
>      % mkdir pkg
> 
>      % touch pkg/__init__.py
> 
>      % cat >pkg/a.py
>      print 'A is being imported (not re-used)'
>      class A: pass
> 
>      % cat >pkg/b.py
>      from a import A as LocalA
>      from pkg.a import A as GlobalA
>      print isinstance(LocalA(), GlobalA)
>      print isinstance(GlobalA(), LocalA)
> 
>      % PYTHONPATH=.; python pkg/b.py
>      A is being imported (not re-used)
>      A is being imported (not re-used)
>      0
>      0
> 
> Since pkg/a gets loaded twice, we end up with two versions of class A, 
> which are not compatible.  In practice this usually comes up if a 
> package uses local imports between submodules, and then an outside user 
> uses a global import to get a submodule.

That's actually a slightly different problem; sys.path should not
include any directories that are also packages on other directories in
sys.path.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 15 13:42:00 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 13:42:06 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 12:48:32 EST."
	<Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com> 
References: <Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com> 
Message-ID: <200312151842.hBFIg0K17763@c-24-5-183-134.client.comcast.net>

> I don't understand the desire to totally prevent relative imports.

It's because of TOOWTDI.

> It would be quite useful in avoiding growing module-naming problems,
> where you have to be careful about shadowing a global module with
> one in your package.

I don't see the problem, or I misuderstand what "it" refers to; it
seems you have this backwards if it refers to relative imports.  Say
my package P defines a submodule sys.  If we require absolute imports,
there is no ambiguity: the submodule sys must be imported as P.sys
while the standard sys module can be imported as simply sys.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Mon Dec 15 13:49:36 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Mon Dec 15 13:50:02 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>
Message-ID: <a06010202bc03b13b3134@[172.24.18.98]>

At 11:05 AM -0500 12/15/03, Tim Peters wrote:
>[Michael Hudson]
>>  If something important -- like a pieing <wink> -- depends on it, and
>>  the ints are long enough, it's not that hard to do better than
>>  Karatsuba multiplication...
>
>"Not that hard" depends on your background.  The idea is to bury Dan under
>esoteric algorithms from a huge number of obscure specialties <wink>.

Heh. You're expecting me to do far too much work. I plan on tackling 
bytecode optimization only if it turns out that a straightforward 
transform from python bytecode to parrot bytecode is isn't 
sufficiently faster than CPython. Funky algorithms expressed entirely 
in python bytecode won't make much of a difference...
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From barry at python.org  Mon Dec 15 13:51:07 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 13:51:10 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151836.hBFIaie17722@c-24-5-183-134.client.comcast.net>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<200312151707.hBFH7DI17519@c-24-5-183-134.client.comcast.net>
	<1071509987.970.108.camel@anthem>
	<200312151836.hBFIaie17722@c-24-5-183-134.client.comcast.net>
Message-ID: <1071514266.28081.2.camel@geddy>

On Mon, 2003-12-15 at 13:36, Guido van Rossum wrote:

> I think this ought to be a *global* flag rather than a per-module
> flag.  E.g. after setting
> 
>     sys.allow_relative_import = False
> 
> all imports anywhere would be interpreted as absolute imports.
> 
> This would mean you couldn't have some code that still uses relative
> imports, but the problem with the __future__ statement is that it
> seems so pointless: packages that only use absolute imports don't need
> it, and packages that use relative imports break if it is used.

You probably want something at the package level.  This may not be
feasible, but it seems like you want to be able to say, "this package
needs to allow relative imports for backwards compatibility".

-Barry





From lkcl at lkcl.net  Mon Dec 15 14:10:23 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Mon Dec 15 14:10:14 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <16349.63203.797449.677617@montanaro.dyndns.org>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
Message-ID: <20031215191023.GB26055@lkcl.net>

On Mon, Dec 15, 2003 at 12:01:07PM -0600, Skip Montanaro wrote:
> 
>     Luke> well, uhm, if [rexec]'s broken, why hasn't it been reverted to the
>     Luke> 2.0 code?
> 
> My understanding was that in 2.0 it was also broken, but that (relatively)
> recent changes to Python made the breakage easier to exploit.

the issue is [back-tracking to the 2.0 code],

were those "breakages" actually features, or were they definitely
"breakages"?

i.e.

was it 1)

	was it just unexpected behaviour in that a function
	_called_ by a rexec'd restricted routine could do,
	for example writes;

	was it that an unrestricted function, module or
	variable imported by a rexec'd restricted routine or
	module could be accessed.

	e.g. like this:

	contents of bypassmodule.py:

		import open as unrestrictedopen

	then code run under rexec:

		from bypassmodule import unrestrictedopen


	both of these things i would consider to be
	totally acceptable behaviour of the [2.] rexec.py
	module!!!

	i would NOT consider them to be a bug, and in fact
	are desirable behaviour in order to get my code working.

	i believe that there exists in [2.0] rexec a mechanism
	to specify what functions are allowed in a particular
	module, so i could restrict access to the
	bypassmodule.unrestrictedopen function, if necessary.


OR was it 2)

	a genuine bug.


> As for alternatives, I suggest you post a query on comp.lang.python (aka
> python-list@python.org).
 
  ta!

> Skip

-- 
-- 
expecting email to be received and understood is a bit like
picking up the telephone and immediately dialing without
checking for a dial-tone; speaking immediately without listening
for either an answer or ring-tone; hanging up immediately and
then expecting someone to call you (and to be able to call you).
--
every day, people send out email expecting it to be received
without being tampered with, read by other people, delayed or
simply - without prejudice but lots of incompetence - destroyed.
--
please therefore treat email more like you would a CB radio
to communicate across the world (via relaying stations):
ask and expect people to confirm receipt; send nothing that
you don't mind everyone in the world knowing about...

From python at rcn.com  Mon Dec 15 14:26:03 2003
From: python at rcn.com (Raymond Hettinger)
Date: Mon Dec 15 14:26:30 2003
Subject: [Python-Dev] Christmas Wishlist
In-Reply-To: <20031215030347.GC27067@unpythonic.net>
Message-ID: <004301c3c341$48053fe0$e841fea9@oemcomputer>

> On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:
> > I. eval() to accept custom mapping arguments for globals and locals.
> > This makes it possible to write a smart __getitem__ method for
> > applications like spreadsheets or case-insensitive evaluation.
> 
> http://mail.python.org/pipermail/python-dev/2002-October/029770.html
> [and the rest of that thread]

[Jeff Epler]
> A 3% slowdown in the common case was considered too big to allow a
> feature like this to be included.  The slowdown might be smaller if
> the builtin module wasn't allowed to be anything but a dict instance,
or
> might disappear if there were two versions of ceval.c were included,
and
> one used PyObject_ APIs and the other used PyDict_ APIs (as long as
only
> one was fighting for space in the cpu cache, main memory, etc)

Ordinarily, I would say that 3% is a reasonable price to pay for this
functionality but there is that pesky pie throwing event coming up.


Raymond Hettinger


From pje at telecommunity.com  Mon Dec 15 14:32:27 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec 15 14:32:37 2003
Subject: [Python-Dev] Christmas Wishlist
In-Reply-To: <004301c3c341$48053fe0$e841fea9@oemcomputer>
References: <20031215030347.GC27067@unpythonic.net>
Message-ID: <5.1.1.6.0.20031215142906.01e00ac0@telecommunity.com>

At 02:26 PM 12/15/03 -0500, Raymond Hettinger wrote:
> > On Sun, Dec 14, 2003 at 04:31:09PM -0500, Raymond Hettinger wrote:
> > > I. eval() to accept custom mapping arguments for globals and locals.
> > > This makes it possible to write a smart __getitem__ method for
> > > applications like spreadsheets or case-insensitive evaluation.
> >
> > http://mail.python.org/pipermail/python-dev/2002-October/029770.html
> > [and the rest of that thread]
>
>[Jeff Epler]
> > A 3% slowdown in the common case was considered too big to allow a
> > feature like this to be included.  The slowdown might be smaller if
> > the builtin module wasn't allowed to be anything but a dict instance,
>or
> > might disappear if there were two versions of ceval.c were included,
>and
> > one used PyObject_ APIs and the other used PyDict_ APIs (as long as
>only
> > one was fighting for space in the cpu cache, main memory, etc)
>
>Ordinarily, I would say that 3% is a reasonable price to pay for this
>functionality but there is that pesky pie throwing event coming up.

There is a workaround for this in CPython, which used to be used by Zope, 
and which I have used on occasion: scan the code object for LOAD_NAME 
opcodes, and note what names are used by the code block.  Then, look up 
those names in your non-dictionary dictionary, and copy them into a real 
dictionary.

So, it's got to be something pretty specialized to really need PyObject_* 
APIs instead of PyDict_* APIs in CPython at present.  Why penalize 
*everything* for such an specialized need?


From guido at python.org  Mon Dec 15 14:34:10 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 14:34:15 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 13:51:07 EST."
	<1071514266.28081.2.camel@geddy> 
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<200312151707.hBFH7DI17519@c-24-5-183-134.client.comcast.net>
	<1071509987.970.108.camel@anthem>
	<200312151836.hBFIaie17722@c-24-5-183-134.client.comcast.net> 
	<1071514266.28081.2.camel@geddy> 
Message-ID: <200312151934.hBFJYAB17994@c-24-5-183-134.client.comcast.net>

> > This would mean you couldn't have some code that still uses relative
> > imports, but the problem with the __future__ statement is that it
> > seems so pointless: packages that only use absolute imports don't need
> > it, and packages that use relative imports break if it is used.
> 
> You probably want something at the package level.  This may not be
> feasible, but it seems like you want to be able to say, "this package
> needs to allow relative imports for backwards compatibility".

But there is no backwards compatibility issue.  Absolute imports
always work, so instead of adding a "relative imports allowed" to your
package, you can just change the package to use only absolute imports.
Using only absolute imports is totally backwards compatible (unless
your toplevel package name is the same as the name of a submodule of
the package, which seems pretty sick).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 15 14:37:34 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 14:37:40 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Mon, 15 Dec 2003 13:49:36 EST."
	<a06010202bc03b13b3134@[172.24.18.98]> 
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>  
	<a06010202bc03b13b3134@[172.24.18.98]> 
Message-ID: <200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net>

> At 11:05 AM -0500 12/15/03, Tim Peters wrote:
> >[Michael Hudson]
> >>  If something important -- like a pieing <wink> -- depends on it, and
> >>  the ints are long enough, it's not that hard to do better than
> >>  Karatsuba multiplication...
> >
> >"Not that hard" depends on your background.  The idea is to bury Dan under
> >esoteric algorithms from a huge number of obscure specialties <wink>.

[Dan Sugalski]
> Heh. You're expecting me to do far too much work. I plan on tackling 
> bytecode optimization only if it turns out that a straightforward 
> transform from python bytecode to parrot bytecode is isn't 
> sufficiently faster than CPython. Funky algorithms expressed entirely 
> in python bytecode won't make much of a difference...

But Karatsuba multiplication and Timsort are implemented highly
efficiently in C, and are part of the "full set of standard builtins"
to which we agreed.

BTW, eval() is also part of that full set, and exec is part of the
language.  These invoke the Python bytecode compiler.  (And Jim
Hugunin's IronPython is really slow on this, like 60x slower than
CPython; I think Jim reported that Jython was 20x slower on exec().)

How do you plan to handle these?  What if the entire benchmark was
given as a triple quoted string that was passed to the exec statement?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Mon Dec 15 14:40:30 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 15 14:40:55 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031215191023.GB26055@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
Message-ID: <16350.3630.307765.836000@montanaro.dyndns.org>


    >> My understanding was that in 2.0 it was also broken, but that
    >> (relatively) recent changes to Python made the breakage easier to
    >> exploit.

    Luke> the issue is [back-tracking to the 2.0 code],

    Luke> were those "breakages" actually features, or were they definitely
    Luke> "breakages"?

Dunno.  Take a look at the 2.3 Whats New:

    http://www.python.org/doc/current/whatsnew/node18.html

Search for "rexec".  I don't know more than that.  It only mentions back as
far as 2.1, so I don't know if rexec could be made to work based upon the
code in 2.0.  Also, check this thread from python-dev:

    http://mail.python.org/pipermail/python-dev/2003-January/thread.html#31842

That thread includes some indication of what the problem is and how it got
worse in 2.2 and 2.3.

Skip

From klm at zope.com  Mon Dec 15 14:34:33 2003
From: klm at zope.com (Ken Manheimer)
Date: Mon Dec 15 14:40:59 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151836.hBFIaie17722@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>

On Mon, 15 Dec 2003, Guido van Rossum wrote:

> > It would be quite useful in avoiding growing module-naming problems,
> > where you have to be careful about shadowing a global module with
> > one in your package.
> 
> I don't see the problem, or I misuderstand what "it" refers to; it
> seems you have this backwards if it refers to relative imports.  Say
> my package P defines a submodule sys.  If we require absolute imports,
> there is no ambiguity: the submodule sys must be imported as P.sys
> while the standard sys module can be imported as simply sys.

I guess i think there is a case to be made for relative imports, and
it becomes apparent as an enterprise grows.

Increasingly at Zope corp we are mixing and matching packages to
deliver applications to a customer.  This is very encouraging, because
it means we are actually getting reuse out of the packages.

Currently, we can just plop the things in place, and use them.
Without relative imports, we would have to be editing the imports in
the packages in each place we use it.  This would increase the burden
of using the packages and even more of feeding back actual fixes -
which we then have to distinguish from what i would see as gratuitous
changes to edit import names.

If you would grant there's use to avoiding those edits, and so there's
use to having relative imports, then you might see a reason to solve
the problems where the names in a package conflict with those along
the load path.  Otherwise, if there's only absolute imports, there's
no ambiguity to resolve.  I'd say there's a legitimate need for
relative imports, and we need some explicit gesture to disambiguate
between a relative and absolute import, one way or the other.

Ken


From dan at sidhe.org  Mon Dec 15 14:51:39 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Mon Dec 15 14:51:56 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>            
	<a06010202bc03b13b3134@[172.24.18.98]>
	<200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010203bc03be7149c2@[172.24.18.98]>

At 11:37 AM -0800 12/15/03, Guido van Rossum wrote:
>  > At 11:05 AM -0500 12/15/03, Tim Peters wrote:
>>  >[Michael Hudson]
>>  >>  If something important -- like a pieing <wink> -- depends on it, and
>>  >>  the ints are long enough, it's not that hard to do better than
>>  >>  Karatsuba multiplication...
>>  >
>>  >"Not that hard" depends on your background.  The idea is to bury Dan under
>>  >esoteric algorithms from a huge number of obscure specialties <wink>.
>
>[Dan Sugalski]
>>  Heh. You're expecting me to do far too much work. I plan on tackling
>>  bytecode optimization only if it turns out that a straightforward
>>  transform from python bytecode to parrot bytecode is isn't
>>  sufficiently faster than CPython. Funky algorithms expressed entirely
>>  in python bytecode won't make much of a difference...
>
>But Karatsuba multiplication and Timsort are implemented highly
>efficiently in C, and are part of the "full set of standard builtins"
>to which we agreed.

In which case, well... time for me to write some C code.

>BTW, eval() is also part of that full set, and exec is part of the
>language.  These invoke the Python bytecode compiler.  (And Jim
>Hugunin's IronPython is really slow on this, like 60x slower than
>CPython; I think Jim reported that Jython was 20x slower on exec().)
>
>How do you plan to handle these?  What if the entire benchmark was
>given as a triple quoted string that was passed to the exec statement?

We agreed no eval as part of this challenge, since the point was the 
speed of the interpreter engine not of the compiler. I don't see that 
on the list of challenge points you posted earlier, though, so it was 
either omitted inadvertently or I got snookered. The former, I expect.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Mon Dec 15 14:54:46 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 14:54:54 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: Your message of "Mon, 15 Dec 2003 14:34:33 EST."
	<Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com> 
Message-ID: <200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>

> I guess i think there is a case to be made for relative imports, and
> it becomes apparent as an enterprise grows.
> 
> Increasingly at Zope corp we are mixing and matching packages to
> deliver applications to a customer.  This is very encouraging, because
> it means we are actually getting reuse out of the packages.
> 
> Currently, we can just plop the things in place, and use them.
> Without relative imports, we would have to be editing the imports in
> the packages in each place we use it.  This would increase the burden
> of using the packages and even more of feeding back actual fixes -
> which we then have to distinguish from what i would see as gratuitous
> changes to edit import names.

I know this line of reasoning fairly well.

You are taking 3rd party packages (or perhaps internally developed
packages) and copy them to a *different place in the package namespace
tree*.  Right?

But why do you have to give those packages a different full name?
That's the question that I've never seen answered adequately.

> If you would grant there's use to avoiding those edits, and so there's
> use to having relative imports, then you might see a reason to solve
> the problems where the names in a package conflict with those along
> the load path.  Otherwise, if there's only absolute imports, there's
> no ambiguity to resolve.  I'd say there's a legitimate need for
> relative imports, and we need some explicit gesture to disambiguate
> between a relative and absolute import, one way or the other.

I think that moving packages around in the package namespace is a bad
idea.  But maybe you can give me an answer to the question above to
convince me.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 15 14:57:32 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 14:57:38 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Mon, 15 Dec 2003 14:51:39 EST."
	<a06010203bc03be7149c2@[172.24.18.98]> 
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net>
	<a06010202bc03b13b3134@[172.24.18.98]>
	<200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net> 
	<a06010203bc03be7149c2@[172.24.18.98]> 
Message-ID: <200312151957.hBFJvWi18151@c-24-5-183-134.client.comcast.net>

> >BTW, eval() is also part of that full set, and exec is part of the
> >language.  These invoke the Python bytecode compiler.  (And Jim
> >Hugunin's IronPython is really slow on this, like 60x slower than
> >CPython; I think Jim reported that Jython was 20x slower on exec().)
> >
> >How do you plan to handle these?  What if the entire benchmark was
> >given as a triple quoted string that was passed to the exec statement?
> 
> We agreed no eval as part of this challenge, since the point was the 
> speed of the interpreter engine not of the compiler. I don't see that 
> on the list of challenge points you posted earlier, though, so it was 
> either omitted inadvertently or I got snookered. The former, I expect.

Fair enough.  "No exec or eval" is now officially part of the
challenge.

You know, I *want* you to win, at least if you can win by a great big
margin.  Because then we could switch to Parrot to make Python
faster.  I just very much doubt that you'll be able to bat CPython.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Mon Dec 15 15:21:16 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 15 15:21:22 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <a06010202bc03b13b3134@[172.24.18.98]>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEMPHMAB.tim.one@comcast.net>

[Tim]
>> "Not that hard" depends on your background.  The idea is to bury Dan
>> under esoteric algorithms from a huge number of obscure specialties
>> <wink>.

[Dan Sugalski]
> Heh. You're expecting me to do far too much work. I plan on tackling
> bytecode optimization only if it turns out that a straightforward
> transform from python bytecode to parrot bytecode is isn't
> sufficiently faster than CPython. Funky algorithms expressed entirely
> in python bytecode won't make much of a difference...

There is a serious point buried in this.  This is how Python spells
small-int multiplication in the bytecode:

    BINARY_MULTIPLY

And this is how it spells unbounded-int Karatsuba multiplication in the
bytecode:

    BINARY_MULTIPLY

Same thing, and floating multiply, complex multiply, catenating N copies of
a string, and arbitrary user-defined Python code overloading infix "*" for a
class are spelled BINARY_MULTIPLY in the bytecode too.  There's simply
nothing in the bytecode to distinguish these cases.

It wouldn't be hard for Guido to construct a vanilla-looking Python program
that tickles some of the extreme-win special cases in the type-dependent
implementations of the operators.  That wouldn't be fair, so I'll hit him if
he does <wink>, but in some smaller senses it can't be avoided.  For
example, all idiomatic Python programs use dicts (hashes), and the dict
implementation dynamically keeps track of whether any non-string key has
been added (Python dicts can be indexed by lots of things, not just
strings).  So long as it hasn't, special methods specialized to string keys
are used; but as soon as, e.g., a float key is added, a Python dict switches
to using slower, more-general methods.  None of that is reflected in the
bytecode, of course.  There's a lot of stuff like that.

pystone tickles little of that nature, because pystone is such an *a*typical
Python program.  So, regardless of what the official benchmark turns out to
be, I'll probably be more interested in your pystone result <wink>.


From barry at python.org  Mon Dec 15 15:23:15 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 15:23:23 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
Message-ID: <1071519794.2808.24.camel@anthem>

On Mon, 2003-12-15 at 14:34, Ken Manheimer wrote:

> Currently, we can just plop the things in place, and use them.
> Without relative imports, we would have to be editing the imports in
> the packages in each place we use it.

I've heard this before, but I still don't get it, so I'd like to tease
this one out more.  What does "plop the things in place" mean?

Zope 2 has a few standard directories it adds to its PYTHONPATH.  If I
distribute a 3rd party package Zope wants to use, and that package uses
absolute imports, you can just drop the package in one of those
directories and it'll just work.

Maybe you mean that you want to drop the mythical 3rd party package in
somewhere other than a directory on Zope's PYTHONPATH and have it just
work.  I'd say that's too perverse of a use-case for Python to worry
about.

One problem with banning relative imports is that Python doesn't allow a
top-level package name to live in more than one place on PYTHONPATH.  It
would be useful for Python to support such things out of the box, say if
we wanted to adopt a Java style naming convention for package
organizations.  

Jim uses the term "pure container package" to describe top-level
packages that contain no real content in the package except
subpackages.  "zope" and "zodb" are examples of this in Zope 3.  IWBNI
Python had better/easier support for weaving together pure container
package contents where subdirectories live in different locations on the
file system.

-Barry



From tim.one at comcast.net  Mon Dec 15 15:28:26 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 15 15:28:32 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <200312151957.hBFJvWi18151@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGENBHMAB.tim.one@comcast.net>

[Guido, to Dan Sugalski]
> ...
> You know, I *want* you to win, at least if you can win by a great big
> margin.  Because then we could switch to Parrot to make Python
> faster.  I just very much doubt that you'll be able to bat CPython.

Note that Fredrik boosted his pystone result from 2267 under CPython, to
58038 under his Pytte1 "small Python-like language" interpreter:

    http://effbot.org/zone/pyttestone.htm

The way to prevent this is to make the benchmark exercise lots of Python's
features.  I suggest at least one instance of each of the 7016 ways to call
a function <wink>.


From fred at zope.com  Mon Dec 15 15:51:27 2003
From: fred at zope.com (Fred L. Drake, Jr.)
Date: Mon Dec 15 15:51:35 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <1071519794.2808.24.camel@anthem>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<1071519794.2808.24.camel@anthem>
Message-ID: <16350.7887.116951.920720@sftp.fdrake.net>


Barry Warsaw writes:
 > Jim uses the term "pure container package" to describe top-level
 > packages that contain no real content in the package except
 > subpackages.  "zope" and "zodb" are examples of this in Zope 3.  IWBNI
 > Python had better/easier support for weaving together pure container
 > package contents where subdirectories live in different locations on the
 > file system.

+1

The .pth files processed by site.py support two interesting kinds of
lines:

- directories that should be added to sys.path, and

- import statements

The later alone is interesting enough, since it allows arbitrary
initialization code to run when the import is executed.  This still
happens before all the .pth files have been processed; I suspect we
really want the container package to be a little more flexible than
that, at least allowing their __path__ to be computed after all .pth
files have been processed.

Perhaps it would be nice to have a hook (probably in sys) that gets
called after site/sitecustomize are done?  A "pure container package"
could use a .pth file using "import mypackage", and the package's
__init__.py would register a hook function in sys that searches
sys.path for whatever directories should be used in
mypackage.__path__.  This also avoids the code being too specific to
one way of thinking about packages.


  -Fred

-- 
Fred L. Drake, Jr.  <fred at zope.com>
PythonLabs at Zope Corporation

From lists at webcrunchers.com  Mon Dec 15 16:05:51 2003
From: lists at webcrunchers.com (JD)
Date: Mon Dec 15 16:06:09 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <16349.52054.372100.462135@montanaro.dyndns.org>
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
	<AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
	<16349.52054.372100.462135@montanaro.dyndns.org>
Message-ID: <76BB1B8C-2F42-11D8-BFA1-0030656C6B9E@webcrunchers.com>


On Dec 15, 2003, at 6:55 AM, Skip Montanaro wrote:

>
>     John> Then, where should I submit my core dump (if anywhere?)?
>
> Submit bug reports here:
>
>     http://sourceforge.net/tracker/?group_id=5470&atid=105470

Thanx,   OK,   I logged in,   and got the main page which confirmed my 
login was good.
But I cannot find any links that allow me to submit bug reports.   
Where is the link?
I cannot find it.

On the left part of the screen,  I get this?    Does  one of these 
links go to the bug
reporting page?

?SF.net Resources

???Site Docs
???Site Status
???Site Map
???SF.net Supporters
???Compile Farm
???Foundries
???Project Help Wanted
???New Releases
???Get Support

There are two columns in the main page....    One titled...

My Assigned Tracker Items  and another titled My Tasks

I see no place on this site where there is a link I can use to report 
any bugs...   If it don't say "Bug reporting"
or anything similar,  then I need to know the name of the link,  
because it clearly isn't very obvious.

Another thing also....  if I click on Python Bugs link in the left 
column,  it goes into a "Browse Bugs" section,
but then there is yet another "Please log in" in red.   OK,  so I log 
in again.....   but when I do,   I don't
see the "Submit New" button,  but I do see a "Submit New" link above 
the buttons.    I DO have Cookies
enabled of course,  but I'm using Safari browser.   By the way,  I DO 
click on SSL.   I just tried it with IE
and get the same results....

I think I finally submitted it....  but can I go back and add some 
additional info?   If so,  how?

Further details on the Python core dump:

It core dumps (segmentation fault) when I check the database for a 
certain
key. The offending statement is: x=database.has_key(keystring). database
is open and otherwise normal.  The database is "gdbm-1.8.3". The 
offending
key is "3012a3ec5c796329fee7c242d4df00d6".

John
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/enriched
Size: 2601 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20031215/4aba71e1/attachment.bin
From skip at pobox.com  Mon Dec 15 16:15:42 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 15 16:15:44 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <76BB1B8C-2F42-11D8-BFA1-0030656C6B9E@webcrunchers.com>
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
	<AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
	<16349.52054.372100.462135@montanaro.dyndns.org>
	<76BB1B8C-2F42-11D8-BFA1-0030656C6B9E@webcrunchers.com>
Message-ID: <16350.9342.865204.274114@montanaro.dyndns.org>


    >> Submit bug reports here:
    >> 
    >> http://sourceforge.net/tracker/?group_id=5470&atid=105470

    JD> Thanx, OK, I logged in, and got the main page which confirmed my
    JD> login was good.  But I cannot find any links that allow me to submit
    JD> bug reports.  Where is the link?  I cannot find it.

Pop up that page (while logged in) and search for "Submit New".

    JD> There are two columns in the main page....    One titled...

    JD> My Assigned Tracker Items  and another titled My Tasks

Wrong page.  Once you log in, SF takes you to your "my sf.net" page.  After
logging in, pop back to

    http://sourceforge.net/projects/python

then poke the "Bugs" link, which will take you to the bug tracker (the first
URL above).

    JD> Another thing also....  if I click on Python Bugs link in the left
    JD> column, it goes into a "Browse Bugs" section, but then there is yet
    JD> another "Please log in" in red.

Sounds like you have cookies disabled.  To use SourceForge you need to run
with cookie support turned on.

Skip

From jepler at unpythonic.net  Mon Dec 15 16:34:03 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec 15 16:34:07 2003
Subject: [Python-Dev] Christmas Wishlist
In-Reply-To: <004301c3c341$48053fe0$e841fea9@oemcomputer>
References: <20031215030347.GC27067@unpythonic.net>
	<004301c3c341$48053fe0$e841fea9@oemcomputer>
Message-ID: <20031215213354.GH17242@unpythonic.net>

On Mon, Dec 15, 2003 at 02:26:03PM -0500, Raymond Hettinger wrote:
> Ordinarily, I would say that 3% is a reasonable price to pay for this
> functionality but there is that pesky pie throwing event coming up.

Maybe we can add it in right after the benchmarks get run?

Jeff

From jeremy at alum.mit.edu  Mon Dec 15 16:36:35 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Mon Dec 15 16:39:25 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031215191023.GB26055@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
Message-ID: <1071524194.15985.660.camel@localhost.localdomain>

One kind of problem is that newer Python features were designed without
taking rexec into account.  It's possible for untrusted code to cause
the trusted interpreter to execute its code without restriction using
descriptors.  It would be really difficult to reconcile new-style
classes and rexec.  Perhaps a worthwhile project, but probably one
accomplished by design a new restriction mechanism that builds on some
of the basic mechanism of rexec.

Jeremy



From greg at cosc.canterbury.ac.nz  Mon Dec 15 17:12:39 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Mon Dec 15 17:12:44 2003
Subject: [Python-Dev] Changing semantics of issubclass when
	accepting	atuple
In-Reply-To: <200312150559.hBF5xw616334@c-24-5-183-134.client.comcast.net>
Message-ID: <200312152212.hBFMCdh06520@oma.cosc.canterbury.ac.nz>

Guido:

> while an instance of my_favourite_types would indeed be an instance of
> float, 3.14 is not an instance of my_favourite_types!

You're right. BrainFartError. :-(

Hmmm... If there were an __isinstance__ protocol for
types, a suitable object could be constructed...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From janssen at parc.com  Mon Dec 15 17:49:50 2003
From: janssen at parc.com (Bill Janssen)
Date: Mon Dec 15 17:50:17 2003
Subject: [Python-Dev] Re: Christmas Wishlist 
In-Reply-To: Your message of "Mon, 15 Dec 2003 11:34:33 PST."
	<Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com> 
Message-ID: <03Dec15.144959pst."58611"@synergy1.parc.xerox.com>

Ken Manheimer writes:
> I'd say there's a legitimate need for
> relative imports, and we need some explicit gesture to disambiguate
> between a relative and absolute import, one way or the other.

I'd agree with Ken here.  I think the importance of being able to do
relative imports grows as well with the ability to compile Python into
foreign frameworks, like Jython or .NET.  I'd invent an ugly syntax
for relative imports, and let the default be absolute.

Bill

From barry at python.org  Mon Dec 15 17:53:41 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 15 17:53:48 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <03Dec15.144959pst."58611"@synergy1.parc.xerox.com>
References: <03Dec15.144959pst."58611"@synergy1.parc.xerox.com>
Message-ID: <1071528820.14604.13.camel@anthem>

On Mon, 2003-12-15 at 17:49, Bill Janssen wrote:

> I'd agree with Ken here.  I think the importance of being able to do
> relative imports grows as well with the ability to compile Python into
> foreign frameworks, like Jython or .NET.

Why?  I'd really like to see some concrete use cases for relative
imports, other than just saving some typing.

-Barry



From aahz at pythoncraft.com  Mon Dec 15 18:18:49 2003
From: aahz at pythoncraft.com (Aahz)
Date: Mon Dec 15 18:18:56 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <16350.9342.865204.274114@montanaro.dyndns.org>
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
	<AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
	<16349.52054.372100.462135@montanaro.dyndns.org>
	<76BB1B8C-2F42-11D8-BFA1-0030656C6B9E@webcrunchers.com>
	<16350.9342.865204.274114@montanaro.dyndns.org>
Message-ID: <20031215231849.GB17244@panix.com>

On Mon, Dec 15, 2003, Skip Montanaro wrote:
>
> Sounds like you have cookies disabled.  To use SourceForge you need to run
> with cookie support turned on.

...and you can't use Lynx, either.  :-(
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From lkcl at lkcl.net  Mon Dec 15 18:36:39 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Mon Dec 15 18:36:34 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <1071524194.15985.660.camel@localhost.localdomain>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
Message-ID: <20031215233639.GE26055@lkcl.net>

On Mon, Dec 15, 2003 at 04:36:35PM -0500, Jeremy Hylton wrote:
> One kind of problem is that newer Python features were designed without
> taking rexec into account.  It's possible for untrusted code to cause
> the trusted interpreter to execute its code without restriction using
> descriptors.  It would be really difficult to reconcile new-style
> classes and rexec.  Perhaps a worthwhile project, but probably one
> accomplished by design a new restriction mechanism that builds on some
> of the basic mechanism of rexec.
 
 okay.

 i think the only really sensible way forward is to begin from a
 sound basis - one that is going to be a big job to add retrospectively,
 but a simple beginning can be made.

 proposal: how about building ACLs into the python codebase?

 the basic operations are read, write, execute and modify_acl

 the advanced operations are "applies-to-sub-objects" and
 "automatically-added-to-created-sub-objects".

 it should be easy to add more operations, both default ones AND
 user-defined ones.

 [the advanced operations are based on the new NT 5.0 (aka
  windows 2000) enhancements to the original VMS security
  model back from dave cutler's time when he worked for DEC
  and then he and his team got poached by microsoft to do
  NT, yes it's a long story.]
 
 the basic primitives are __get_acl__(), __set_acl__() and
 __check_acl__(operation_type).

 the acl should consist of a dictionary of aces [access control entries]
 where the name is the function or module name, and the value is an ace
 object.

 the objects should contain a __get_ace__() function and
 a __set_ace__() function.  the ace should contain a permission
 (read, write, execute, modify_acl etc.) and an "ALLOW" or "DENY"
 qualifier.


 the algorithm for evaluating an acl has been worked out already,
 and implemented originally by matthew chapman of the samba team,
 so code under the GPL already exists [in an NT-like environment which
 is over-the-top].
 
 the basics of the evaluation algorithm are that you pass in who
 you are (in this case, that's the function or module name), what
 type of operation you want to perform (read, write, execute, or
 combination of these) and the ACL.

 the aces are evaluated, looking through the dictionary (actually it's
 going to have to be a list of tuples now that i think of it because
 the order is important, due to the ALLOW and DENY attributes),
 checking first that the ACE name matches the "who am i" and then
 checking what type of permission was requested against the ACE
 entries.

 the tricky bit is the "inheritance" permissions, which create
 "shadow" ACLs or, more specifically, they "throw shadows" down
 the sub-objects, adding or removing permissions from all
 sub-objects.

 in this way, it is possible to implement rexec by simply removing
 the write permissions at the top level as an "inherited" permission.


 an empty acl means "anything goes" (including that of "any acl may be
 added to this object").

 an empty acl also means that there is no performance penalty for having
 acl code in the python codebase.

 l.


From janssen at parc.com  Mon Dec 15 18:45:11 2003
From: janssen at parc.com (Bill Janssen)
Date: Mon Dec 15 18:45:25 2003
Subject: [Python-Dev] Re: Christmas Wishlist 
In-Reply-To: Your message of "Mon, 15 Dec 2003 14:53:41 PST."
	<1071528820.14604.13.camel@anthem> 
Message-ID: <03Dec15.154514pst."58611"@synergy1.parc.xerox.com>

> Why?  I'd really like to see some concrete use cases for relative
> imports, other than just saving some typing.

Suppose you've developed a Python top-level application, composed of a
package 'foo' containing a number of submodules and subpackages.
You'd now like to add this to a Jython application as a library, part
of a larger application.  It's nice to be able to just add a
higher-level package structure, com.mycompany.myapp, to the Python
package, to fit in with the style of your application.  Yet you don't
want to edit the Python code to 'import
com.mycompany.myapp.foo.submodule' everywhere; you'd like 'import
submodule' or 'import foo.submodule' to continue working.  Often the
code has to work in both the stand-alone and the federated case.

This is just a special case of the federation problem that we've seen
for years in distributed computing; previously top-level names get
federated when new higher-level constructs are formed.  Ken mentioned
enterprise programming; it's another case of this.  Two companies
merge and start merging their code.  It's much better to be able to
write (in the new higher-level module bletch.py)

     import foo
     import bar

and have the subpackages under foo and bar continue to work without
rewrites, than have to go through all the code under foo and bar and
change it to know about the new higher-level bletch namespace.

Bill

From tismer at tismer.com  Mon Dec 15 21:25:57 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon Dec 15 21:26:09 2003
Subject: [Python-Dev] Does anybody really use frame->f_tstate ?
Message-ID: <3FDE6D35.3090100@tismer.com>

Hi colleagues,

this is my second attempt to get rid of the f_tstate field
in frames. I need to find every user of this field.

What am I talking about?
Well, Python always has a thread state variable which
is a unique handle to the current thread. This variable
is accessed in many places, and there exists a fast macro
to get at it.
Every executing Python frame also gets a copy on creation.
In some cases, this frame->f_tstate field is used,
in other cases the current tstate variable is used.

If this sounds foreign to you, please stop reading here.

-------------------------------------------------------------

I would like to get rid of the frame->f_tstate, and I'm trying
to find out if there is a need for it. I don't need it,
for Stackless, it is the opposite, it disturbs.

There was a small thread about this in June this year, where
Guido convinced me that it is possible to create a traceback
on a frame that comes from a different thread.

http://mail.python.org/pipermail/python-dev/2003-June/036254.html

Ok, this is in fact possible, although I don't think
anybody has a need for this.

My question to all extension writers is this:
If you use frame->f_tstate at all, do you use it just
because it is handy, or do you want to use it for
some other purpose?

One purpose could be that you really want to create a traceback
on a different than the current thread. I have never seen this,
but who knows, so that's why I'm asking the Python world.

In most cases, a traceback will be created on a frame
that is currently processd or just has been processed.
Accessing a frame of a different thread that is being processed
might make sense for special debugger cases.

My proposal is
--------------

a) change semantics of PytraceBack_Here to use the current tstate.

b) if such a special purpose exists, create a new function for it.

c) if urgent, different needs exist to keep f_tstate,
    then let's forget about this proposal.

Especially for Stackless, I'd be keen of getting rid of this.

thanks for input -- chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/




From tismer at tismer.com  Mon Dec 15 21:34:58 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon Dec 15 21:34:49 2003
Subject: [Python-Dev] instancemethod_getattro seems to be partially wrong
In-Reply-To: <200311240126.hAO1Q9I01704@c-24-5-183-134.client.comcast.net>
References: <3FB99A6E.5070000@tismer.com>
	<200311180604.hAI64Ku02457@c-24-5-183-134.client.comcast.net>
	<3FBAC6E4.2020202@tismer.com>
	<200311190133.hAJ1X4J13394@c-24-5-183-134.client.comcast.net>
	<3FBACC4F.7090404@tismer.com>
	<200311190507.hAJ575213691@c-24-5-183-134.client.comcast.net>
	<3FBC3296.1090004@tismer.com>
	<200311200618.hAK6Ikv23729@c-24-5-183-134.client.comcast.net>
	<3FBD7C45.3020607@tismer.com>
	<200311222338.hAMNcnG03504@c-24-5-183-134.client.comcast.net>
	<3FC046BC.3030500@tismer.com>
	<200311240126.hAO1Q9I01704@c-24-5-183-134.client.comcast.net>
Message-ID: <3FDE6F52.5090104@tismer.com>

Guido van Rossum wrote:

Hi Guido, this is a little follow-up. I didn't forget about it!

...

>>So I'm just saying that pickle.py in wrong in just one place:
>>
>>                 reduce = getattr(obj, "__reduce__", None)
>>                 if reduce:
>>                     rv = reduce()
>>
>>should be:
>>
>>                 reduce = getattr(type(obj), "__reduce__", None)
>>                 if reduce:
>>                     rv = reduce(obj)
>>"""
> 
> 
> Right.  (That's what I was trying to say, too. :-)
> 
> 
>>An almost trivial change, although I also had to change copy.py,
>>and overall I was unhappy since this extends my patch set to more
>>than replacing python2x.dll, but I hope this will become an
>>official patch and back-patch.
> 
> 
> Give it to me baby. (On SF. :-)

So here the bad news:
After changing the C code to use the new style, and also changing
all occourances in Python code, I used this on Zope, and
ExtensionClass did not play well, it cannot stand this protocol
unchanged. Zope won't start.

Therefore, I reverted my changes and now I'm waiting for the
day where ExtensionClass is no loger used.
(or should I patch *that* complicated stuff as well?)

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From guido at python.org  Mon Dec 15 23:25:52 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 15 23:26:06 2003
Subject: [Python-Dev] instancemethod_getattro seems to be partially wrong
In-Reply-To: Your message of "Tue, 16 Dec 2003 03:34:58 +0100."
	<3FDE6F52.5090104@tismer.com> 
References: <3FB99A6E.5070000@tismer.com>
	<200311180604.hAI64Ku02457@c-24-5-183-134.client.comcast.net>
	<3FBAC6E4.2020202@tismer.com>
	<200311190133.hAJ1X4J13394@c-24-5-183-134.client.comcast.net>
	<3FBACC4F.7090404@tismer.com>
	<200311190507.hAJ575213691@c-24-5-183-134.client.comcast.net>
	<3FBC3296.1090004@tismer.com>
	<200311200618.hAK6Ikv23729@c-24-5-183-134.client.comcast.net>
	<3FBD7C45.3020607@tismer.com>
	<200311222338.hAMNcnG03504@c-24-5-183-134.client.comcast.net>
	<3FC046BC.3030500@tismer.com>
	<200311240126.hAO1Q9I01704@c-24-5-183-134.client.comcast.net> 
	<3FDE6F52.5090104@tismer.com> 
Message-ID: <200312160425.hBG4PqN18886@c-24-5-183-134.client.comcast.net>

> >>So I'm just saying that pickle.py in wrong in just one place:
> >>
> >>                 reduce = getattr(obj, "__reduce__", None)
> >>                 if reduce:
> >>                     rv = reduce()
> >>
> >>should be:
> >>
> >>                 reduce = getattr(type(obj), "__reduce__", None)
> >>                 if reduce:
> >>                     rv = reduce(obj)
> >>"""
> > 
> > 
> > Right.  (That's what I was trying to say, too. :-)
> > 
> >>An almost trivial change, although I also had to change copy.py,
> >>and overall I was unhappy since this extends my patch set to more
> >>than replacing python2x.dll, but I hope this will become an
> >>official patch and back-patch.
> > 
> > Give it to me baby. (On SF. :-)
> 
> So here the bad news:
> After changing the C code to use the new style, and also changing
> all occourances in Python code, I used this on Zope, and
> ExtensionClass did not play well, it cannot stand this protocol
> unchanged. Zope won't start.
> 
> Therefore, I reverted my changes and now I'm waiting for the
> day where ExtensionClass is no loger used.
> (or should I patch *that* complicated stuff as well?)

No, just wait until it dies.  I think Jim wants it to die to; the
latest Zope version requires Python 2.3.2 but it will take time to
retire it, and old Zope versions are persistent.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From DavidA at ActiveState.com  Tue Dec 16 01:49:51 2003
From: DavidA at ActiveState.com (David Ascher)
Date: Tue Dec 16 01:53:45 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <03Dec15.154514pst."58611"@synergy1.parc.xerox.com>
References: <03Dec15.154514pst."58611"@synergy1.parc.xerox.com>
Message-ID: <3FDEAB0F.6010605@ActiveState.com>

Bill Janssen wrote:

> It's much better to be able to
> write (in the new higher-level module bletch.py)
> 
>      import foo
>      import bar
> 
> and have the subpackages under foo and bar continue to work without
> rewrites, than have to go through all the code under foo and bar and
> change it to know about the new higher-level bletch namespace.

Right.  Or as someone said:

Namespaces are one honking great idea -- let's do more of those!



From tjreedy at udel.edu  Tue Dec 16 02:00:10 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Dec 16 02:00:17 2003
Subject: [Python-Dev] Re: Pie-thon benchmarks
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net><a06010202bc03b13b3134@[172.24.18.98]><200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net>
	<a06010203bc03be7149c2@[172.24.18.98]>
	<200312151957.hBFJvWi18151@c-24-5-183-134.client.comcast.net>
Message-ID: <brmahq$27a$1@sea.gmane.org>

> You know, I *want* you to win, at least if you
can win by a great big
> margin.  Because then we could switch to Parrot
to make Python
> faster.  I just very much doubt that you'll be
able to bat CPython.

For this to be thinkable, and for the test to be
fair, Parrot must be at least as semantically
'broad' as CPython (ie, handle every possible
meaning of every bytecode).  So I would include at
least the syntax and standard lib test files (-
exec and eval tests) as part of the benchmark.
This also gets to the 'provably correct' aspect of
the rules.

Terry J. Reedy




From tjreedy at udel.edu  Tue Dec 16 02:13:29 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Dec 16 02:13:35 2003
Subject: [Python-Dev] Re: rexec.py unuseable
References: <20031215173156.GC25203@lkcl.net>
Message-ID: <brmbap$39c$1@sea.gmane.org>


"Luke Kenneth Casson Leighton" <lkcl@lkcl.net>
wrote in message

If you search the pydev summaries (available on
the site) for perhaps the past 15 months for
'rexec', you should find several threads on why it
was dropped and how it might be replaced.

TJR




From press at webdevmagazine.co.uk  Tue Dec 16 04:02:41 2003
From: press at webdevmagazine.co.uk (Mr.Bogomil Shopov)
Date: Tue Dec 16 02:59:19 2003
Subject: [Python-Dev] Web Development Conference
Message-ID: <3FDECA31.4030400@webdevmagazine.co.uk>

 Hello, 
 I am coordinator of  a Web Development Conference that will be held
 in March 2003 in Sofia, Bulgaria. We would like to present to the
 Conference Python language- new directions and some programmer's
 tricks. Is there any Bulgarian member in your community or what are
 your conditions to arrive for this Conf here?

Regards, 
Bogomil Shopov
WebDevMagazine

phone: +359 (0) 98 505914

<?Web Dev?> Conference 2004 Sofia
http://webdevmagazine.co.uk/conf/


CONFIDENTIALITY NOTICE
This Email is confidential and may also be privileged.
If you are not the intended recipient, please notify the sender IMMEDIATELY; 
you should not copy the email or use it for any purpose or disclose its contents to any other person.
privacy report: privacy@spisanie.com



From mwh at python.net  Tue Dec 16 05:03:00 2003
From: mwh at python.net (Michael Hudson)
Date: Tue Dec 16 05:03:04 2003
Subject: [Python-Dev] Please tell me where I can submit bug reports....
In-Reply-To: <20031215231849.GB17244@panix.com> (aahz@pythoncraft.com's
	message of "Mon, 15 Dec 2003 18:18:49 -0500")
References: <LNBBLJKPBEHFEDALKOLCCEEOHMAB.tim.one@comcast.net>
	<AE94D680-2ED3-11D8-BFA1-0030656C6B9E@webcrunchers.com>
	<16349.52054.372100.462135@montanaro.dyndns.org>
	<76BB1B8C-2F42-11D8-BFA1-0030656C6B9E@webcrunchers.com>
	<16350.9342.865204.274114@montanaro.dyndns.org>
	<20031215231849.GB17244@panix.com>
Message-ID: <2mfzflhygb.fsf@starship.python.net>

Aahz <aahz@pythoncraft.com> writes:

> On Mon, Dec 15, 2003, Skip Montanaro wrote:
>>
>> Sounds like you have cookies disabled.  To use SourceForge you need to run
>> with cookie support turned on.
>
> ...and you can't use Lynx, either.  :-(

It seems that you can, so long as you "attach" /dev/null to the bug
report each time you do anything...

Cheers,
mwh

-- 
  surely, somewhere, somehow, in the history of computing, at least
  one manual has been written that you could at least remotely
  attempt to consider possibly glancing at.              -- Adam Rixey

From ncoghlan at iinet.net.au  Tue Dec 16 05:32:16 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Dec 16 05:32:27 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151842.hBFIg0K17763@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com>
	<200312151842.hBFIg0K17763@c-24-5-183-134.client.comcast.net>
Message-ID: <3FDEDF30.1050904@iinet.net.au>

Guido van Rossum wrote:

>>I don't understand the desire to totally prevent relative imports.
>>It would be quite useful in avoiding growing module-naming problems,
>>where you have to be careful about shadowing a global module with
>>one in your package.
> 
> I don't see the problem, or I misuderstand what "it" refers to; it
> seems you have this backwards if it refers to relative imports.  Say
> my package P defines a submodule sys.  If we require absolute imports,
> there is no ambiguity: the submodule sys must be imported as P.sys
> while the standard sys module can be imported as simply sys.

If absolute imports were to be the only type allowed, then it would seem 
that the only possible location for naming conflicts is in the _first_ 
element.

So if I wanted to use two different third party modules, both of which 
have unfortunately chosen the same name for the top-level package, the 
only way to let them co-exist is to redo _all_ of the imports in one or 
the other of them.

Whereas, if relative pathing is possible, I believe that all I have to 
do is push them one level down in the package heirarchy (using distinct 
names that I invent), and neither of them ever even knows about the 
other's existence. I can get at both of them unambiguously, by using my 
new top=level names, and neither package even knows that it is no longer 
starting at the top of the import heirarchy.

Or is there some other solution being proposed to this problem, and I 
just haven't understood it?

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From mcherm at mcherm.com  Tue Dec 16 08:37:34 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Dec 16 08:37:54 2003
Subject: [Python-Dev] RE: rexec.py unuseable
Message-ID: <1071581854.3fdf0a9eba3ea@mcherm.com>

Luke writes:
>  i think the only really sensible way forward is to begin from a
> sound basis...

I agree completely.

>  proposal: how about building ACLs into the python codebase?

Actually, I rather prefer the approach that has been mentioned before
of trying to use capabilities. See, for instance, the threads on
Capabilities found here: 
http://mail.python.org/pipermail/python-dev/2003-March/thread.html#33854
Of course, that's not trivial to add either, since some way would
need to be provided for disabling Python's powerful introspection.

> the algorithm for evaluating an acl has been worked out already,
> and implemented originally by matthew chapman of the samba team,
> so code under the GPL already exists [in an NT-like environment which
> is over-the-top].

Unfortunately, code under the GPL is of no use at all for Python which
is released with a more flexible (but GPL compatible) liscense. And
I'm not quite sure what you mean by an "NT-like environment" being
"over-the-top".

> an empty acl also means that there is no performance penalty for having
> acl code in the python codebase.

Sorry, but I simply don't believe this. Really. Think about it a bit,
and if you still think you're right, I'll provide some reasons, but
I suspect you'll realize that it's simply not true.

I *DO* think though, that some amount of slow-down *IS* acceptable
(pie or no pie <wink>) if it provided powerful new capabilities like
*reliable* restricted execution environments.

-- Michael Chermside


From lkcl at lkcl.net  Tue Dec 16 09:12:07 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 09:12:02 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <1071581854.3fdf0a9eba3ea@mcherm.com>
References: <1071581854.3fdf0a9eba3ea@mcherm.com>
Message-ID: <20031216141207.GA17021@lkcl.net>

On Tue, Dec 16, 2003 at 05:37:34AM -0800, Michael Chermside wrote:
> Luke writes:
> >  i think the only really sensible way forward is to begin from a
> > sound basis...
> 
> I agree completely.
> 
> >  proposal: how about building ACLs into the python codebase?
> 
> Actually, I rather prefer the approach that has been mentioned before
> of trying to use capabilities. 

 capabilities, acls, schmapabilities, same thiiing :)

 a la "cambridge capability system" [a historic architecture that
 provided hardware-assisted restrictions on function calls!]


> See, for instance, the threads on
> Capabilities found here: 
> http://mail.python.org/pipermail/python-dev/2003-March/thread.html#33854
> Of course, that's not trivial to add either, since some way would
> need to be provided for disabling Python's powerful introspection.
 

> > the algorithm for evaluating an acl has been worked out already,
> > and implemented originally by matthew chapman of the samba team,
> > so code under the GPL already exists [in an NT-like environment which
> > is over-the-top].
> 
> Unfortunately, code under the GPL is of no use at all for Python which
> is released with a more flexible (but GPL compatible) liscense. And
> I'm not quite sure what you mean by an "NT-like environment" being
> "over-the-top".
 
 the algorithm is there, for people to examine.

 i wasn't recommending that people go for complete code reuse of what
 is in samba (even if licenses were compatible) because the code in
 samba is a _complete_ reimplementation of the NT acl system
 _including_ recreating the internals of ACLs and their on-the-wire
 representation.

 however, there is a less low-level library which uses ACLs as
 "blobs", and it is the implementation of the check security access
 function [name i forget] which is actually a _re_implementation
 of the exact NT equivalent function that i recommend people to
 examine.


> > an empty acl also means that there is no performance penalty for having
> > acl code in the python codebase.
> 
> Sorry, but I simply don't believe this. Really. Think about it a bit,
> and if you still think you're right, I'll provide some reasons, but
> I suspect you'll realize that it's simply not true.

 tiny, then.

 in function calling:

 if len(acl) == 0 or check_allowed_access(callee_function, acl,
		 EXECUTE_CAPABILITY):
 	proceed...

 and in file write, again:
 
 if len(acl) == 0 or check_allowed_access(callee_function, acl,
 	 	WRITE_CAPABILITY):
		proceed...

 yes, the len(acl) == 0 checks will stack up, but not terribly
 significantly IMO.

 if an acl _does_ exist, even if it's one that grants permission
 to all functions/modules the ability to perform _all_ operations,
 _then_ you get a performance hit as the check_allowed_access()
 function needs to "do work".


> I *DO* think though, that some amount of slow-down *IS* acceptable
> (pie or no pie <wink>) if it provided powerful new capabilities like
> *reliable* restricted execution environments.

 ack!

 btw yes you're right, it's better off called "capabilities"
 but the name CCLs - capability control lists - is a bit of
 a mouthful :)


 btw, as a first implementation, doing "recursive" capability
 permissions could be avoided, by simply "copying" permissions
 down the object tree.

 as can be imagined, this is rather wasteful of both time and
 resources, which is why someone at microsoft came up with
 the concept of "apply-to-sub-object" capabilities.

 but remember: if all your ACLs (sorry, CCLs) are 0-length, it's
 not a waste of either time _or_ resources.

 l.


From niemeyer at conectiva.com  Tue Dec 16 09:17:46 2003
From: niemeyer at conectiva.com (Gustavo Niemeyer)
Date: Tue Dec 16 09:20:04 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
Message-ID: <20031216141746.GA3145@burma.localdomain>

> But why do you have to give those packages a different full name?
> That's the question that I've never seen answered adequately.

I have done this many times, so let me try to describe at least one
legitimate usage case. 

A couple of weeks ago I wrote a software which needs a third
party package to work. OTOH, since it's a small package, I don't
want to force the user to go after the package, even because I'd
have to ensure that my code always work with the available version
of that package.

Thanks to the relative module importing mechanism, solving that is no
harder than copying the third party package into my own package
namespace.

This idea could probably be expressed in some other way, hacking
sys.path or whatever, but I belive this is a fairly common pattern,
and I vote for introducing a scheme to differ between local/global
importing which would not break the current flexibility.

-- 
Gustavo Niemeyer
http://niemeyer.net

From mcherm at mcherm.com  Tue Dec 16 10:13:21 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Tue Dec 16 10:13:41 2003
Subject: [Python-Dev] RE: rexec.py unuseable
Message-ID: <1071587601.3fdf2111afc59@mcherm.com>

I (Michael) wrote:
    [Speaking of how to provide restricted execution in Python]
> Actually, I rather prefer the approach that has been mentioned before
> of trying to use capabilities. 
> See, for instance, the threads on
> Capabilities found here: 
> http://mail.python.org/pipermail/python-dev/2003-March/thread.html#33854

Luke replied:
> capabilities, acls, schmapabilities, same thiiing :)

No... they're not. Read the thread I mentioned above, or read this, 
and some of the other documentation for the language E:

  http://www.erights.org/elib/capability/ode/ode-capabilities.html


Later Luke writes:
> btw yes you're right, it's better off called "capabilities"
> but the name CCLs - capability control lists - is a bit of
> a mouthful :)

Again, I mean something different, NOT CCLs, but capabilities.

Hmm... how about a quick summary from me (but I'm not an expert).

Suppose that had some code which had an object representing a
directory, with a method "open(filename, mode='r')" that opened 
files in the directory. Given this object, you could imagine 
constructing new objects with more limited capabilities. For 
instance, you might create a readOnlyDirectory object which had
a method "open(filename)" that didn't allow specifying the mode
as anything but 'r'. Or you might open a file and then pass a
file object with "read()", "write()", "seek()", and other such
methods, which would only access that file.

So _IF_ the only way to access files were through this object (and
that's a BIG if), then you could imagine a world where HAVING and
object was equivalent to being able to do something. If a bit of
code had access to a read-only-file object then it could read that
file, but couldn't write to it, or do anything else with the file 
system unless it ALSO had access to some OTHER objects. That's
capabilities... and it would work for most kinds of restricted
resources, not just the file system. The key idea is that HAVING
a pointer to the object is equivalent to having the permission to
USE that object, and whatever it provides access to.

There are several ways to "escape" out of this system. One way is
to access some global ability... for instance, you might use the
"open()" function from __builtins__.  For capabilities to work in
Python access to "dangerous" globals would need to be restricted.
Another way is to just create an object out of nowhere, or forge
a pointer to an object. Fortunately, in Python these are already
impossible... pure Python code cannot forge references or create
objects without access to the type or class. (C extensions can
do anything and are unsafe.)

Another way is to access the more powerful object that "empowers"
a less powerful one... perhaps using the (supposed to be private)
_directory field in the readOnlyfile object. So capabilities in
Python are impossible without some sort of private data for
objects (this is not a particularly onerous requirement). Yet
*another* approach would be to use introspection... for instance,
in Python today, given any file object, the class itself ("file")
can be obtained via introspection, and then used to create other
file objects. Using capabilities in Python would require that
"restricted" objects have some more limited form of introspection...
perhaps one could only obtain a "fake" class object which had
a __equals__ method that simulated being the real class, but
which had no __init__ so it couldn't be used to create new
instances.

At any rate, you get the idea. Capabilities are possible in Python
only if some sort of "restricted mode" is created, which restricts
access to some built-in abilities and which creates "restricted"
objects with some private data and limited introspection. But IF
you had these things (and they're NOT trivial), then capabilities
may be a conceptually more elegant approach than ACLs, lleading to
more elegant programs.

-- Michael Chermside


From tjreedy at udel.edu  Tue Dec 16 11:11:21 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Tue Dec 16 11:11:30 2003
Subject: [Python-Dev] Re: Web Development Conference
References: <3FDECA31.4030400@webdevmagazine.co.uk>
Message-ID: <brnar8$11l$1@sea.gmane.org>


"Mr.Bogomil Shopov" <press@webdevmagazine.co.uk>
wrote in message
news:3FDECA31.4030400@webdevmagazine.co.uk...
> Hello,
>  I am coordinator of  a Web Development
Conference that will be held
>  in March 2003 in Sofia, Bulgaria. We would like
to present to the

presumably 2004

>  Conference Python language- new directions and
some programmer's
>  tricks. Is there any Bulgarian member in your
community or what are
>  your conditions to arrive for this Conf here?

This 'Python development' list has a relatively
restricted readership.  If you do not get a
positive response here, try reposting on
comp.lang.python or equivalent mainling list.
Good luck with your conference.

TJR




From lkcl at lkcl.net  Tue Dec 16 11:16:03 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 11:16:11 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <1071587601.3fdf2111afc59@mcherm.com>
References: <1071587601.3fdf2111afc59@mcherm.com>
Message-ID: <20031216161603.GG17021@lkcl.net>

On Tue, Dec 16, 2003 at 07:13:21AM -0800, Michael Chermside wrote:

> I (Michael) wrote:
>     [Speaking of how to provide restricted execution in Python]
> > Actually, I rather prefer the approach that has been mentioned before
> > of trying to use capabilities. 
> > See, for instance, the threads on
> > Capabilities found here: 
> > http://mail.python.org/pipermail/python-dev/2003-March/thread.html#33854
 
 ta.

> Luke replied:
> > capabilities, acls, schmapabilities, same thiiing :)
> 
> No... they're not. Read the thread I mentioned above, or read this, 
> and some of the other documentation for the language E:
> 
>   http://www.erights.org/elib/capability/ode/ode-capabilities.html

 no offense intended: i'll read that later, i'm running out of time.

 without going into too many definitions, consider what i am advocating
 to be _like_ an access control list but instead to be a capabilities
 control list, instead.

 a "capabilities list", where it's a list ordered on a
 per-caller-function-name basis [with a special wildcard function name
 called "absolutely everything"]

> 
> Later Luke writes:
> > btw yes you're right, it's better off called "capabilities"
> > but the name CCLs - capability control lists - is a bit of
> > a mouthful :)
> 
> Again, I mean something different, NOT CCLs, but capabilities.

 anyone have a one-para summary of the difference between
 capabilities and access control lists?

 even if it's "access control lists are lists of capabilities"
 which i don't know if that's true.

 access control lists are typically made on a per-user basis,
 but what i am recommending in _this_ case is that a "user"
 be considered to be a *function*.

 so maybe i _do_ mean access control list and should stick to my
 guns, here :)

 so.

 is the difference between capabilities and access control lists
 simply that capabilities lists restrict caller-function rights
 and access control lists restrict user rights?



> Hmm... how about a quick summary from me (but I'm not an expert).
> 
> Suppose that had some code which had an object representing a
> directory, with a method "open(filename, mode='r')" that opened 
> files in the directory. Given this object, you could imagine 
> constructing new objects with more limited capabilities. For 
> instance, you might create a readOnlyDirectory object which had
> a method "open(filename)" that didn't allow specifying the mode
> as anything but 'r'. Or you might open a file and then pass a
> file object with "read()", "write()", "seek()", and other such
> methods, which would only access that file.

 [i'll have to read this in more depth later, i'm out of time, sorry]

> So _IF_ the only way to access files were through this object (and
> that's a BIG if), then you could imagine a world where HAVING and
> object was equivalent to being able to do something. If a bit of
> code had access to a read-only-file object then it could read that
> file, but couldn't write to it, or do anything else with the file 
> system unless it ALSO had access to some OTHER objects. That's
> capabilities... and it would work for most kinds of restricted
> resources, not just the file system. The key idea is that HAVING
> a pointer to the object is equivalent to having the permission to
> USE that object, and whatever it provides access to.

 [again, i'll have to read this in more depth later, i'm out of time,
 sorry]

> There are several ways to "escape" out of this system. One way is
> to access some global ability... for instance, you might use the
> "open()" function from __builtins__.  

 i would expect __builtins__.open() to have _its_ own mmm...
 capabilities list, and _if_ that list contained a permission
 for the restricted function to "execute" it, then _yes_ it
 would be allowed, but otherwise no, definitely not.

 therefore, the author of the [new] rexec.py module should simply
 be a matter of creating the right ACLs, and applying them.

 in the case of __builtins__ it could be a matter of just applying
 a capabilities list of one item:
 
 	[("all functions and modules", DENY, "write, apply-to-sub-objects")]

 and that would be _it_!
 
 all functions in __builtins__ would be dealt with, and restricted!

 of course, i don't believe it will be _quite_ that simple, but it
 might.

 

> For capabilities to work in
> Python access to "dangerous" globals would need to be restricted.

 simple: apply a capabilities list that denies dangerous actions, on
 all such "dangerous" globals.

 OH!

 one important "permission" is "change acl" of course, which must be "DENIED"!


> Another way is to just create an object out of nowhere, or forge
> a pointer to an object. Fortunately, in Python these are already
> impossible... pure Python code cannot forge references or create
> objects without access to the type or class. (C extensions can
> do anything and are unsafe.)

 that is why i mentioned about "create object" permissions
 [capabilities].

 if there is a permission/capability whereby the capabilities list of
 a parent object is "inherited" when a create object action is carried
 out, the problem you describe is alleviated.

 _at_ object create time (and object create itself could be a separate
 permission / capability, granted on a per-function and per-module
 basis like everything else), the capabilities are examined.

 what happens is that _if_ the "inherit capabilities" flag is set,
 then the newly created object receives a COPY of the parent object's
 capabilities list.

 ta-da, problem solved.

 this is a _normal_ bog-standard approach that is used in the windows
 nt security model, and has been for over twenty years, now, and if
 you count the VAX/VMS history as well, a lot longer than twenty years.


> Another way is to access the more powerful object that "empowers"
> a less powerful one... perhaps using the (supposed to be private)
> _directory field in the readOnlyfile object. So capabilities in
> Python are impossible without some sort of private data for
> objects (this is not a particularly onerous requirement). Yet
> *another* approach would be to use introspection... for instance,
> in Python today, given any file object, the class itself ("file")
> can be obtained via introspection, and then used to create other
> file objects. Using capabilities in Python would require that
> "restricted" objects have some more limited form of introspection...
> perhaps one could only obtain a "fake" class object which had
> a __equals__ method that simulated being the real class, but
> which had no __init__ so it couldn't be used to create new
> instances.

 i don't quite follow, even after reading the capabilities thread,
 what introspection is.

 but let me try to clarify so you can correct me if necessary:
 introspection is the ability to go via the __XXXXX__ functions
 etc. including the __class__ stuff, yes?

 well, if i follow you correctly, you simply put a capabilities
 list on all those things, or you put one on the entire object.

 or better yet, you put an "inherited" one on the class, such that
 _any_ object created will receive restricted capabilities.

 etc.



> At any rate, you get the idea. Capabilities are possible in Python
> only if some sort of "restricted mode" is created, which restricts
> access to some built-in abilities and which creates "restricted"
> objects with some private data and limited introspection. 

> But IF
> you had these things (and they're NOT trivial), then capabilities
> may be a conceptually more elegant approach than ACLs, lleading to
> more elegant programs.
 
  i believe that if you understand what i am suggesting, and i
  hope that filling in some examples above, i believe that what
  i am suggesting covers your concerns.

  the capabilities lists that i am recommending can be applied
  on a per-function and per-object basis and can be inherited.

  in this way, useful restrictions can be made to achieve the
  expected results.

  ... i didn't say, however, that it wouldn't require quite a lot
  of thought about _which_ functions to apply restricted capabilities
  lists to!

  starting, of course, with the "nothing goes" one, and repeatedly
  attempting to run code.

  l.

From guido at python.org  Tue Dec 16 11:21:21 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 11:21:27 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: Your message of "Sun, 14 Dec 2003 14:27:52 PST."
	<200312142227.hBEMRr915819@c-24-5-183-134.client.comcast.net> 
References: <a06010201bc023e3253de@[10.0.1.2]>
	<200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net>
	<a06010202bc025bc5dc5a@[10.0.1.2]> 
	<200312142227.hBEMRr915819@c-24-5-183-134.client.comcast.net> 
Message-ID: <200312161621.hBGGLLs19795@c-24-5-183-134.client.comcast.net>

Dan, I have one more clarification request.

I assume you will support import (of Python modules); that shouldn't
be a problem right?  The explicit ban on extensions and the insistence
of the full builtins would suggest that import is okay.  I'm not
assuming the standard library (beyond the builtins) is available, but
I *am* assuming I can split the benchmark into multiple files.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From raymond.hettinger at verizon.net  Tue Dec 16 12:30:46 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Dec 16 12:31:02 2003
Subject: [Python-Dev] Filename suffix default for command line
Message-ID: <003b01c3c3fa$57a6dee0$f1bd958d@oemcomputer>

Would there be any disadvantage to checking for filename.py whenever
filename is not found so that "super.py" could be invoked with either
"python super.py" or "python super"?
 
 
Raymond Hettinger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031216/2ff80142/attachment.html
From guido at python.org  Tue Dec 16 12:41:06 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 12:41:11 2003
Subject: [Python-Dev] Filename suffix default for command line
In-Reply-To: Your message of "Tue, 16 Dec 2003 12:30:46 EST."
	<003b01c3c3fa$57a6dee0$f1bd958d@oemcomputer> 
References: <003b01c3c3fa$57a6dee0$f1bd958d@oemcomputer> 
Message-ID: <200312161741.hBGHf6w20055@c-24-5-183-134.client.comcast.net>

> Would there be any disadvantage to checking for filename.py whenever
> filename is not found so that "super.py" could be invoked with either
> "python super.py" or "python super"?

It feels horribly DOS-like.  Why?  To save typing three characters?
Why not rename the file then?

Would "python super.py" also look for super.py.py if super.py didn't
exist?  What would the error message say?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From DavidA at ActiveState.com  Tue Dec 16 12:49:17 2003
From: DavidA at ActiveState.com (David Ascher)
Date: Tue Dec 16 12:52:24 2003
Subject: [Python-Dev] instancemethod_getattro seems to be partially wrong
In-Reply-To: <200312160425.hBG4PqN18886@c-24-5-183-134.client.comcast.net>
References: <3FB99A6E.5070000@tismer.com>	<200311180604.hAI64Ku02457@c-24-5-183-134.client.comcast.net>	<3FBAC6E4.2020202@tismer.com>	<200311190133.hAJ1X4J13394@c-24-5-183-134.client.comcast.net>	<3FBACC4F.7090404@tismer.com>	<200311190507.hAJ575213691@c-24-5-183-134.client.comcast.net>	<3FBC3296.1090004@tismer.com>	<200311200618.hAK6Ikv23729@c-24-5-183-134.client.comcast.net>	<3FBD7C45.3020607@tismer.com>	<200311222338.hAMNcnG03504@c-24-5-183-134.client.comcast.net>	<3FC046BC.3030500@tismer.com>	<200311240126.hAO1Q9I01704@c-24-5-183-134.client.comcast.net>
	<3FDE6F52.5090104@tismer.com>
	<200312160425.hBG4PqN18886@c-24-5-183-134.client.comcast.net>
Message-ID: <3FDF459D.30507@ActiveState.com>

Guido van Rossum wrote:

> the latest Zope version requires Python 2.3.2 but it will take time to
>
>retire it, and old Zope versions are persistent.
>  
>
I thought that was a *feature* of Zope and the whole ZODB thing. <wink/>

--da


From dan at sidhe.org  Tue Dec 16 13:12:13 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Tue Dec 16 13:12:23 2003
Subject: [Python-Dev] Pie-thon benchmarks
In-Reply-To: <200312161621.hBGGLLs19795@c-24-5-183-134.client.comcast.net>
References: <a06010201bc023e3253de@[10.0.1.2]>
	<200312141723.hBEHNvg15553@c-24-5-183-134.client.comcast.net>
	<a06010202bc025bc5dc5a@[10.0.1.2]>             
	<200312142227.hBEMRr915819@c-24-5-183-134.client.comcast.net>
	<200312161621.hBGGLLs19795@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010204bc04f9c3db02@[10.0.1.2]>

At 8:21 AM -0800 12/16/03, Guido van Rossum wrote:
>Dan, I have one more clarification request.
>
>I assume you will support import (of Python modules); that shouldn't
>be a problem right?  The explicit ban on extensions and the insistence
>of the full builtins would suggest that import is okay.  I'm not
>assuming the standard library (beyond the builtins) is available, but
>I *am* assuming I can split the benchmark into multiple files.

Oh, absolutely. Loading in external bytecode files are fine.
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Tue Dec 16 13:26:59 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 13:27:05 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Tue, 16 Dec 2003 12:17:46 -0200."
	<20031216141746.GA3145@burma.localdomain> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net> 
	<20031216141746.GA3145@burma.localdomain> 
Message-ID: <200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>

A lot people have presented a good case for relative imports.  Nobody
has argued to keep the status quo (where imports are ambiguous about
whether they are meant to be absolute or relative).  So I suggest that
in 2.4, we introduce the leading dot notation for relative import,
while still allowing relative import without a leading dot.  In
2.5 we can start warning about relative import without a leading dot
(although that will undoubtedly get complaints from folks who have
code that needs to work with 2.3 still).  In 3.0 we can retire
ambiguous import.

The use case for multiple dots should be obvious: inside a highly
structured package, modules inside one subpackage must have a way to
do relative import from another subpackage of the same parent package.

There is the remaining issue of what exactly the syntax would be.  I
propose to extend the from clause to allow one or more dots before the
dotted name, and to make the dotted name optional if at least one
leading dot is found.  I propose not to change from-less import.
Examples:

  from .foo import bar
  from .foo.bar import xxx
  from . import foobar as barfoo
  from ..foo.bar import *
  from ... import foobar, barfoo

Grammar change (see Grammar/Grammar):

dotted_name: NAME ('.' NAME)* | '.'+ [NAME ('.' NAME)*]

--Guido van Rossum (home page: http://www.python.org/~guido/)

From gerrit at nl.linux.org  Tue Dec 16 13:29:12 2003
From: gerrit at nl.linux.org (Gerrit Holl)
Date: Tue Dec 16 13:29:47 2003
Subject: [Python-Dev] Filename suffix default for command line
In-Reply-To: <003b01c3c3fa$57a6dee0$f1bd958d@oemcomputer>
References: <003b01c3c3fa$57a6dee0$f1bd958d@oemcomputer>
Message-ID: <20031216182912.GA4204@nl.linux.org>

Raymond Hettinger wrote:
> Would there be any disadvantage to checking for filename.py whenever
> filename is not found so that "super.py" could be invoked with either
> "python super.py" or "python super"?

How about super.pyc? What if both super.pyc and super.py exist but are
not equal in effect? I think the idea is not necesarry. I think most
people have <tab> configured to do commandline completion, so 'python
supe<tab>' would probably achieve the same effect with the same amount
of keystrokes. The disadvantage is, in my opinion, sideways: people get
used to it, and it is not a good habit, because it's called super.py,
not super. I don't like this proposal, I'd give it a -0 in case anyone
is interested.

yours,
Gerrit.

-- 
202. If any one strike the body of a man higher in rank than he, he
shall receive sixty blows with an ox-whip in public.
          -- 1780 BC, Hammurabi, Code of Law
-- 
Asperger's Syndrome - a personal approach:
	http://people.nl.linux.org/~gerrit/english/

From guido at python.org  Tue Dec 16 13:30:43 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 13:30:48 2003
Subject: [Python-Dev] Re: Pie-thon benchmarks
In-Reply-To: Your message of "Tue, 16 Dec 2003 02:00:10 EST."
	<brmahq$27a$1@sea.gmane.org> 
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net><a06010202bc03b13b3134@[172.24.18.98]><200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net>
	<a06010203bc03be7149c2@[172.24.18.98]>
	<200312151957.hBFJvWi18151@c-24-5-183-134.client.comcast.net> 
	<brmahq$27a$1@sea.gmane.org> 
Message-ID: <200312161830.hBGIUhE20354@c-24-5-183-134.client.comcast.net>

> > You know, I *want* you to win, at least if you can win by a great
> > big margin.  Because then we could switch to Parrot to make Python
> > faster.  I just very much doubt that you'll be able to beat
> > CPython.
> 
> For this to be thinkable, and for the test to be
> fair, Parrot must be at least as semantically
> 'broad' as CPython (ie, handle every possible
> meaning of every bytecode).  So I would include at
> least the syntax and standard lib test files (-
> exec and eval tests) as part of the benchmark.
> This also gets to the 'provably correct' aspect of
> the rules.

Right.  I don't want to include the entire test suite, but I do want
to make sure the benchmark visits most dark corners of the language:
metaclasses, dynamically changing operator overloading, and the like.
I trust that Dan can write a VM that executes integer operations
faster than anyone, but that's not the point of Python.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jacobs at penguin.theopalgroup.com  Tue Dec 16 13:35:14 2003
From: jacobs at penguin.theopalgroup.com (Kevin Jacobs)
Date: Tue Dec 16 13:35:25 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312161330001.7068-100000@penguin.theopalgroup.com>

On Tue, 16 Dec 2003, Guido van Rossum wrote:
> There is the remaining issue of what exactly the syntax would be.  I
> propose to extend the from clause to allow one or more dots before the
> dotted name, and to make the dotted name optional if at least one
> leading dot is found.  I propose not to change from-less import.
> Examples:
> 
>   from .foo import bar
>   from .foo.bar import xxx
>   from . import foobar as barfoo
>   from ..foo.bar import *
>   from ... import foobar, barfoo
> 
> Grammar change (see Grammar/Grammar):
> 
> dotted_name: NAME ('.' NAME)* | '.'+ [NAME ('.' NAME)*]

On a related and minor note, can we please expand the grammar to allow
from...import statements to continue to the next line after a trailing
comma?  For those of us with large, complex, and componentized projects, it
would be very nice.  e.g.:

from OPAL.Modules.Financial.Reports.COGS import generate, webOptions,
                                                normalize, row_ops,
                                                summary

instead of:

from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \
                                                normalize, row_ops,   \
                                                summary


This has been a minor nit that bothers just under the threshold of fixing it
myself.  ;)

Thanks,
-Kevin

-- 
--
Kevin Jacobs
The OPAL Group - Enterprise Systems Architect
Voice: (440) 871-6725 x 19         E-mail: jacobs@theopalgroup.com
Fax:   (440) 871-6722              WWW:    http://www.theopalgroup.com/


From lkcl at lkcl.net  Tue Dec 16 13:35:41 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 13:35:47 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <1071587601.3fdf2111afc59@mcherm.com>
References: <1071587601.3fdf2111afc59@mcherm.com>
Message-ID: <20031216183541.GH17021@lkcl.net>

On Tue, Dec 16, 2003 at 07:13:21AM -0800, Michael Chermside wrote:

> Suppose that had some code which had an object representing a
> directory, with a method "open(filename, mode='r')" that opened 
> files in the directory. Given this object, you could imagine 
> constructing new objects with more limited capabilities. 

 yep.  in a very simple manner, with capabilities lists.

> For 
> instance, you might create a readOnlyDirectory object which had
> a method "open(filename)" that didn't allow specifying the mode
> as anything but 'r'. Or you might open a file and then pass a
> file object with "read()", "write()", "seek()", and other such
> methods, which would only access that file.

 this could be achieved very simply by setting a CCL with
 "deny everything" permissions on an object that, ordinarily,
 had a full set of functions with NO restrictions, but leaving
 read() alone.

 _and_ also, importantly, adding a CCL onto the __builtins__ etc
 as already discussed to restrict bypassing.


> So _IF_ the only way to access files were through this object (and
> that's a BIG if), then you could imagine a world where HAVING and
> object was equivalent to being able to do something. If a bit of
> code had access to a read-only-file object then it could read that
> file, but couldn't write to it, or do anything else with the file 
> system unless it ALSO had access to some OTHER objects. 

 yep.

> That's
> capabilities... and it would work for most kinds of restricted
> resources, not just the file system. The key idea is that HAVING
> a pointer to the object is equivalent to having the permission to
> USE that object, and whatever it provides access to.

 ah... that's where you and i differ, based on my experience of
 ACLs in NT.

 i envisage a separate "execute" permission from a "read" and "write"
 permission, a bit like directory permissions on a POSIX filesystem.

 under such a scheme where "execute" capability permission exists
 as well as "read" and "write", simply having access to an object
 does NOT guarantee you any rights to CALL that object, because if
 it doesn't have "execute" permission you can't execute [viz, call]
 it!

 ... but what you _can_ do is pass that object to another function which
 _does_ have execute permissions.

 how the hell is _that_ achieved, i hear you say???

 well imagine a CCL as follows:

 [('totally_restricted_function, 'DENY', 'All permissions'),
  ('slightly_less_restricted_function', 'ALLOW', 'execute and read')]

 and this CCL is applied to  the above-mentioned readOnlyDirectory
 object.

 if totally_restricted_function() calls
 slightly_less_restricted_function(), passing a readOnlyDirectory object
 to it, then according to the CCL above, ONLY
 slightly_less_restricted_function() may call functions in that
 object.

 cute, huh?

 l.


From FBatista at uniFON.com.ar  Tue Dec 16 13:37:39 2003
From: FBatista at uniFON.com.ar (Batista, Facundo)
Date: Tue Dec 16 13:39:03 2003
Subject: [Python-Dev] Relative import
Message-ID: <A128D751272CD411BC9200508BC2194D03383383@escpl.tcp.com.ar>

Guido van Rossum wrote:

#- There is the remaining issue of what exactly the syntax would be.  I
#- propose to extend the from clause to allow one or more dots 
#- before the
#- dotted name, and to make the dotted name optional if at least one
#- leading dot is found.  I propose not to change from-less import.

Don't know the restrictions, as never used the package feature in modules,
but seems natural to me to use the same syntax that in a unix filesystem
(maybe without the '/'?). 

This way, you can reference a module in your level, in an upper level,
etc...

.	Facundo

From guido at python.org  Tue Dec 16 13:43:46 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 13:43:52 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Tue, 16 Dec 2003 13:35:14 EST."
	<Pine.LNX.4.44.0312161330001.7068-100000@penguin.theopalgroup.com> 
References: <Pine.LNX.4.44.0312161330001.7068-100000@penguin.theopalgroup.com>
Message-ID: <200312161843.hBGIhkE20496@c-24-5-183-134.client.comcast.net>

> On a related and minor note, can we please expand the grammar to allow
> from...import statements to continue to the next line after a trailing
> comma?  For those of us with large, complex, and componentized projects, it
> would be very nice.  e.g.:
> 
> from OPAL.Modules.Financial.Reports.COGS import generate, webOptions,
>                                                 normalize, row_ops,
>                                                 summary
> 
> instead of:
> 
> from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \
>                                                 normalize, row_ops,   \
>                                                 summary
> 
> 
> This has been a minor nit that bothers just under the threshold of fixing it
> myself.  ;)

I guess this could be implemented by allowing [NEWLINE] after the
comma in the grammar.

But I'm -0 on this change; I fear that someone accidentally adding a
trailing comma (or leaving it in after some rearrangements) will be
very puzzled at the syntax error that comes somewhere in the middle of
the next statement:

  from foo import bar,

  bufsize = 12
          ^syntax error

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fredrik at pythonware.com  Tue Dec 16 14:36:36 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Tue Dec 16 14:37:00 2003
Subject: [Python-Dev] Re: Relative import
References: <Pine.LNX.4.44.0312161330001.7068-100000@penguin.theopalgroup.com>
	<200312161843.hBGIhkE20496@c-24-5-183-134.client.comcast.net>
Message-ID: <brnmsi$21p$1@sea.gmane.org>

Guido van Rossum wrote:

> > from OPAL.Modules.Financial.Reports.COGS import generate, webOptions,
> >                                                 normalize, row_ops,
> >                                                 summary
> >
> > instead of:
> >
> > from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \
> >                                                 normalize, row_ops,   \
> >                                                 summary
> >
> >
> > This has been a minor nit that bothers just under the threshold of fixing it
> > myself.  ;)
>
> I guess this could be implemented by allowing [NEWLINE] after the
> comma in the grammar.
>
> But I'm -0 on this change; I fear that someone accidentally adding a
> trailing comma (or leaving it in after some rearrangements) will be
> very puzzled at the syntax error that comes somewhere in the middle of
> the next statement:
>
>   from foo import bar,
>
>   bufsize = 12
>           ^syntax error

you could require parentheses:

    from OPAL.Modules.Financial.Reports.COGS import (
        generate, webOptions, normalize, row_ops,
        summary
    )

(after all, from-import is a kind of assignment, and "a, b = c" is
the same thing as "(a, b) = c"...)

</F>




From martin at v.loewis.de  Tue Dec 16 14:39:37 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Dec 16 14:41:07 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031215233639.GE26055@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
Message-ID: <m3zndsft6u.fsf@mira.informatik.hu-berlin.de>

Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:

>  i think the only really sensible way forward is to begin from a
>  sound basis - one that is going to be a big job to add retrospectively,
>  but a simple beginning can be made.
> 
>  proposal: how about building ACLs into the python codebase?

I fail to see how ACLs are a sound basis to solve the problem that
rexec solves. ACLs, in my view, are a sound basis to solve a different
problem (that of different identities accessing the same resources).

Also, it seems that nowhere in your proposal you state how ACLs should
be integrated into Python: I.e. what objects are protected by ACLs,
and how do you classify the various actions in a Python program as
read/write/execute/modify_acl? E.g. given

  3 * 4

Is that read, write, execute, and which ACL(s) is(are) considered?

Regards,
Martin

From lkcl at lkcl.net  Tue Dec 16 14:52:00 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 14:53:16 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031216195200.GI17021@lkcl.net>

On Tue, Dec 16, 2003 at 08:39:37PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> >  i think the only really sensible way forward is to begin from a
> >  sound basis - one that is going to be a big job to add retrospectively,
> >  but a simple beginning can be made.
> > 
> >  proposal: how about building ACLs into the python codebase?
> 
> I fail to see how ACLs are a sound basis to solve the problem that
> rexec solves. ACLs, in my view, are a sound basis to solve a different
> problem (that of different identities accessing the same resources).
 
 substitute "identity" for "calling object" i.e. function and i
 believe that there is a viable proposal.

 l.


From lkcl at lkcl.net  Tue Dec 16 14:55:25 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 14:57:22 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031216195525.GJ17021@lkcl.net>

On Tue, Dec 16, 2003 at 08:39:37PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> >  i think the only really sensible way forward is to begin from a
> >  sound basis - one that is going to be a big job to add retrospectively,
> >  but a simple beginning can be made.
> > 
> >  proposal: how about building ACLs into the python codebase?
> 
> I fail to see how ACLs are a sound basis to solve the problem that
> rexec solves. ACLs, in my view, are a sound basis to solve a different
> problem (that of different identities accessing the same resources).
> 
> Also, it seems that nowhere in your proposal you state how ACLs should
> be integrated into Python: I.e. what objects are protected by ACLs,

 all objects - if an acl is non-null.

> and how do you classify the various actions in a Python program as
> read/write/execute/modify_acl? E.g. given

 
>   3 * 4
> 
> Is that read, write, execute, and which ACL(s) is(are) considered?
 
  execute, and execute only, because there's no I/O involved.

  on the multiply operation.

  but _only_ if there's an actual ACL set _on_ the multiply function.

  if there's no acl set on the multiply function, there's no
  restrictions on the multiply function.

  l.


From ws-news at gmx.at  Tue Dec 16 14:53:10 2003
From: ws-news at gmx.at (Werner Schiendl)
Date: Tue Dec 16 14:57:56 2003
Subject: [Python-Dev] Re: Relative import
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com><200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
Message-ID: <brnnrp$47f$1@sea.gmane.org>

Hi,

"Guido van Rossum" <guido@python.org> wrote

> A lot people have presented a good case for relative imports.  Nobody
> has argued to keep the status quo (where imports are ambiguous about
> whether they are meant to be absolute or relative).  So I suggest that
> in 2.4, we introduce the leading dot notation for relative import,
> while still allowing relative import without a leading dot.

I'm +1 on it, I'd only wish an additional

from __future__ import strict_import

(or whatever seems a better name)

to allow a module to indicate that every not explicitly relative import
should be absolute.

This would allow module authors to be explicit from the start and it is also
a safety net
against accidentially adding a relative module with the name of some
absolute one.


best regards
Werner





From lkcl at lkcl.net  Tue Dec 16 14:58:56 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 14:59:30 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031216195856.GK17021@lkcl.net>

On Tue, Dec 16, 2003 at 08:39:37PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> >  i think the only really sensible way forward is to begin from a
> >  sound basis - one that is going to be a big job to add retrospectively,
> >  but a simple beginning can be made.
> > 
> >  proposal: how about building ACLs into the python codebase?
> 
> I fail to see how ACLs are a sound basis to solve the problem that

 in the initial message to the list, i confused the concept of
 acls with "capabilities".

 sorry about that.
 
 l.


From edloper at gradient.cis.upenn.edu  Tue Dec 16 15:17:49 2003
From: edloper at gradient.cis.upenn.edu (Edward Loper)
Date: Tue Dec 16 15:18:07 2003
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 5, Issue 57
In-Reply-To: <E1AWK6r-0002EO-8o@mail.python.org>
References: <E1AWK6r-0002EO-8o@mail.python.org>
Message-ID: <EBA34D5A-3004-11D8-89D3-000393C78C88@gradient.cis.upenn.edu>


Guido said:
> There is the remaining issue of what exactly the syntax would be.  I
> propose to extend the from clause to allow one or more dots before the
> dotted name, and to make the dotted name optional if at least one
> leading dot is found.  I propose not to change from-less import.

What's the motivation for restricting it to from-imports?  I suspect 
that this will cause confusion, and it doesn't feel self-consistent to 
me: why can you call a module ".foo" in one context, and not in 
another?  In particular, I think that if people see that you can do:

     from .foo import *

Then they'll assume that they can also do:

     import .foo

With the obvious semantics, i.e. equivalence to:

     from . import foo

What do we gain by not extending the syntax to from-less imports?

-Edward


From martin at v.loewis.de  Tue Dec 16 15:14:40 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Dec 16 15:20:46 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031216195525.GJ17021@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
Message-ID: <m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>

Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:

> > Also, it seems that nowhere in your proposal you state how ACLs should
> > be integrated into Python: I.e. what objects are protected by ACLs,
> 
>  all objects - if an acl is non-null.

Does this include functions and bound and unbound methods?

> >   3 * 4
> > 
> > Is that read, write, execute, and which ACL(s) is(are) considered?
>  
>   execute, and execute only, because there's no I/O involved.
> 
>   on the multiply operation.

So what operations require read or write access?

>   but _only_ if there's an actual ACL set _on_ the multiply function.
> 
>   if there's no acl set on the multiply function, there's no
>   restrictions on the multiply function.

So ACLs on the objects 3 and 4 would be irrelevant?

Generalizing this, given a class Foo, with a method bar(), and two
instances foo1 and foo2:

foo1 = Foo("/etc/passwd")
foo2 = Foo("/tmp/nonexistant.yet")

and considering the calls foo1.bar() and foo2.bar():

Given that *only* the ACL on the operation bar matters: Does that mean
that the calls either both succeed or both fail, for a given caller?

Regards,
Martin

From martin at v.loewis.de  Tue Dec 16 15:08:37 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Tue Dec 16 15:31:30 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031216195200.GI17021@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195200.GI17021@lkcl.net>
Message-ID: <m3k74wfrui.fsf@mira.informatik.hu-berlin.de>

Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:

> > I fail to see how ACLs are a sound basis to solve the problem that
> > rexec solves. ACLs, in my view, are a sound basis to solve a different
> > problem (that of different identities accessing the same resources).
>  
>  substitute "identity" for "calling object" i.e. function and i
>  believe that there is a viable proposal.

I don't think so. Suppose I have an object A protected by ACLs. Now I
write a new method

def doit():
  A.doit()

Question 1. What is the calling object?
Question 2. Assuming it is "the doit function", and assuming the doit
function is not listed in the ACL: Will the call be executed?
If yes: What is the purpose of the ACL then?
If no: I believe the proposal is not usable in practice. For example,
what should be the ACL on calling open()?

Regards,
Martin


From ws-news at gmx.at  Tue Dec 16 16:02:30 2003
From: ws-news at gmx.at (Werner Schiendl)
Date: Tue Dec 16 16:03:00 2003
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 5, Issue 57
References: <E1AWK6r-0002EO-8o@mail.python.org>
	<EBA34D5A-3004-11D8-89D3-000393C78C88@gradient.cis.upenn.edu>
Message-ID: <brnrtt$crr$1@sea.gmane.org>

Hi,

"Edward Loper" <edloper@gradient.cis.upenn.edu> wrote
>
> Guido said:
> > [...]  I propose not to change from-less import.
>
> What's the motivation for restricting it to from-imports?  I suspect
> that this will cause confusion, and it doesn't feel self-consistent to
> me: why can you call a module ".foo" in one context, and not in
> another?  In particular, I think that if people see that you can do:
>
>      from .foo import *
>
> Then they'll assume that they can also do:
>
>      import .foo

The problem is probably, to what name should the module be bound?
Variable names are not allowed to contain a leading dot.

But it would be possible to say:

import .foo as my_foo
import foo

and access both through modules through the name they're bound to.

Therefore I'd propose to allow relative from-less imports, but require
them to always use the "as" clause.


regards

Werner





From Jack.Jansen at cwi.nl  Tue Dec 16 16:04:40 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue Dec 16 16:04:46 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <20031216161603.GG17021@lkcl.net>
References: <1071587601.3fdf2111afc59@mcherm.com>
	<20031216161603.GG17021@lkcl.net>
Message-ID: <76A3670A-300B-11D8-B3E2-000A27B19B96@cwi.nl>


On 16-dec-03, at 17:16, Luke Kenneth Casson Leighton wrote:
>> Luke replied:
>>> capabilities, acls, schmapabilities, same thiiing :)
>>
>> No... they're not. Read the thread I mentioned above, or read this,
>> and some of the other documentation for the language E:
>>
>>   http://www.erights.org/elib/capability/ode/ode-capabilities.html
>
>  no offense intended: i'll read that later, i'm running out of time.
>
>  without going into too many definitions, consider what i am advocating
>  to be _like_ an access control list but instead to be a capabilities
>  control list, instead.

The distinction between capabilities and ACLs is really important, 
because
they are almost each others opposite. With capabilities you have an 
(unforgable)
right to do something and no-one cares about your identity, with ACLs 
you have
an unforgable identity which is checked against the ACL.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From Jack.Jansen at cwi.nl  Tue Dec 16 16:09:59 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Tue Dec 16 16:10:09 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
Message-ID: <35391BC0-300C-11D8-B3E2-000A27B19B96@cwi.nl>


On 16-dec-03, at 19:26, Guido van Rossum wrote:

> There is the remaining issue of what exactly the syntax would be.  I
> propose to extend the from clause to allow one or more dots before the
> dotted name, and to make the dotted name optional if at least one
> leading dot is found.  I propose not to change from-less import.
> Examples:
>
>   from .foo import bar
>   from .foo.bar import xxx
>   from . import foobar as barfoo
>   from ..foo.bar import *
>   from ... import foobar, barfoo

Is "from . import *" allowed? Whenever I start to think about it I
tend to go into oscillation (yes because within package foo its the same
as "from foo import *" on the outside; no because "import *" isn't
allowed today either).
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From lkcl at lkcl.net  Tue Dec 16 16:23:12 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 16:23:14 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031216212312.GL17021@lkcl.net>

On Tue, Dec 16, 2003 at 09:14:40PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> > > Also, it seems that nowhere in your proposal you state how ACLs should
> > > be integrated into Python: I.e. what objects are protected by ACLs,
> > 
> >  all objects - if an acl is non-null.
> 
> Does this include functions and bound and unbound methods?
 
 if it's a function, it can potentially have an ACL (or a
 better description is CCL - capabilities control list) added,
 and that CCL applies to all sub-objects of the function.

 same with objects, classes - everything.

 

> > >   3 * 4
> > > 
> > > Is that read, write, execute, and which ACL(s) is(are) considered?
> >  
> >   execute, and execute only, because there's no I/O involved.
> > 
> >   on the multiply operation.
> 
> So what operations require read or write access?

 short answer: all of them.
 
 longer answer: just like with exceptions, that has to be classified
 according to conventions that are yet to be decided.

 i propose that, for example, a capability be created to control _file_
 read access and _file_ write access, with large provisos on the
 documentation attached to these two capabilities that ensure they are
 distinguished from the right to _modify_ an object.


 basically it's all conventions, just like exceptions are conventions.

 all i can do is recommend a framework and some guidelines on what
 conventions could be fitted over that framework.

 heck, it may even be worthwhile defining a low-level object, named
 a Capability, just like an Exception is defined, and then expect
 people to create their own, and recommend the creation of certain
 Capabilities that are inherited from the Capability base class.


> >   but _only_ if there's an actual ACL set _on_ the multiply function.
> > 
> >   if there's no acl set on the multiply function, there's no
> >   restrictions on the multiply function.
> 
> So ACLs on the objects 3 and 4 would be irrelevant?

 well... actually.... no :)

 but are such ACLs relevant?

 can you do 3 += 5?

 no you can't: python throws up a SyntaxError: can't assign to a
 literal.

 so, whilst the answer is yes, you _can_ associate an ACL [or
 better-named, a CCL] with the object "3" and the object "4",
 i would be surprised if there were any situations where they got...
 hang on...

 >>> dir(3)
 ['__abs__', '__add__', '__and__', '__class__', '__cmp__', '__coerce__',
 '__delattr__', '__div__', '__divmod__', '__doc__', '__float__',
 '__floordiv__', '__getattribute__', '__getnewargs__', '__hash__',
 '__hex__', '__init__', '__int__', '__invert__', '__long__',
 '__lshift__', '__mod__', '__mul__', '__neg__', '__new__',
 '__nonzero__', '__oct__', '__or__', '__pos__', '__pow__', '__radd__',
 '__rand__', '__rdiv__', '__rdivmod__', '__reduce__', '__reduce_ex__',
 '__repr__', '__rfloordiv__', '__rlshift__', '__rmod__', '__rmul__',
 '__ror__', '__rpow__', '__rrshift__', '__rshift__', '__rsub__',
 '__rtruediv__', '__rxor__', '__setattr__', '__str__', '__sub__',
 '__truediv__', '__xor__']

 yes, i believe it _might_ be relevant to set a CCL on an object "3"
 because someone might want to restrict how... oh, i dunno... mmm.
 __setattr__ gets used.

 >>> 3.__xor__
            ^
  SyntaxError: invalid syntax

 oh.  maybe not :)


 however, a UserLong or a UserInt is a different matter.

 [btw, change of subject: why is there no __eq__, no __gt__, no __le__
  in that list above?]




> Generalizing this, given a class Foo, with a method bar(), and two
> instances foo1 and foo2:
> 
> foo1 = Foo("/etc/passwd")
> foo2 = Foo("/tmp/nonexistant.yet")
> 
> and considering the calls foo1.bar() and foo2.bar():
> 
> Given that *only* the ACL on the operation bar matters: Does that mean
> that the calls either both succeed or both fail, for a given caller?

 i'm not entirely sure what you mean, but let's assume that objects
 (and their instances!) all have an __set_ccl__() function.

 let's also define a CCL:

 deny_ccl = [('all functions', DENY, 'execute')]

 then, let's try this:

 foo2 = Foo("/etc/passwd")

 Foo.bar.__set_ccl__(deny_ccl)
 foo1 = Foo("/etc/passwd")

 foo2.bar() // executed okay
 
 foo1.bar()
 *** EXCEPTION: execute capability barred to "all functions".

 
 okay, let's try this, first making sure there's no ambiguity by
 re-setting the CCL on the Foo class:

 Foo.bar.__set_ccl__(None)

 okay, now we go:

 foo3 = Foo("/etc/passwd")

 foo3.bar() // executed okay
 foo3.bar.__set_ccl__(deny_ccl)

 foo3.bar() 
 *** EXCEPTION: execute capability barred to "all functions".

 
 note that there are three separate uses of __set_ccl__().

 first is to set a CCL on the _class_ Foo.
 second is to clear a CCL the class Foo.
 third is to set a CCL on an instance of a Foo object.

 just to clarify: i _may_ have just violated my own proposal
 by not setting an "inherited-on-create" capability on the CCL named
 deny_ccl.

 the reason for needing to have such a capability is to be able to
 distinguish between setting CCLs on a class and a CCL being
 created on an object when an instance _of_ that class is created.

 this is part of the lesson learned from NT's security model
 (and originally VMS's) and its implementation.

 
 i won't go into more details on this inherit-on-create thing
 until the rest of the stuff sinks in :)

 l.


From lkcl at lkcl.net  Tue Dec 16 16:30:41 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 16:30:45 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3k74wfrui.fsf@mira.informatik.hu-berlin.de>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195200.GI17021@lkcl.net>
	<m3k74wfrui.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031216213041.GM17021@lkcl.net>

On Tue, Dec 16, 2003 at 09:08:37PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> > > I fail to see how ACLs are a sound basis to solve the problem that
> > > rexec solves. ACLs, in my view, are a sound basis to solve a different
> > > problem (that of different identities accessing the same resources).
> >  
> >  substitute "identity" for "calling object" i.e. function and i
> >  believe that there is a viable proposal.
> 
> I don't think so. Suppose I have an object A protected by ACLs. Now I
> write a new method
> 
> def doit():
>   A.doit()
> 
> Question 1. What is the calling object?

 the globally-defined doit function.

> Question 2. Assuming it is "the doit function", and assuming the doit
> function is not listed in the ACL: Will the call be executed?

 see below.

> If yes: What is the purpose of the ACL then?
> If no: I believe the proposal is not usable in practice. For example,
> what should be the ACL on calling open()?
 
 okay.  that is why in ACLs (or better CCLs) it is necessary
 to have an "all functions" wildcard "name".

 in NT, this is the "world" security id "S-1-1" (i think) which
 represents "everyone".

 if you set a CCL, and you expect to deny everything but a particular
 function, you must have a two-entry CCL:

 [('function_to_be_denied_access', DENY, 'permissions_to_be_denied'),
  ('all functions', 'ALLOW', 'anything')]

 and yes, this is _normal_ practice in the use of ACLs in NT and VMS.

 and yes, the default action is (IIRC), "if not explicitly stated,
 deny".

 with the exception to that, of course, that if no CCL is set,
 it's "as if" there was a CCL set of [('all functions', 'ALLOW',
 'anything')].

 l.


From lkcl at lkcl.net  Tue Dec 16 16:36:58 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Tue Dec 16 16:37:02 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <76A3670A-300B-11D8-B3E2-000A27B19B96@cwi.nl>
References: <1071587601.3fdf2111afc59@mcherm.com>
	<20031216161603.GG17021@lkcl.net>
	<76A3670A-300B-11D8-B3E2-000A27B19B96@cwi.nl>
Message-ID: <20031216213658.GN17021@lkcl.net>

On Tue, Dec 16, 2003 at 10:04:40PM +0100, Jack Jansen wrote:
> 
> On 16-dec-03, at 17:16, Luke Kenneth Casson Leighton wrote:
> >>Luke replied:
> >>>capabilities, acls, schmapabilities, same thiiing :)
> >>
> >>No... they're not. Read the thread I mentioned above, or read this,
> >>and some of the other documentation for the language E:
> >>
> >>  http://www.erights.org/elib/capability/ode/ode-capabilities.html
> >
> > no offense intended: i'll read that later, i'm running out of time.
> >
> > without going into too many definitions, consider what i am advocating
> > to be _like_ an access control list but instead to be a capabilities
> > control list, instead.
> 
> The distinction between capabilities and ACLs is really important, 
> because
> they are almost each others opposite. With capabilities you have an 
> (unforgable)
> right to do something and no-one cares about your identity, with ACLs 
> you have
> an unforgable identity which is checked against the ACL.

 i'd like to introduce you to a new concept which is idential
 in form to an ACL - access control list - except that instead
 of "users" being allowed or denied access to perform certain
 operations you have instead _functions_ being allowed or
 denied access to perform certain operations.

 perhaps a better way to explain the concept to you is to introduce
 a concept called "qualified" capabilities, where what you know of
 as capabilities is "qualified" on a per-function (that's per-caller)
 basis.

 obviously, any object (by object i am referring generically to
 classes, class instances, functions, modules, absolutely anything)
 can potentially have many "callers", consequently it is necessary
 to create a _list_ of qualified capabilities, and for the
 relevant QCap in that list to be looked up and applied as needed.

 where, of course, the special wildcard name 'all functions' applies
 to _all_ callers.


 which makes what i am proposing to be named
 
 "QCCL" - qualified-capabilities control list.

 yuk :)

 l.


From guido at python.org  Tue Dec 16 17:07:56 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 17:08:01 2003
Subject: [Python-Dev] Re: Python-Dev Digest, Vol 5, Issue 57
In-Reply-To: Your message of "Tue, 16 Dec 2003 15:17:49 EST."
	<EBA34D5A-3004-11D8-89D3-000393C78C88@gradient.cis.upenn.edu> 
References: <E1AWK6r-0002EO-8o@mail.python.org>  
	<EBA34D5A-3004-11D8-89D3-000393C78C88@gradient.cis.upenn.edu> 
Message-ID: <200312162207.hBGM7uN20814@c-24-5-183-134.client.comcast.net>

[Ed]
> What's the motivation for restricting it to from-imports?  I suspect 
> that this will cause confusion, and it doesn't feel self-consistent to 
> me: why can you call a module ".foo" in one context, and not in 
> another?  In particular, I think that if people see that you can do:
> 
>      from .foo import *
> 
> Then they'll assume that they can also do:
> 
>      import .foo
> 
> With the obvious semantics, i.e. equivalence to:
> 
>      from . import foo
> 
> What do we gain by not extending the syntax to from-less imports?

Currently, the rule is that after

  import <something>

you can use

  <something>

in an expression to refer to whatever you imported.  This applies to
both

  import foo          <-->  foo      in expressions

and

  import foo.bar      <-->  foo.bar  in expressions

But if we were to allow

  import .foo

we'd have to allow

  .foo

in expressions, and I would like to reserve that syntax for the "with"
statement.

[Werner]
> But it would be possible to say:
> 
> import .foo as my_foo
> import foo
> 
> and access both through modules through the name they're bound to.
> 
> Therefore I'd propose to allow relative from-less imports, but require
> them to always use the "as" clause.

This just adds more complexity; there's nothing you can do without
this addition.

[Jack]
> Is "from . import *" allowed? Whenever I start to think about it I
> tend to go into oscillation (yes because within package foo its the same
> as "from foo import *" on the outside; no because "import *" isn't
> allowed today either).

Yes, "from . import <whatever>" has the exact same meaning as spelling
out the corresponding absolute module path after "from".

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Tue Dec 16 18:24:52 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec 16 18:24:59 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE: rexec.py
	unuseable)
In-Reply-To: <1071587601.3fdf2111afc59@mcherm.com>
Message-ID: <200312162324.hBGNOq916269@oma.cosc.canterbury.ac.nz>

Michael Chermside <mcherm@mcherm.com>:

> Suppose that had some code which had an object representing a
> directory, with a method "open(filename, mode='r')" that opened 
> files in the directory. Given this object, you could imagine 
> constructing new objects with more limited capabilities.

With regard to files, we'd almost have something like that
now, if it weren't for the unfortunate fact that the file
type's constructor can be used to open any file.

In light of this, it may have been a serious mistake to
unify the 'file' type and the 'open' function. Is it too
late to back out of that decision?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Tue Dec 16 18:38:20 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 18:38:37 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE: rexec.py
	unuseable)
In-Reply-To: Your message of "Wed, 17 Dec 2003 12:24:52 +1300."
	<200312162324.hBGNOq916269@oma.cosc.canterbury.ac.nz> 
References: <200312162324.hBGNOq916269@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312162338.hBGNcKD21087@c-24-5-183-134.client.comcast.net>

> Michael Chermside <mcherm@mcherm.com>:
> 
> > Suppose that had some code which had an object representing a
> > directory, with a method "open(filename, mode='r')" that opened 
> > files in the directory. Given this object, you could imagine 
> > constructing new objects with more limited capabilities.

Greg Ewing:

> With regard to files, we'd almost have something like that
> now, if it weren't for the unfortunate fact that the file
> type's constructor can be used to open any file.

Good observation.

> In light of this, it may have been a serious mistake to
> unify the 'file' type and the 'open' function. Is it too
> late to back out of that decision?

I think so.  But to remedy this, the file constructor could simply
refuse to open any files when called from restricted mode.  (Although
this would open up the same kind of holes that Samuele so cleverly
showed exist in other built-ins that do this kind of checking.)

(Re the capabilities discussion, this is Python 3.0 material if I ever
saw some; no time to respond in more detail, given that we've had this
thread before.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Tue Dec 16 19:22:49 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec 16 19:23:16 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
Message-ID: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>

Guido:

> I suggest that in 2.4, we introduce the leading dot notation for
> relative import, while still allowing relative import without a
> leading dot. In 2.5 we can start warning about relative import
> without a leading dot

If I understand correctly, the rule you're working towards is that if
the path starts with a dot, it's relative, otherwise it's absolute. Is
that correct?

I'm not sure whether eliminating the "ambiguous" category of
module references is the best idea. There are situations in which
it's actually what you want.

Suppose there are two top-level modules A and B, and A imports B
using a bare name. Then someone decides to put A into a package.
The reference to B still works. Then they decide to put B into
the same package as well. The reference to B still works.

But if only unambiguously absolute or relative references are
possible, there is no way for A to refer to B that works in all
these combinations.

So I think we really want *three* kinds of module reference:

   A: Explicitly absolute
   B: Explicitly relative to the current module
   C: Searched for upwards in the package hierarchy from the current
      module

(Note that C is a generalisation of the current "ambiguous"
references which only look in two places.)

Suggested syntaxes for these:

  A:   a.b.c.          Path ends with a dot

  B:   .a.b.c          Path begins with a dot

  C:   a               Path neither begins nor ends with a dot
       a.b.c

Note that ALL current module references would be type C according
to this. Yes, this is a semantic change, but one that I believe
would be for the best in the long run, and don't think would cause
a lot of upheaval.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From tjreedy at UDel.Edu  Tue Dec 16 20:04:41 2003
From: tjreedy at UDel.Edu (Terry Reedy)
Date: Tue Dec 16 20:05:14 2003
Subject: [Python-Dev] Re: Pie-thon benchmarks
References: <LNBBLJKPBEHFEDALKOLCOEKOHMAB.tim.one@comcast.net><a06010202bc03b13b3134@[172.24.18.98]><200312151937.hBFJbYX18012@c-24-5-183-134.client.comcast.net>
	<a06010203bc03be7149c2@[172.24.18.98]>
	<200312151957.hBFJvWi18151@c-24-5-183-134.client.comcast.net>
	<brmahq$27a$1@sea.gmane.org>
	<200312161830.hBGIUhE20354@c-24-5-183-134.client.comcast.net>
Message-ID: <001a01c3c439$c83a74c0$0400a8c0@HPPAVILION>


> > meaning of every bytecode).  So I would include at
> > least the syntax and standard lib test files (-
> > exec and eval tests) as part of the benchmark.

> Right.  I don't want to include the entire test suite, but I do want

Sorry, I meant builtins test, *not* the whole suite of stdlib tests.

Terrj

From greg at cosc.canterbury.ac.nz  Tue Dec 16 20:09:14 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec 16 20:09:21 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312161843.hBGIhkE20496@c-24-5-183-134.client.comcast.net>
Message-ID: <200312170109.hBH19DG16699@oma.cosc.canterbury.ac.nz>

Guido:

> > from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \
> >                                                 normalize, row_ops,   \
> >                                                 summary
> 
> But I'm -0 on this change; I fear that someone accidentally adding a
> trailing comma ... will be very puzzled

How about a "suite" of things to import?

  from OPAL.Modules.Financial.Reports.COGS import:
    generate, webOptions, normalize
    row_ops, summary

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Tue Dec 16 20:16:18 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 16 20:16:24 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Wed, 17 Dec 2003 14:09:14 +1300."
	<200312170109.hBH19DG16699@oma.cosc.canterbury.ac.nz> 
References: <200312170109.hBH19DG16699@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312170116.hBH1GId21276@c-24-5-183-134.client.comcast.net>

> > > from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \
> > >                                                 normalize, row_ops,   \
> > >                                                 summary
> > 
> > But I'm -0 on this change; I fear that someone accidentally adding a
> > trailing comma ... will be very puzzled
> 
> How about a "suite" of things to import?
> 
>   from OPAL.Modules.Financial.Reports.COGS import:
>     generate, webOptions, normalize
>     row_ops, summary

Nah.  But I liked the suggestion seen here earlier of allowing
parenthesis:

from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions,
                                          normalize, row_ops, summary)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pje at telecommunity.com  Tue Dec 16 20:19:51 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Tue Dec 16 20:19:57 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312170109.hBH19DG16699@oma.cosc.canterbury.ac.nz>
References: <200312161843.hBGIhkE20496@c-24-5-183-134.client.comcast.net>
Message-ID: <5.1.1.6.0.20031216201656.02363ec0@telecommunity.com>

At 02:09 PM 12/17/03 +1300, Greg Ewing wrote:
>Guido:
>
> > > from OPAL.Modules.Financial.Reports.COGS import generate, webOptions, \
> > >                                                 normalize, row_ops,   \
> > >                                                 summary
> >
> > But I'm -0 on this change; I fear that someone accidentally adding a
> > trailing comma ... will be very puzzled
>
>How about a "suite" of things to import?
>
>   from OPAL.Modules.Financial.Reports.COGS import:
>     generate, webOptions, normalize
>     row_ops, summary

First impression: cool!

Second impression: "suites" contain statements, not a list.

Third impression: why no comma after 'normalize'?


From greg at cosc.canterbury.ac.nz  Tue Dec 16 20:53:35 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec 16 20:53:40 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <5.1.1.6.0.20031216201656.02363ec0@telecommunity.com>
Message-ID: <200312170153.hBH1rZG16888@oma.cosc.canterbury.ac.nz>

> Second impression: "suites" contain statements, not a list.

That's why I put "suite" in quotes. It wouldn't really be
a suite, just an indented block of stuff.

But if you want, it could be a real suite that only
happens to allow import statements...

  from OPAL.Modules.Financial.Reports.COGS:
    import generate, webOptions, normalize
    import row_ops, summary

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From janssen at parc.com  Tue Dec 16 21:05:42 2003
From: janssen at parc.com (Bill Janssen)
Date: Tue Dec 16 21:06:02 2003
Subject: [Python-Dev] Relative import 
In-Reply-To: Your message of "Tue, 16 Dec 2003 16:22:49 PST."
	<200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz> 
Message-ID: <03Dec16.180547pst."58611"@synergy1.parc.xerox.com>

> Note that ALL current module references would be type C according
> to this. Yes, this is a semantic change, but one that I believe
> would be for the best in the long run, and don't think would cause
> a lot of upheaval.

Yes, I like this.

Bill


From andrew-pythondev at puzzling.org  Tue Dec 16 21:45:50 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Tue Dec 16 21:45:56 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312170153.hBH1rZG16888@oma.cosc.canterbury.ac.nz>
References: <5.1.1.6.0.20031216201656.02363ec0@telecommunity.com>
	<200312170153.hBH1rZG16888@oma.cosc.canterbury.ac.nz>
Message-ID: <20031217024550.GB473@frobozz>

On Wed, Dec 17, 2003 at 02:53:35PM +1300, Greg Ewing wrote:
> > Second impression: "suites" contain statements, not a list.
> 
> That's why I put "suite" in quotes. It wouldn't really be
> a suite, just an indented block of stuff.
> 
> But if you want, it could be a real suite that only
> happens to allow import statements...
> 
>   from OPAL.Modules.Financial.Reports.COGS:
>     import generate, webOptions, normalize
>     import row_ops, summary

But this is also a very odd suite, because you can't use other statements...

But it does make me think of:

  with OPAL.Modules.Financial.Reports.COGS:
    import .generate, .webOptions, .normalize
    import .row_ops, .summary

Which is like a bastard child of the relative import proposal and the with
statement proposal... <wink>

-Andrew.


From eppstein at ics.uci.edu  Tue Dec 16 22:07:52 2003
From: eppstein at ics.uci.edu (David Eppstein)
Date: Tue Dec 16 22:07:46 2003
Subject: [Python-Dev] Re: Relative import
References: <200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
Message-ID: <eppstein-F99476.19075216122003@sea.gmane.org>

In article <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>,
 Greg Ewing <greg@cosc.canterbury.ac.nz> wrote:

> So I think we really want *three* kinds of module reference:
> 
>    A: Explicitly absolute
>    B: Explicitly relative to the current module
>    C: Searched for upwards in the package hierarchy from the current
>       module
> 
> (Note that C is a generalisation of the current "ambiguous"
> references which only look in two places.)
> 
> Suggested syntaxes for these:
> 
>   A:   a.b.c.          Path ends with a dot
> 
>   B:   .a.b.c          Path begins with a dot
> 
>   C:   a               Path neither begins nor ends with a dot
>        a.b.c

Is funny punctuation really the right way to spell an important 
distinction like this?

-- 
David Eppstein                      http://www.ics.uci.edu/~eppstein/
Univ. of California, Irvine, School of Information & Computer Science


From tdelaney at avaya.com  Tue Dec 16 22:20:32 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Tue Dec 16 22:20:38 2003
Subject: [Python-Dev] Re: Relative import
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0B9B@au3010avexu1.global.avaya.com>

> From: David Eppstein
> 
> Is funny punctuation really the right way to spell an important 
> distinction like this?

That's what I've been wondering. I really don't like the proposal to use . in various places to signify relative and absolute imports. It seems non-obvious both in understandability and readability.

Every example I've seen has looked wrong to me. I'll try to think of something I prefer, but I am definitely -1 on the proposed syntax changes.

Tim Delaney

From anthony at interlink.com.au  Tue Dec 16 22:27:52 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Tue Dec 16 22:28:23 2003
Subject: [Python-Dev] Re: Relative import 
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0B9B@au3010avexu1.global.avaya.com>
Message-ID: <200312170327.hBH3RrpU004309@localhost.localdomain>


>>> "Delaney, Timothy C (Timothy)" wrote
> > Is funny punctuation really the right way to spell an important 
> > distinction like this?
> 
> That's what I've been wondering. I really don't like the proposal to use
> . in various places to signify relative and absolute imports. It seems 
> non-obvious both in understandability and readability.

I'd have to agree. In the same way that backticks are painful to deal 
with (they're small and non-obvious), the difference between 

  from foo import bar

and 

  from .foo import bar

is too small to be pleasant. I also have to wonder about the use 
case for .. and ... - looking through my code, I can't see anything
where using ... rather than a fully qualified name would make the 
code any more readable.

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From greg at cosc.canterbury.ac.nz  Tue Dec 16 22:31:50 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Tue Dec 16 22:32:35 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <eppstein-F99476.19075216122003@sea.gmane.org>
Message-ID: <200312170331.hBH3Vog17551@oma.cosc.canterbury.ac.nz>

David Eppstein <eppstein@ics.uci.edu>:

> Is funny punctuation really the right way to spell an important 
> distinction like this?

Maybe not, but I haven't seen anything obviously better
proposed yet.

I've no idea what else to to for relative paths. A leading dot
doesn't seem all that bad to me for that. I concede that 
a trailing dot would look rather odd, though.

The absolute-path case could be spelled using a special name
for the root of the package hierarchy, but then we need to
think of a name that's sufficiently unlikely as a real module
name.

I don't think I like the idea of re-using the word "global",
although I'm not sure.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From tim.one at comcast.net  Tue Dec 16 23:42:45 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Dec 16 23:42:46 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312170116.hBH1GId21276@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEDHHNAB.tim.one@comcast.net>

[Guido]
> Nah.  But I liked the suggestion seen here earlier of allowing
> parenthesis:
>
> from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions,
>                                           normalize, row_ops, summary)

So do I.  Something I'm still acutely aware of despite not using Emacs for
years is how much trouble it would be to teach python-mode.el about a unique
new way to continue a statement.  Parens would "just work".  Ditto for IDLE,
and tokenize.py, etc etc etc etc.  I bet tools parsing import statements for
*content* (like pychecker, and perhaps pyclbr.py -- the others mentioned are
just looking at form) would need modification regardless, though.


From duncan at rcp.co.uk  Wed Dec 17 04:17:25 2003
From: duncan at rcp.co.uk (Duncan Booth)
Date: Wed Dec 17 04:17:39 2003
Subject: [Python-Dev] rexec.py unuseable
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
	<20031216212312.GL17021@lkcl.net>
Message-ID: <n2m-g.Xns94545D60B83E1duncanrcpcouk@127.0.0.1>

Luke Kenneth Casson Leighton <lkcl@lkcl.net> wrote in 
news:20031216212312.GL17021@lkcl.net:

>  foo3.bar() // executed okay
>  foo3.bar.__set_ccl__(deny_ccl)
> 
>  foo3.bar() 
>  *** EXCEPTION: execute capability barred to "all functions".

This behaviour seems to imply a rather fundamental change to the way that 
Python handles bound methods. At present every time you access foo.bar you 
get a new object created, but I think from your description that you want 
to be able to associate a ccl with all the foo.bar objects.

Or to put it another way:

fn1 = foo3.bar
fn1() // executed okay
fn2 = foo3.bar
fn2.__set_ccl__(deny_ccl)

fn2()
 *** EXCEPTION: execute capability barred to "all functions".
fn1()
 *** ??? What should this do.

If I set a ccl on fn2, should it also affect access to fn1? They are 
separate objects, so shouldn't they have separate ccls? If they share a 
common ccl, then how is it going to be stored without impacting heavily on 
every method call?

-- 
Duncan Booth                      duncan.booth at suttoncourtenay.org.uk
int month(char *p){return(124864/((p[0]+p[1]-p[2]&0x1f)+1)%12)["\5\x8\3"
"\6\7\xb\1\x9\xa\2\0\4"];} // Who said my code was obscure?

From theller at python.net  Wed Dec 17 06:32:32 2003
From: theller at python.net (Thomas Heller)
Date: Wed Dec 17 06:32:47 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1) -
	custom Py_Initialize()
In-Reply-To: <002301c3c29c$f2753480$0201010a@kret> (Wiktor Sadowski's
	message of "Mon, 15 Dec 2003 00:49:42 +0100")
References: <002301c3c29c$f2753480$0201010a@kret>
Message-ID: <wu8vel2n.fsf@python.net>

"Wiktor Sadowski" <art@wiktorsadowski.com> writes:

> Adding a few lines (below) to Python would enable custom
> Py_Initialize() and would allow to setup on time different than
> existing (import.c-zipimport.c) import mechanisms and changing some
> other Python-initialization internals without messing with Python
> core.
>
> It should make life easier for embedders and help writing specialized
> Python executables (like Zope or Quixote).  (We have such specialized
> executable to run packed Python programs remotely,kind of Java Web
> Start).  From what I know Python installers (py2exe,Gordon McMillan's
> installer) could benefit from such addition as well.
>
> All these functions simply expose some modules globals. They don't
> change anything in Python core.
>
> Best regards,
> Wiktor Sadowski
>
>
> /*pythonrun.c*/
> void Internal_Py_SetInitialized(int){initialized=i;}
> void  Internal_PyUnicode_Init(void){_PyUnicode_Init();}
> void  Internal_PyUnicode_Fini(void){_PyUnicode_Fini();}
> void  Internal_PyGILState_Init(PyInterpreterState *s, PyThreadState
> *ts){_PyGILState_Init(s,ts);}
> void  Internal_PyGILState_Fini(void){_PyGILState_Fini();}
> long Internal_PyGC_Collect(void){return PyGC_Collect();}
>
> /*pythonrun.h*/
> PyAPI_FUNC(void) Internal_Py_SetInitialized(int i);
> PyAPI_FUNC(void) Internal_PyUnicode_Init(void);
> PyAPI_FUNC(void) Internal_PyUnicode_Fini(void);
> PyAPI_FUNC(void) Internal_PyGILState_Init(PyInterpreterState *,
> PyThreadState *);
> PyAPI_FUNC(void) Internal_PyGILState_Fini(void);
> PyAPI_FUNC(long) Internal_PyGC_Collect(void);
>

I would rather like a function like Py_InitializeEx(struct _init*),
where struct _init would have components like PYTHONPATH, optimize, and
so on.

Thomas


From ncoghlan at iinet.net.au  Wed Dec 17 07:20:17 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Dec 17 07:20:24 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <20031216213658.GN17021@lkcl.net>
References: <1071587601.3fdf2111afc59@mcherm.com>	<20031216161603.GG17021@lkcl.net>	<76A3670A-300B-11D8-B3E2-000A27B19B96@cwi.nl>
	<20031216213658.GN17021@lkcl.net>
Message-ID: <3FE04A01.7010500@iinet.net.au>

Luke Kenneth Casson Leighton wrote:
>  i'd like to introduce you to a new concept which is idential
>  in form to an ACL - access control list - except that instead
>  of "users" being allowed or denied access to perform certain
>  operations you have instead _functions_ being allowed or
>  denied access to perform certain operations.

I don't think this discussion is going to get anywhere until you _do_ 
take the time to understand the difference between the model you are 
proposing and the capability model that seemed to be preferred the last 
time executing untrusted Python code safely was discussed.

ONE LINE SUMMARIES

ACL/CCL/whatever you want to call it:
   Access the feature, then have permission either granted or denied

Capabilities:
   If you can access it, you have permission to use it.
   If you aren't allowed to use it, you won't even be able to see it.

Below is my attempt at explaining in more detail the difference between 
the two as I understand it. I must admit I am quite curious about the 
fact that you seem to have plenty of time to _write_ about your 
proposal, but no time to listen to some quite reasonable responses.

ACCESS CONTROL LISTS
An access control list is neutral to what a 'user' is, so I'll switch to 
the term 'Entity' instead. The important part is that it is a mapping: 
"Entity X is allowed to perform action Y on other Entity Z".

It serves as a post hoc check - _after_ Entity X tries to do Y to Z, the 
control list for Y & Z is checked, then X is either allowed to proceed, 
or told to rack off, depending on what the control list reports. That 
is, just because X _can_ try to perform action Y on Z (the relevant 
method or whatever is accessible to X), doesn't mean that X _may_ 
perform action Y (the control list grants permission to X).

With control lists, the concepts of access to a feature, and permission 
to use that feature, are handled separately. E.g. I can try to delete 
the windows directory on my machine at work (I have access to that 
directory, and access to the delete command), but I'm not an 
administrator, so the attempt will fail (I don't have permission to 
delete that directory). This is exactly what you are describing, even 
though the users are 'functions' rather than 'logged in users'.

CAPABILITIES
Capabilities work differently. Instead of applying a post-hoc check that 
slows down _every_ attempt to perform an action, they unify the concepts 
of 'can' and 'may'. If X is able to express the command "do Y to Z", 
then X is also _permitted_ to do Y to Z. No post-hoc checks required.


For example, suppose I wasn't allowed to open writable files.

I call:
 >>> target = file("myfile.txt", "w")

With ACL's, I would get an error along the lines of:
    AccessDenied: No permission to open writable files

With capabilities, I would get an error along the lines of:
    InvalidArgument: "w" is not a recognised file mode

Does this make any more sense?

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From Jack.Jansen at cwi.nl  Wed Dec 17 07:51:12 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Dec 17 07:51:14 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1) - custom
	Py_Initialize()
In-Reply-To: <wu8vel2n.fsf@python.net>
References: <002301c3c29c$f2753480$0201010a@kret> <wu8vel2n.fsf@python.net>
Message-ID: <B15416B0-308F-11D8-B000-000A958D1666@cwi.nl>


On 17-dec-03, at 12:32, Thomas Heller wrote:
> I would rather like a function like Py_InitializeEx(struct _init*),
> where struct _init would have components like PYTHONPATH, optimize, and
> so on.

You'd have to be very careful to design the struct in such a way that 
you'd
have good compatibility.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From ncoghlan at iinet.net.au  Wed Dec 17 07:58:20 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Dec 17 07:58:29 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
Message-ID: <3FE052EC.90705@iinet.net.au>

Greg Ewing wrote:

> So I think we really want *three* kinds of module reference:
> 
>    A: Explicitly absolute
>    B: Explicitly relative to the current module
>    C: Searched for upwards in the package hierarchy from the current
>       module
> 
> (Note that C is a generalisation of the current "ambiguous"
> references which only look in two places.)
> 

Alternate spellings (plus category D, which is "Python 2.3 semantics"):

A: from __absolute__.dotted.name import foo
B: from __relative__.dotted.name import bar
C: from __heirarchy__.dotted.name import foobar
D: from dotted.name import barfoo

I believe this spelling would only require import to handle the special 
cases as the first 'package' name (although having the compiler 
recognise them may be a later optimisation), and should also work with 
straight "import __absolute__.dotted.name.foo" statements.

Then Guido's migration plan would apply to the progression of the D 
semantics from the Python 2.3 model to being a shorthand for the 
absolute semantics.

And once _that_ has happened, then we could get rid of the 
'__absolute__' version in the name of TOOWTDI.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From mcherm at mcherm.com  Wed Dec 17 08:56:37 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Dec 17 08:56:58 2003
Subject: [Python-Dev] Re: rexec.py unuseable
Message-ID: <1071669397.3fe06095c9f3c@mcherm.com>

Luke:

Seeing as you've now had time enough to read what I'd written, I was going to 
respond to some of the points that you made, but then I read Nick Coghlan's 
posting[1], and I really don't have anything else to add.

[1] http://mail.python.org/pipermail/python-dev/2003-December/041105.html

Well, that's not quite true... I guess I _do_ have something small to add.
In designing a security policy for file systems, there are very few things
that you can do with the protected items. Separate "read", "write", and
"execute" permissions seem appropriate. Furthermore, there tends to be a
clear idea of a "user", and in many situations we want all access to items
(files) to depend only on who that "user" is (sandboxing an email program
is an exception... are you listening MS?). Finally, we often want to
provide centralized administration of access rights to file systems.
ACLs are a good match for these design specs.

However, controlling what arbitrary programs may do is a very different
beast. First of all, the kinds of restrictions that we want are HIGHLY
variable. Sometimes we want to prohibit all file access. Sometimes we
want to allow read-only access to a certain file only. Sometimes we
want access only to a certain directory. Sometimes we don't care at
all about file system access, we're concerned instead with providing
an object which can be queried but only in certain ways -- and the
specifics of what is permitted and what isn't are completely dependent
on the particular application.

On top of that, there's rarely a clear concept of "users" that controls
what actions we want to permit. Instead, we usually want to allow or
deny certain capabilities to particular _sections of code_. (Sometimes
that will be a module, sometimes a plug-in, sometimes a child program.)

All of these things suggest that maintaining a list of who is permitted
to access what is simply not the best model for providing security in
a programming language. I think that capabilities ARE an excellent
model. Sections of code already have the concept of "access" to certain
objects and variables (at least since the invention of structured
programming in the late 1960s). And _if_ unforgability and private
data structures are provided as elemental building tools, then it is
relatively straightforward to write code that enforces whatever set of
access rules is required, regardless of how complex it may be.

-- Michael Chermside


From mickey at tm.informatik.uni-frankfurt.de  Wed Dec 17 09:01:00 2003
From: mickey at tm.informatik.uni-frankfurt.de (Michael Lauer)
Date: Wed Dec 17 09:01:08 2003
Subject: [Python-Dev] bug in python arm-linux?: start_new_thread fails after
	popen
Message-ID: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>

Hi, I have a python program which works fine on x86 but doesn't work on
any of my arm-linux devices (Zaurus, Ipaq, SIMpad) - all of them are
running Python 2.3.2 on top of glibc 2.3.2+linuxthreads on top of kernel
2.4.18-rmk6-pxa3 respectively kernel 2.4.19-rmk7.

After a long week of debugging I now can reproduce the behaviour with
the following minimal case:

----------------------------------------------------------------------
import thread
from time import sleep
import sys
import os

def createPipe():
    print os.popen( "ls -l" ).read()

def threadMain( name ):
    while True:
        sys.stderr.write( name+": i'm still running" )
        sys.stderr.flush()
        sleep( 1 )

if __name__ == '__main__':

    createPipe()

    print "BEFORE start_new_thread"
    thread.start_new_thread( threadMain, ( "1", ) )
    print "AFTER start_new_thread"

    createPipe()

    print "BEFORE start_new_thread"
    thread.start_new_thread( threadMain, ( "2", ) )
    print "AFTER start_new_thread"

sleep( 5 )
-----------------------------------------------------------------------

This program - as is - just hangs in the first start_new_thread() and
never comes back.
If you outcomment the first call to createPipe() out, then it works
fine.

In the first - hanging - case, an strace shows:
-------------------------------------------------------------------------
write(1, "BEFORE start_new_thread\n", 24BEFORE start_new_thread
) = 24
rt_sigprocmask(SIG_BLOCK, ~[33], [RTMIN], 8) = 0
pipe([3, 4])                            = 0
clone(child_stack=0x14e080,
flags=CLONE_VM|CLONE_FS|CLONE_FILES|CLONE_SIGHAND) = 949
write(4, "\314V\1@\5\0\0\0\0\0\0\0\0\0\0\0\34\213\37@\10c\1@\0\0"...,
148) = 148
rt_sigprocmask(SIG_SETMASK, NULL, ~[KILL STOP 33], 8) = 0
write(4, "\0\21\3@\0\0\0\0\0\0\0\0hO\n\0\2503\23\0\377\376\373\377"...,
148) = 148
rt_sigprocmask(SIG_SETMASK, NULL, ~[KILL STOP 33], 8) = 0
rt_sigsuspend(~[KILL STOP RTMIN 33]
--------------------------------------------------------------------------
The program must be kill -9'ed at this point.

Can anyone explain this behaviour to me - did I find a bug?

Sincerely,

-- 
:M:
--------------------------------------------------------------------------
Dipl.-Inf. Michael 'Mickey' Lauer   mickey@tm.informatik.uni-frankfurt.de 
  Raum 10b - ++49 69 798 28358       Fachbereich Informatik und Biologie  
--------------------------------------------------------------------------


From barry at python.org  Wed Dec 17 09:45:40 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 09:45:48 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312170116.hBH1GId21276@c-24-5-183-134.client.comcast.net>
References: <200312170109.hBH19DG16699@oma.cosc.canterbury.ac.nz>
	<200312170116.hBH1GId21276@c-24-5-183-134.client.comcast.net>
Message-ID: <1071672339.27043.4.camel@anthem>

On Tue, 2003-12-16 at 20:16, Guido van Rossum wrote:

> Nah.  But I liked the suggestion seen here earlier of allowing
> parenthesis:
> 
> from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions,
>                                           normalize, row_ops, summary)

+1

-Barry



From jepler at unpythonic.net  Wed Dec 17 10:14:50 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Dec 17 10:15:01 2003
Subject: [Python-Dev] bug in python arm-linux?: start_new_thread fails
	after popen
In-Reply-To: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
Message-ID: <20031217151450.GB28436@unpythonic.net>

On Wed, Dec 17, 2003 at 03:01:00PM +0100, Michael Lauer wrote:
> Hi, I have a python program which works fine on x86 but doesn't work on
> any of my arm-linux devices (Zaurus, Ipaq, SIMpad) - all of them are
> running Python 2.3.2 on top of glibc 2.3.2+linuxthreads on top of kernel
> 2.4.18-rmk6-pxa3 respectively kernel 2.4.19-rmk7.

Using threads and fork together seems to be a big smelly armpit in Python.
There are also problems on redhat9, where signals in a fork+exec'd
subprocess are blocked, for instance.  This seemed to be a consequence
of blocking all signals in thread_pthread.h:PyThread_start_new_thread().
Perhaps pthread_atfork() could be used to fix this problem, though I
know next to nothing about pthreads beyond the documentation I glanced
at back when I first became aware of my thread+fork problem and before
writing this message.

Jeff

From ncoghlan at iinet.net.au  Wed Dec 17 10:22:05 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Wed Dec 17 10:22:18 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <Pine.LNX.4.58.0312171348310.911@alice>
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
	<3FE052EC.90705@iinet.net.au>
	<Pine.LNX.4.58.0312171348310.911@alice>
Message-ID: <3FE0749D.2030705@iinet.net.au>

John J Lee wrote:
> I can see that people not knowing how to spell "hierarchy" might be a
> problem here <wink>

Heh, good point. I always get confused by the place of the 'heir' in the 
'hierarchy'. I had a seriously hard time coming up with a one word name 
that suggested the right semantics for that category. If the idea gets 
any interest, it'd be nice to find a better word for that case. I was 
tempted to use __ambiguous__, though :)

Hmm, __scan__ could be an option. The other two are fairly specific 
about where to look, but category 'C' requests a scan of the package 
heirarchy.

A: from __absolute__.dotted.name import foo
B: from __relative__.dotted.name import bar
C: from __scan__.dotted.name import foobar
D: from dotted.name import barfoo

I suspect the 'as' clause would be heavily used with this construct, but 
it wouldn't be compulsory - as far as module referencing in the code 
goes, I don't think there's anything illegal about the special names.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From Boris.Boutillier at arteris.net  Wed Dec 17 11:13:19 2003
From: Boris.Boutillier at arteris.net (Boris Boutillier)
Date: Wed Dec 17 11:13:30 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <3FE052EC.90705@iinet.net.au>
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
	<3FE052EC.90705@iinet.net.au>
Message-ID: <3FE0809F.2020702@arteris.net>

Just to give you a practical example, we are using python for internal 
tools in the electronic Company I'm working for.
I needed to have a non ambiguous file lookup when using import ( We set 
external dependencies for the flow when opening a file, so you can't use 
a 'if this file exist' check, when opening a file, it must always exists)
I made a special import function, that was looking for a prefix to the 
package name:
import loc.dotted.name
import glb.dotted.name
import __.dotted.name
import std.dotted.name
import xxx.dotted.name -> raise an error

"loc", look in the current directory of the module (looking to his 
__file__ attribute)
"__ "look to the parent directory
"glb" look at the root of the user working repository ( a builtins set 
before anything else is done , users don't call python they the program 
myPython )
"std" look outside the database ( in the python installation dir mainly) 
and set back the old import behaviour to ensure that import inside 
standard modules don't fail )

I know the loc,glb __ and std names are no good for a general case as 
modules can have these names, but this is just to give you an actual use 
of these relatives imports.

Boris

Nick Coghlan wrote:

> Greg Ewing wrote:
>
>> So I think we really want *three* kinds of module reference:
>>
>>    A: Explicitly absolute
>>    B: Explicitly relative to the current module
>>    C: Searched for upwards in the package hierarchy from the current
>>       module
>>
>> (Note that C is a generalisation of the current "ambiguous"
>> references which only look in two places.)
>>
>
> Alternate spellings (plus category D, which is "Python 2.3 semantics"):
>
> A: from __absolute__.dotted.name import foo
> B: from __relative__.dotted.name import bar
> C: from __heirarchy__.dotted.name import foobar
> D: from dotted.name import barfoo
>
> I believe this spelling would only require import to handle the 
> special cases as the first 'package' name (although having the 
> compiler recognise them may be a later optimisation), and should also 
> work with straight "import __absolute__.dotted.name.foo" statements.
>
> Then Guido's migration plan would apply to the progression of the D 
> semantics from the Python 2.3 model to being a shorthand for the 
> absolute semantics.
>
> And once _that_ has happened, then we could get rid of the 
> '__absolute__' version in the name of TOOWTDI.
>
> Regards,
> Nick.
>



From skip at pobox.com  Wed Dec 17 11:14:03 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 17 11:14:03 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0B9B@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0B9B@au3010avexu1.global.avaya.com>
Message-ID: <16352.32971.627159.669571@montanaro.dyndns.org>


    >> Is funny punctuation really the right way to spell an important
    >> distinction like this?

    Tim> That's what I've been wondering. I really don't like the proposal
    Tim> to use . in various places to signify relative and absolute
    Tim> imports. It seems non-obvious both in understandability and
    Tim> readability.

Agreed.  Let's shoot for something more Pythonic.  I can imagine having to
count periods in import statements in much the way I used to have to count
closing parens in Lisp code on a CDC Cyber (way before I had ever heard of
language-sensitive editing).

Skip

From pb at nexus.co.uk  Wed Dec 17 11:39:34 2003
From: pb at nexus.co.uk (Phil Blundell)
Date: Wed Dec 17 11:39:42 2003
Subject: [Python-Dev] Re: bug in python arm-linux?: start_new_thread fails
	after popen
In-Reply-To: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
Message-ID: <20031217163934.GA13660@nexus.co.uk>

on Wed, Dec 17, 2003 at 03:01:00PM +0100, Michael Lauer wrote:
> This program - as is - just hangs in the first start_new_thread() and
> never comes back.
> If you outcomment the first call to createPipe() out, then it works
> fine.

I can't reproduce this problem on my Debian/ARM machine.  I'll try it on
the iPAQ under Familiar later.  My guess is that your glibc is too old.

p.

From theller at python.net  Wed Dec 17 11:42:14 2003
From: theller at python.net (Thomas Heller)
Date: Wed Dec 17 11:42:33 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1) -
	custom Py_Initialize()
In-Reply-To: <B15416B0-308F-11D8-B000-000A958D1666@cwi.nl> (Jack Jansen's
	message of "Wed, 17 Dec 2003 13:51:12 +0100")
References: <002301c3c29c$f2753480$0201010a@kret> <wu8vel2n.fsf@python.net>
	<B15416B0-308F-11D8-B000-000A958D1666@cwi.nl>
Message-ID: <ekv3e6qh.fsf@python.net>

Jack Jansen <Jack.Jansen@cwi.nl> writes:

> On 17-dec-03, at 12:32, Thomas Heller wrote:
>> I would rather like a function like Py_InitializeEx(struct _init*),
>> where struct _init would have components like PYTHONPATH, optimize, and
>> so on.
>
> You'd have to be very careful to design the struct in such a way that
> you'd have good compatibility.

Sure, but it can be done.
The alternative would be a Py_InitializeN() call with many parameters.
Would that be preferrable?

Thomas


From charleshixsn at earthlink.net  Wed Dec 17 11:55:40 2003
From: charleshixsn at earthlink.net (Charles Hixson)
Date: Wed Dec 17 11:59:54 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <1071509846.970.105.camel@anthem>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>	<brims1$mml$1@sea.gmane.org>
	<1071500208.970.76.camel@anthem>	<16349.53200.37613.119494@sftp.fdrake.net>	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>	<1071506413.970.102.camel@anthem>	<16349.59071.26004.502201@montanaro.dyndns.org>
	<1071509846.970.105.camel@anthem>
Message-ID: <3FE08A8C.305@earthlink.net>

Barry Warsaw wrote:

>On Mon, 2003-12-15 at 11:52, Skip Montanaro wrote:
>  
>
> ...
>
>Sorry, let me rephrase.  I'd love to ban relative imports.  Personally,
>I think all imports should be specified using fully-qualified absolute
>package paths.
>
>-Barry
>  
>
If I'm understanding this right you want to ban imports that are machine 
independant.  That can't be right, but that's what it sounds like to me.

To me an "absolute import" or a "fully-qualified absolute package path" 
sounds like you know the exact directory that the import is occuring 
from.  E.g.:  
/home/someone/projects/python/workingprogress/currentproject/thisone.py
I can see the use for being able to import that, but usually I would 
prefer to import either from something relative to the current 
directory, or relative to the python site-lib, or some such.   It's also 
quite nice to not need to bother to exactly specify just  which of those 
I will find it in.  (This is what the pythonpath is for.)

Now to me an import relative to the current directory is a relative 
import, and I assume that this is also what is meant by a local import.  
By allowing files to cluster relative to the code being written one is 
able to keep related projects and libraries together rather than just 
having all the code spread out aimlessly in a huge flat directory.  I 
don't see in what what this is inferior.

Since you didn't say why you wanted to ban relative imports, I don't 
understand your reasons.  I can't even guess at them.  Even if it were 
for some efficiency condern about hierarchical directories, that would 
merely say that the compiled modules should be stuffed into a library 
folder.  (Well, more than one...the user needs to have write permission 
to the libraries that he creates, but it's just as well that he not have 
write permissions to the system libraries...without taking special steps.)


From guido at python.org  Wed Dec 17 12:21:57 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 12:22:02 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 01:22:05 +1000."
	<3FE0749D.2030705@iinet.net.au> 
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
	<3FE052EC.90705@iinet.net.au>
	<Pine.LNX.4.58.0312171348310.911@alice> 
	<3FE0749D.2030705@iinet.net.au> 
Message-ID: <200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>

It's interesting that the "scan upwards on the package name" gets so
much support.  Back when packages were first introduced, there was a
period of time when they were a user-level construct, implemented by a
module named "ni" (after the Knights who say "Ni!").  This implemented
a full scan upward as the only way.  When I reimplemented packages in
C, I decided that the scan wasn't used much, and could actually be
detrimental (I feared that too many global modules would be
accidentally hidden), so I changed it to only looking in only two
places: the current package and the toplevel (sys.path).

But at the time it was a given that the import syntax would not be
changed.  Now that we're entertaining new import syntax, we can do
better.  How about this for Python 3.0:

  import P.Q            (1)
  from P.Q import X     (2)

would only look for P in sys.path (I'll call this "absolute import");
and

  from ...P.Q import X  (3)

would scan for P upwards from the current package.  It would require
exactly three dots.  P.Q of course stands for any dotted name.

Python 2.x would continue to use the current ("ambiguous import")
semantics for (1) and (2), but would also support (3) with the Python
3.0 semantics.

It might also be possible to set a "no ambiguous imports" flag to
switch to 3.0 semantics for (1) and (2).  This might be a __future__
statement in the package's __init__.py, e.g.

  from __future__ import absolute_import

I think this addresses all the concerns I've seen brought up in this
thread so far against my original proposal and the variations that
were subsequently suggested.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec 17 12:26:13 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 12:26:23 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
In-Reply-To: Your message of "Wed, 17 Dec 2003 06:19:51 PST."
	<1071670791.3fe0660761aa6@mcherm.com> 
References: <1071670791.3fe0660761aa6@mcherm.com> 
Message-ID: <200312171726.hBHHQDq22544@c-24-5-183-134.client.comcast.net>

[Michael Chermside]
> I disagree. Right now, having access to a class object basically
> gives one the ability to create new objects of that type. I
> think that's just fine... and I don't mind applying it to the
> file object. I'd think that the thing to do with untrusted code
> is to deny it access to the 'file' type object, thus denying it
> the ability to create new 'file's directly.
> 
> After all, it's not like file has lots of useful classmethods...
> client code which is not supposed to create new files has no
> particular need to access the 'file' object. Oh yes, except
> that it may have INSTANCES of 'file' and could access 
> f.__class__. But if it is RESTRICTED code, then the only
> legitimate use of f.__class__ is to do typechecking (it's
> arguable whether that is good design, but it does seem to be
> a legitimate use), so for restricted code we return something
> like this:
> 
>     class FakeFile:
>         def __eq__(self, other):
>             return other == file
> 
>    [...]

Are you aware of the original issue, which is that as soon as you have
a file *instance* (which might have been given to you by a very
restrictive open() variant), you can always get to the file *class*
using the __class__ attribute?  Access to the __class__ attribute is
useful for all sorts of reasons.

> Guido writes:
> > Re the capabilities discussion, this is Python 3.0 material if I ever
> > saw some
> 
> I agree. But I can dream, can't I?  <wink>

Yes, even for 3.0 this is still a dream...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From lkcl at lkcl.net  Wed Dec 17 12:56:32 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Wed Dec 17 12:56:08 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <3FE04CDD.1050109@iinet.net.au>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195856.GK17021@lkcl.net> <3FE04CDD.1050109@iinet.net.au>
Message-ID: <20031217175632.GA2117@lkcl.net>

>  the two as I understand it. I must admit I am quite curious about the 
>  fact that you seem to have plenty of time to _write_ about your 
>  proposal, but no time to listen to some quite reasonable responses.
 
 nick, thank you for pointing this out.

 that i do not explicitly acknowledge other people's point is a
 common failing of mine that has frequently got me into significant
 communication difficulties.

 yes i _do_ appreciate and understand what people say, and take the
 time to read it: i don't necessarily always actually explicitly say
 that and i really appreciate you picking up on this.

 
 if anyone else who is responding on this thread feels offended
 because they do not believe that i am listening to what they have
 to say PLEASE SAY SO.

 thanks,

 l.


From lkcl at lkcl.net  Wed Dec 17 14:06:52 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Wed Dec 17 14:06:49 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <1071669397.3fe06095c9f3c@mcherm.com>
References: <1071669397.3fe06095c9f3c@mcherm.com>
Message-ID: <20031217190652.GE2117@lkcl.net>

On Wed, Dec 17, 2003 at 05:56:37AM -0800, Michael Chermside wrote:
> Luke:
> 
> Seeing as you've now had time enough to read what I'd written, 

 yes i have now, except not yet the link you recommended to 'E',
 which i will get to, soon.


> I was going to 
> respond to some of the points that you made, but then I read Nick Coghlan's 
> posting[1], and I really don't have anything else to add.
> 
> [1] http://mail.python.org/pipermail/python-dev/2003-December/041105.html

 am absorbing that: it looks like a good definition of acls and
 capabilities to me!

> Well, that's not quite true... I guess I _do_ have something small to add.
> In designing a security policy for file systems, there are very few things
> that you can do with the protected items. Separate "read", "write", and
> "execute" permissions seem appropriate. 

 yes.
 
 it's worth pointing out that NT file systems have a lot more than 
 just rwx (which are covered by "generic read" permission, "generic
 write" and "generic execute").

 if i remember correctly, NT file system access permissions also
 include separate permissions for directory access and creation
 that are deliberately distinct from file creation (and maybe
 even file access, too, but it's been a while so i can't be sure).

 they also have "change security descriptor" as a permission.

 ... *click*.

 darn, it.  darn it quite a lot.

 it's been such a long time that i've been confusing "security
 descriptor" with "access control list".  although... thinking
 about it some more i don't think it makes much difference...


 definition of an NT (vax/vms) security descriptor:

	 a security descriptor is a set of "access control lists".

	 there are four ACLs, and they are for different purposes.

	 i can nail down two of them for you: one is a "System"
	 ACL which restricts what the OPERATING SYSTEM can do.
	 another is a "Users" ACL which restricts what _users_
	 can do (and groups, which also under NT have their own
	 identity just like a user has its own identity.)

	there is also some additional information on a per-acl
	basis and it controls the way in which the ACL is used.

 
 ... but all that is by-the-by because, i believe, only _one_
 ACL is needed for the purposes at hand.
 
 anyway, sorry to interrupt the flow, there. :)


> Furthermore, there tends to be a
> clear idea of a "user", and in many situations we want all access to items
> (files) to depend only on who that "user" is (sandboxing an email program
> is an exception... are you listening MS?). Finally, we often want to
> provide centralized administration of access rights to file systems.
> ACLs are a good match for these design specs.

 fully designed, properly designed ACLs are, imo, way over the top for
 file systems, which is why POSIX ACLs are just an extension of
 the standard POSIX filesystem rwx/ugo.

 for NT (and VAX/VMS), the design of the security system came first,
 as a "generic thing", which was then applied to file systems as
 _one_ component, albeit a critical one, of the NT and VMS operating
 systems.

 the NT/VMS security model has room for 32-bits of "permissions" on
 an ACE (access control entry).  16-bit of those are "generic", and
 16-bit of them are "application specific".

 the file-system-specific permissions were created by the writers
 of the NTFS - nt file system - and they gave them life and meaning,
 so to speak, but only in the context of an NTFS and access to it.


> However, controlling what arbitrary programs may do is a very different
> beast. 

 in the context of something like "POSIX ACLs" or in the context of
 something like "POSIX file permissions", yes, absolutely, i agree
 with you entirely.

 in the context of something as flexible as NT/VMS security descriptors,
 no, i don't agree, because NT SDs are very generic and had to be
 "specialised" to fit onto the NTFS.

 which implies that something as flexible as NT/VMS SDs could equally
 and as easily be adapted to fit controlling what arbitrary programs
 do.

 _and_, in the NT operating system and its core [mostly user-space]
 components, this _has_ already been done [and i presume likewise
 in VMS].

 for example, there is a spoolss (printer service) and there is a
 SAM (security accounts manager) database service.

 the authors / designers of these two services created their own
 set of 16-bit application-specific permissions, gave them life
 and meaning in the context of their individual services, and made
 sure that each function _in_ that service double-checked the
 permissions against the type of operation that that function
 performed.

 to give you an example, this is some pseudo-code of the SamrCreateUser
 function:

 static SEC_DES[] sec_des_array = {
	 ...
	 ...
	 samr_create_user_security_descriptor,
	 ...
 };

 STATUS32 SamrCreateUser(pHND *context, UNICODE_STRING *username)
 {
 	USER_TOK *user = GetUserToken(context);
	if (!check_security_descriptor(user, context, SECDES_SAM_CREATE_USER))
	{
		return NT_STATUS_ACCESS_DENIED;
	}
	....
	....
 }

 the context handle stores references to quite a bit of state
 information about the connection (it's a DCE/RPC thing).

 the SECDES_SAM_CREATE_USER permission is an application-specific bit
 that is specifically and only used to check access permission to
 the SamrCreateUser function.


> First of all, the kinds of restrictions that we want are HIGHLY
> variable. Sometimes we want to prohibit all file access. Sometimes we
> want to allow read-only access to a certain file only. Sometimes we
> want access only to a certain directory. Sometimes we don't care at
> all about file system access, we're concerned instead with providing
> an object which can be queried but only in certain ways -- and the
> specifics of what is permitted and what isn't are completely dependent
> on the particular application.
> 
> On top of that, there's rarely a clear concept of "users" that controls
> what actions we want to permit. Instead, we usually want to allow or
> deny certain capabilities to particular _sections of code_. (Sometimes
> that will be a module, sometimes a plug-in, sometimes a child program.)

 yes.

 and i believe that the

> All of these things suggest that maintaining a list of who 

 [or rather "what" - i.e. "Entities", yes?]

> is permitted
> to access what is simply not the best model for providing security in
> a programming language. 

 i am curious: how does that logically follow?

 in my experience (windows nt services and therefore also samba-tng's
 services), access control lists have been an extremely successful
 means to provide security in almost every single program in the
 NT operating system (i don't mean XP i don't mean windows 95) for
 almost 15 to 20 years.

 it seems quite logical therefore to propose that such a successful
 security model be adapted to fit a programming language, or at least
 to learn from its success (and failings).



 change of subject
 -----------------
 
 i believe that it _is_ possible to express what nick coghlan
 defines capabilities to be _into_ an ACL.

 how?

 by creating a security permission called "can you see it and execute it?"
 let's call it SEE_AND_EXECUTE

 then, the language (interpreter) does this (following the example of the
 SamrCreateUser function, above):

 MakeFunctionCall(Context *stack, CodeObject *code)
 {
 	char *callee_fn_name = GetCalleeName(stack);
	SEC_DES *sd = GetSecurityDescriptorForCode(code);

	/* check the capability to even _see_ this function! */
	if (!check_sec_des(sd, callee_fn_name, SEE_AND_EXECUTE))
	{
		return Exception("Function name %s does not exist",
		                 code->function_name);
	}

	/* okay, they can see it [and execute it] */

    ....
	....

 }

 does that make sense at all?

 l.


From exarkun at intarweb.us  Wed Dec 17 14:17:52 2003
From: exarkun at intarweb.us (Jp Calderone)
Date: Wed Dec 17 14:20:03 2003
Subject: [Python-Dev] Re: rexec.py unuseable
In-Reply-To: <20031217190652.GE2117@lkcl.net>
References: <1071669397.3fe06095c9f3c@mcherm.com>
	<20031217190652.GE2117@lkcl.net>
Message-ID: <20031217191752.GA5281@intarweb.us>

On Wed, Dec 17, 2003 at 07:06:52PM +0000, Luke Kenneth Casson Leighton wrote:
> [snip]
>  
>  i believe that it _is_ possible to express what nick coghlan
>  defines capabilities to be _into_ an ACL.
> 
>  how?
> 
>  by creating a security permission called "can you see it and execute it?"
>  let's call it SEE_AND_EXECUTE
> 
>  then, the language (interpreter) does this (following the example of the
>  SamrCreateUser function, above):
> 
>  MakeFunctionCall(Context *stack, CodeObject *code)
>  {
>  	char *callee_fn_name = GetCalleeName(stack);
> 	SEC_DES *sd = GetSecurityDescriptorForCode(code);
> 
> 	/* check the capability to even _see_ this function! */
> 	if (!check_sec_des(sd, callee_fn_name, SEE_AND_EXECUTE))
> 	{
> 		return Exception("Function name %s does not exist",
> 		                 code->function_name);
> 	}
> 
> 	/* okay, they can see it [and execute it] */
> 
>     ....
> 	....
> 
>  }
> 
>  does that make sense at all?
> 

  What appears to be missing here is the ability for a piece of code to take
the existing permissions which it does posess and create new permissions
within them.

  With the system you propose, if a function `foo' has permission to call
function `bar', how does foo delegate that permission to `foobar'?  If it
cannot, then it becomes very difficult to write well factored code which can
also exist within the security framework.

  Jp
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: Digital signature
Url : http://mail.python.org/pipermail/python-dev/attachments/20031217/ec981992/attachment.bin
From lkcl at lkcl.net  Wed Dec 17 14:40:32 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Wed Dec 17 14:40:05 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
Message-ID: <20031217194031.GF2117@lkcl.net>

> Are you aware of the original issue, which is that as soon as you have
> a file *instance* (which might have been given to you by a very
> restrictive open() variant), you can always get to the file *class*
> using the __class__ attribute?  Access to the __class__ attribute is
> useful for all sorts of reasons.

that is why i am proposing that ACLs be added to every "Entity"
in language - functions, modules, attributes etc.

[even if the implementation is such that they aren't _actually_
 every-object-in-python-has-another-damn-pointer]

in this way, you would have one ACL on the open() function which
restricts it to file-read-only, another ACL on the open.__class__
attribute which would be, for the sake of argument
[("all functions", DENY, ABSOLUTELY_EVERYTHING)]

etc.

l.


From lkcl at lkcl.net  Wed Dec 17 14:58:46 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Wed Dec 17 14:58:38 2003
Subject: Access Control (was Re: [Python-Dev] rexec.py unuseable)
In-Reply-To: <20031217190652.GE2117@lkcl.net>
References: <1071669397.3fe06095c9f3c@mcherm.com>
	<20031217190652.GE2117@lkcl.net>
Message-ID: <20031217195846.GG2117@lkcl.net>

okay, to recap.

jack jansen explains the difference between capabilities and ACLs:

	With capabilities you have an (unforgable) right to do something
	and no-one cares about your identity.

	with ACLs you have an unforgable identity which is checked against
	the ACL.


nick coghlan gives a definition of a capability as:

	If you can access it, you have permission to use it.
	If you aren't allowed to use it, you won't even be able to see it.
	

nick coghlan also gives a definition of an access control list as:

	"Entity X is allowed to perform action Y on other Entity Z".

where an "Entity" may be an object, a function, a user, a module,
an identity - anything - but in the context of the python language,
it is presumed that "Entity" definitely NOT include "user" or
anything remotely connected to the host operating system's concept
of "identity".


luke leighton defines "action Y" as sufficiently general to include
"seeing" as well as "doing".

	[the reason for such is that the purpose of capabilities
	 may then be perceived to be a subset of the purpose of ACLs.]


nick coghlan describes capabilities as a "post-hoc check" which he
then concludes that, by nature, the use of capabilities is therefore
fast.

mike chermside succinctly explains the need for fine-grained access
control down to code-section level, but he disagrees with the
necessity for having lists of access controls.

luke leighton endeavours to justify his opinion that capabilities are
appropriate for functional languages c but completely inappropriate for
object-orientated languages like python (the function() and the __class__
thing).

luke leighton gives a waffly explanation of the NT security descriptor
concept and provides some example usage in NT services.  he also gives
some example ways by which NT SDs could be adapted for use by the python
interpreter.

guido goes agog and type-casts this discussion as definitely python
3.0-esque.

l.

From lkcl at lkcl.net  Wed Dec 17 15:34:43 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Wed Dec 17 15:34:18 2003
Subject: [Python-Dev] Re: rexec.py unuseable
Message-ID: <20031217203443.GA3885@lkcl.net>

> 
> 
> What appears to be missing here is the ability for a piece of code to take
> the existing permissions which it does posess and create new permissions
> within them.
> 
> With the system you propose, if a function `foo' has permission to call
> function `bar', how does foo delegate that permission to `foobar'?  If it
> cannot, then it becomes very difficult to write well factored code which can
> also exist within the security framework.

jp,

i briefly touched on this in earlier messages, but have not expanded
on it since.  your question is therefore very useful.

there should exist a permission called
"sub-objects-get-a-copy-of-parent-permissions-at-sub-object-create-time"

and i believe there also exists (in NT 5.0) _yet another_ permission
that says "sub-objects-inherits-parent-permissions"

to be honest, i don't know _all_ of the internal details of the NT 5.0
enhancements to security descriptors.

if i recall correctly, the "gets-copy-of-permissions-on-create" isn't
actually a separate permission but an ACE qualifier bit - a bit like
the DENY and GRANT qualifications.

in this way, permissions can be considered to be recursive (both at
create time and at access-for-purpose-X time).

the _old_ style of permissions, NT 4.0 and below, you had to MANUALLY
add the permissions, recursively, to all sub-objects.


and incidentally, that's one reason why you can only upgrade from an
NT 4.0 NTFS to an NT 5.0 NTFS and not the other way round, because the
security descriptors are jigged around and irreversibly converted.

in that earlier message, i made a recommendation that the concept
of "recursive" application of permissions NOT be considered for
implementation (until at least the old nt4.0 style was implemented
and well understood).

but it _is_ important that the
"sub-objects-get-a-copy-of-parent-permissions-at-sub-object-create-time"
concept be implemented.

otherwise, as you say, there's no way to delegate permissions.

i'm assuming in the above description, perhaps incorrectly, that 
function 'foobar' is a member attribute of the function 'foo'?

please clarify!

l.


From Jack.Jansen at cwi.nl  Wed Dec 17 15:35:33 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Dec 17 15:35:39 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1) - custom
	Py_Initialize()
In-Reply-To: <ekv3e6qh.fsf@python.net>
References: <002301c3c29c$f2753480$0201010a@kret> <wu8vel2n.fsf@python.net>
	<B15416B0-308F-11D8-B000-000A958D1666@cwi.nl>
	<ekv3e6qh.fsf@python.net>
Message-ID: <902A3726-30D0-11D8-B275-000A27B19B96@cwi.nl>


On 17-dec-03, at 17:42, Thomas Heller wrote:

> Jack Jansen <Jack.Jansen@cwi.nl> writes:
>
>> On 17-dec-03, at 12:32, Thomas Heller wrote:
>>> I would rather like a function like Py_InitializeEx(struct _init*),
>>> where struct _init would have components like PYTHONPATH, optimize, 
>>> and
>>> so on.
>>
>> You'd have to be very careful to design the struct in such a way that
>> you'd have good compatibility.
>
> Sure, but it can be done.
> The alternative would be a Py_InitializeN() call with many parameters.
> Would that be preferrable?

I'm unsure. I'm almost tempted to say I'd like to pass in a char ** and
use environ-style (we're not really worried about performance here).
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From Jack.Jansen at cwi.nl  Wed Dec 17 15:40:30 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Wed Dec 17 15:40:38 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
	<3FE052EC.90705@iinet.net.au>
	<Pine.LNX.4.58.0312171348310.911@alice>
	<3FE0749D.2030705@iinet.net.au>
	<200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
Message-ID: <40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>


On 17-dec-03, at 18:21, Guido van Rossum wrote:

> It's interesting that the "scan upwards on the package name" gets so
> much support.

I would actually be surprised if people who say they want "scan upward"
actually want something else: they want to scan upward up to a limit.

If you have a complex package with, say, three levels you may want to
scan up through the levels of your own package. However, if your package
is later incorporated in a third party package I can hardly imagine that
you want to scan in there too.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From guido at python.org  Wed Dec 17 16:00:39 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 16:00:46 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Wed, 17 Dec 2003 21:40:30 +0100."
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl> 
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
	<3FE052EC.90705@iinet.net.au>
	<Pine.LNX.4.58.0312171348310.911@alice>
	<3FE0749D.2030705@iinet.net.au>
	<200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net> 
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl> 
Message-ID: <200312172100.hBHL0dq22885@c-24-5-183-134.client.comcast.net>

[Guido]
> > It's interesting that the "scan upwards on the package name" gets
> > so much support.

[Jack]
> I would actually be surprised if people who say they want "scan
> upward" actually want something else: they want to scan upward up to
> a limit.

There's a negation missing here, but I think I understand you.

> If you have a complex package with, say, three levels you may want
> to scan up through the levels of your own package.  However, if your
> package is later incorporated in a third party package I can hardly
> imagine that you want to scan in there too.

I think someone actually posted a use case: they were referring to a
global 3rd party module from within a package, and then later they
were moving the module into the package.  For both to work without
changes to the import statement the scan would have to go all the way
up.

If we can disregard this use case, I still like my original proposal
better, because it is more explicit: you have to use the right
number of dots, and (like Unix paths starting with ../ or ../../ etc.)
it is still completely explicit what you are referring to (assuming
the current package/directory is known).

But I expect the ... version will work just as well in practice: in
debugged code, the ... refers to a particular ancestor, so
what's above that is irrelevant; and what's below that is typically
only two or at most three levels deep -- I should think a group of
cooperating package authors who can't keep a namespace of that size
coordinated will have bigger problems...

And the advantage of the ... version is that it's more conspicuous
than a single dot (for the most common case), and you don't have to
count dots.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From neal at metaslash.com  Wed Dec 17 16:09:29 2003
From: neal at metaslash.com (Neal Norwitz)
Date: Wed Dec 17 16:09:34 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.304, 2.305
In-Reply-To: <E1AWiW7-0002WZ-00@sc8-pr-cvs1.sourceforge.net>
References: <E1AWiW7-0002WZ-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <20031217210929.GX13702@epoch.metaslash.com>

On Wed, Dec 17, 2003 at 12:43:35PM -0800, rhettinger@users.sourceforge.net wrote:
>
> Guido grants a Christmas wish:  
>   sorted() becomes a regular function instead of a classmethod.

Would sort() be a better name?  When I told someone who's done quite a
bit of java recently about sorted, he asked if it returned a bool
whether the list was sorted or not. :-)

Neal

From python at rcn.com  Wed Dec 17 16:19:27 2003
From: python at rcn.com (Raymond Hettinger)
Date: Wed Dec 17 16:19:49 2003
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.304, 2.305
In-Reply-To: <20031217210929.GX13702@epoch.metaslash.com>
Message-ID: <011701c3c4e3$741ec620$f1bd958d@oemcomputer>

[Raymond]
> > Guido grants a Christmas wish:
> >   sorted() becomes a regular function instead of a classmethod.

[Neal]
> Would sort() be a better name?  When I told someone who's done quite a
> bit of java recently about sorted, he asked if it returned a bool
> whether the list was sorted or not. :-)

I think sorted() is better:

* the context makes it clear that it's not a predicate
* the name helps distinguish it from list.sort()
* it fits with reversed()
* looking at actual code using sorted(), it reads well


my two cents,


Raymond Hettinger


From ws-news at gmx.at  Wed Dec 17 16:35:39 2003
From: ws-news at gmx.at (Werner Schiendl)
Date: Wed Dec 17 16:35:49 2003
Subject: [Python-Dev] Re: Relative import
References: <200312170109.hBH19DG16699@oma.cosc.canterbury.ac.nz>
	<200312170116.hBH1GId21276@c-24-5-183-134.client.comcast.net>
Message-ID: <brqi7g$fq8$1@sea.gmane.org>

Hi,

"Guido van Rossum" <guido@python.org> wrote
> Nah.  But I liked the suggestion seen here earlier of allowing
> parenthesis:
>
> from OPAL.Modules.Financial.Reports.COGS import (generate, webOptions,
>                                           normalize, row_ops, summary)
>

+1

/ Werner





From ws-news at gmx.at  Wed Dec 17 16:53:19 2003
From: ws-news at gmx.at (Werner Schiendl)
Date: Wed Dec 17 16:53:27 2003
Subject: [Python-Dev] Re: Relative import
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz><3FE052EC.90705@iinet.net.au><Pine.LNX.4.58.0312171348310.911@alice><3FE0749D.2030705@iinet.net.au><200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>
	<200312172100.hBHL0dq22885@c-24-5-183-134.client.comcast.net>
Message-ID: <brqj8i$i2k$1@sea.gmane.org>

"Guido van Rossum" <guido@python.org> wrote

>
> I think someone actually posted a use case: they were referring to a
> global 3rd party module from within a package, and then later they
> were moving the module into the package.  For both to work without
> changes to the import statement the scan would have to go all the way
> up.
>

Why should this sort of thing work "automatically"?
When I restructure a package's modules, I don't usually assume that all
imports will continue to work unchanged.

The main use case IMHO is to move an entire package (or self contained
module) in a package to achive some sort of required structuring or, more
important, resolve naming conflicts. This includes adding or re-naming some
organizational package names do to company merging or aquisitions.

> If we can disregard this use case, I still like my original proposal
> better, because it is more explicit: you have to use the right
> number of dots, and (like Unix paths starting with ../ or ../../ etc.)
> it is still completely explicit what you are referring to (assuming
> the current package/directory is known).
>

+1

I like your original proposal more than this one.

There should be some solution about from-less import however, to support the
fully-qualified name syntax. For example, I prefer writing "struct.pack"
over writing just "pack" because it is more explicit to the reader what
happens. Or e. g. os.path.split is another example.

Maybe just omit the dots for the name?

> But I expect the ... version will work just as well in practice: in
> debugged code, the ... refers to a particular ancestor, so
> what's above that is irrelevant; and what's below that is typically
> only two or at most three levels deep -- I should think a group of
> cooperating package authors who can't keep a namespace of that size
> coordinated will have bigger problems...

I'd prefer to explicitly specify the number of steps "up" the hierarchy.

- It prevents accidentially taking a wrong module (e. g. a "global" one)
because a module is missing.

- It prevents changing semantics because someone (else) adds a module with
that name on some other level of the hierarchy.

- Explicit is better than implicit ;-)

>
> And the advantage of the ... version is that it's more conspicuous
> than a single dot (for the most common case), and you don't have to
> count dots.
>

If there are more than 2 (or maybe 3) dots in the original proposal's use, I
think something is wrong
with package structure - not with the syntax.

And you are normally importing modules from your own package with relative
imports, so there should hardly be a need to "count the dots" I think.

/ Werner





From greg at cosc.canterbury.ac.nz  Wed Dec 17 17:36:36 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec 17 17:36:42 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <3FE052EC.90705@iinet.net.au>
Message-ID: <200312172236.hBHMaaw25148@oma.cosc.canterbury.ac.nz>

Nick Coghlan <ncoghlan@iinet.net.au>:

> A: from __absolute__.dotted.name import foo
> B: from __relative__.dotted.name import bar
> C: from __heirarchy__.dotted.name import foobar
> D: from dotted.name import barfoo
> 
> Then Guido's migration plan would apply to the progression of the D 
> semantics from the Python 2.3 model to being a shorthand for the 
> absolute semantics.
> 
> And once _that_ has happened, then we could get rid of the 
> '__absolute__' version in the name of TOOWTDI.

That's where I disagree - I think that type C semantics should
be the default/least ugly/easiest to get, not type A.

But I wouldn't like any of the others to be as ugly as those,
either.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From edcjones at erols.com  Wed Dec 17 17:41:20 2003
From: edcjones at erols.com (Edward C. Jones)
Date: Wed Dec 17 17:45:37 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
Message-ID: <3FE0DB90.9060604@erols.com>

I use Gentoo 1.4 Linux with gcc 3.2.2. The C program given below gives 
the following compiler message:

silly.c: In function `silly':
silly.c:5: warning: suggest explicit braces to avoid ambiguous `else'

I doubt if there is any bug in Py_DECREF. But would it be possible to 
jiggle the Py_DECREF macros to suppress this message? In my real code, 
I, of course, just added braces.

-----------------------
#include </usr/local/include/python2.3/Python.h>

static PyObject* silly(PyObject* obj, PyObject *args)
{
     if (1)
         Py_DECREF(Py_None);
     return NULL;
}

static PyMethodDef silly_Methods[] = {
     {"silly", silly, METH_VARARGS, ""},
     {NULL, NULL, 0, NULL}   /* sentinel */
};

void initsilly(void)
{
     (void) Py_InitModule("silly", silly_Methods);
}
-----------------------


From greg at cosc.canterbury.ac.nz  Wed Dec 17 17:50:33 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec 17 17:50:37 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
In-Reply-To: <1071670791.3fe0660761aa6@mcherm.com>
Message-ID: <200312172250.hBHMoXt25172@oma.cosc.canterbury.ac.nz>

Michael Chermside <mcherm@mcherm.com>:

> Greg:
> > In light of this, it may have been a serious mistake to
> > unify the 'file' type and the 'open' function. Is it too
> > late to back out of that decision?
> 
> Guido:
> > I think so.
> 
> I disagree. Right now, having access to a class object basically
> gives one the ability to create new objects of that type. I
> think that's just fine... and I don't mind applying it to the
> file object. I'd think that the thing to do with untrusted code
> is to deny it access to the 'file' type object, thus denying it
> the ability to create new 'file's directly.

It would be a lot better if we could get away from the idea
of a "restricted mode" in the sense of a flag somewhere that
a bunch of things have to take notice of in order to behave
securely, because that model of security is prone to springing
leaks -- as happened in a big way when new-style classes were
introduced.

The spirit behind my suggestion was to start thinking about
ways in which functionality could be separated out so that
this kind of special-casing for security purposes isn't
needed.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From jjl at pobox.com  Wed Dec 17 18:08:17 2003
From: jjl at pobox.com (John J Lee)
Date: Wed Dec 17 18:08:58 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <brqj8i$i2k$1@sea.gmane.org>
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz><3FE052EC.90705@iinet.net.au><Pine.LNX.4.58.0312171348310.911@alice><3FE0749D.2030705@iinet.net.au><200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>
	<200312172100.hBHL0dq22885@c-24-5-183-134.client.comcast.net>
	<brqj8i$i2k$1@sea.gmane.org>
Message-ID: <Pine.LNX.4.58.0312172259450.898@alice>

On Wed, 17 Dec 2003, Werner Schiendl wrote:
[...]
> There should be some solution about from-less import however, to support the
> fully-qualified name syntax. For example, I prefer writing "struct.pack"
> over writing just "pack" because it is more explicit to the reader what
> happens. Or e. g. os.path.split is another example.

Yes.


> Maybe just omit the dots for the name?

As Guido pointed out, it would be a shame to break the rule that 'import
x' currently lets you refer to plain old 'x'.  And it would be ugly to
have '.x' appear in programs, I think (and Guido mentioned with
statements, though maybe he's kidding about that).

So, why not this?

from ... import x


John

From greg at cosc.canterbury.ac.nz  Wed Dec 17 18:24:12 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec 17 18:24:24 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
Message-ID: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz>

>   import P.Q            (1)
>   from P.Q import X     (2)
> 
> would only look for P in sys.path (I'll call this "absolute import");
> and
> 
>   from ...P.Q import X  (3)
> 
> would scan for P upwards from the current package.
> 
> I think this addresses all the concerns I've seen brought up in this
> thread so far

No, it doesn't address my concern that scan-upwards semantics
should be the default, in the sense of being what should be
used in the absence of a reason to do otherwise, and should
therefore have the most straightforward syntax.

Also, you don't seem to have anything for the explicitly-relative
case, which might not be strictly necessary, but I think
I'd feel more comfortable if it were available somehow.

Thinking about the double_underscore names, it might not
be so bad if they were a bit shorter, e.g.

   A: __root__.P.Q
   B: __here__.P.Q
      __here__.__parent__.P.Q
   C: P.Q

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From jepler at unpythonic.net  Wed Dec 17 18:31:26 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Wed Dec 17 18:31:19 2003
Subject: [Python-Dev] Python threads end up blocking signals in subprocesses
In-Reply-To: <20031217151450.GB28436@unpythonic.net>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
Message-ID: <20031217233126.GA4692@unpythonic.net>

On Wed, Dec 17, 2003 at 09:14:50AM -0600, Jeff Epler wrote:
> There are also problems on redhat9, where signals in a fork+exec'd
> subprocess are blocked, for instance.  This seemed to be a consequence
> of blocking all signals in thread_pthread.h:PyThread_start_new_thread().

To follow up on my own message, here is a program that demonstrates the
problem I ran into with threads and signals.

doit() uses system() to create a process that should kill itself with a
signal nearly instantly, otherwise it sleeps for one second.  If doit()
is called from the main thread or before any threads are created, it
prints a time well under 1 second.  If it is run from a thread, it takes
just over a second, because the delivery of the signal in the subprocess
is blocked.

Typical output:
	$ python thread-signal-problem.py 
	not threaded
	Elapsed time 0.00420594215393
	subthread
	Elapsed time 1.00974297523
	main thread
	Elapsed time 0.00419104099274

import thread, time, os

def doit():
	t = time.time()
	os.system("kill $$; sleep 1")
	t1 = time.time()
	print "Elapsed time", t1-t

print "not threaded"
doit()
print "subthread"
thread.start_new(doit, ())
time.sleep(2)
print "main thread"
doit()

From tdelaney at avaya.com  Wed Dec 17 18:39:48 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 17 18:39:54 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0D23@au3010avexu1.global.avaya.com>

> From: Edward C. Jones
> 
> I use Gentoo 1.4 Linux with gcc 3.2.2. The C program given 
> below gives 
> the following compiler message:
> 
> silly.c: In function `silly':
> silly.c:5: warning: suggest explicit braces to avoid ambiguous `else'
> 
> -----------------------
> #include </usr/local/include/python2.3/Python.h>
> 
> static PyObject* silly(PyObject* obj, PyObject *args)
> {
>      if (1)
>          Py_DECREF(Py_None);

If I were you, I would pay attention to the warning.

> I, of course, just added braces.

If you mean you changed it to:

 static PyObject* silly(PyObject* obj, PyObject *args)
 {
      if (1)
      {
          Py_DECREF(Py_None);
      }

then I would say that's the right thing to do - all the time, in all code.

Tim Delaney

From skip at pobox.com  Wed Dec 17 18:46:36 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 17 18:47:14 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0D23@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0D23@au3010avexu1.global.avaya.com>
Message-ID: <16352.60124.324511.476586@montanaro.dyndns.org>


    Tim> If you mean you changed it to:

    Tim>  static PyObject* silly(PyObject* obj, PyObject *args)
    Tim>  {
    Tim>       if (1)
    Tim>       {
    Tim>           Py_DECREF(Py_None);
    Tim>       }

    Tim> then I would say that's the right thing to do - all the time, in
    Tim> all code.

Then you also agree we should bracify the Py_*REF macros, right? <wink>

Skip

From klm at zope.com  Wed Dec 17 18:57:23 2003
From: klm at zope.com (Ken Manheimer)
Date: Wed Dec 17 19:04:22 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz>
Message-ID: <Pine.LNX.4.44.0312171840290.10960-100000@slab.zope.com>

On Thu, 18 Dec 2003, Greg Ewing wrote:

> Thinking about the double_underscore names, it might not
> be so bad if they were a bit shorter, e.g.
> 
>    A: __root__.P.Q
>    B: __here__.P.Q
>       __here__.__parent__.P.Q
>    C: P.Q

Why separate __here__ and __parent__?  (I may just be confused, here.)
Aren't both about getting the relative package container?  Discussing
schemes for explicit relative and absolute imports with some
colleagues a few days ago, we liked '__pkg__' for the package
containing the current module - and __pkg__.__pkg__ to mean its'
container (and so on, for those non-programmers out there:-).  Someone
also wanted explicit expression of absolute, which we thought might be
'__python__'.

A: __python__.P.Q
B: __pkg__.P.Q
   __pkg__.__pkg__.P.Q
C: P.Q

That said, *i* much prefer the leading '.' scheme - i think the
underscores are distinctly *not* easier to read, and i'm not bothered,
anyway, by the "punctuality" of the leading-'.' - i think it stands
out enough to be noticable, and the fractured-path implication of it
conveys the relativeness...

Ken
klm@zope.com



From guido at python.org  Wed Dec 17 19:12:09 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 19:12:16 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
In-Reply-To: Your message of "Thu, 18 Dec 2003 11:50:33 +1300."
	<200312172250.hBHMoXt25172@oma.cosc.canterbury.ac.nz> 
References: <200312172250.hBHMoXt25172@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312180012.hBI0C9E23257@c-24-5-183-134.client.comcast.net>

> It would be a lot better if we could get away from the idea
> of a "restricted mode" in the sense of a flag somewhere that
> a bunch of things have to take notice of in order to behave
> securely, because that model of security is prone to springing
> leaks -- as happened in a big way when new-style classes were
> introduced.

Right.  Restricted mode currently uses both paradigms: you only have
access to the builtins that are given to you in the __builtins__ dict
-- this is pure capability stuff, and IMO it works well -- and some
builtin operations behave differently when you're in restricted mode
-- this is the ACL stuff, and Samuele revealed serious holes in it.

> The spirit behind my suggestion was to start thinking about
> ways in which functionality could be separated out so that
> this kind of special-casing for security purposes isn't
> needed.

Right.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec 17 19:20:07 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 19:20:14 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 12:24:12 +1300."
	<200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz> 
References: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312180020.hBI0K7T23311@c-24-5-183-134.client.comcast.net>

> > I think this addresses all the concerns I've seen brought up in this
> > thread so far
> 
> No, it doesn't address my concern that scan-upwards semantics
> should be the default, in the sense of being what should be
> used in the absence of a reason to do otherwise, and should
> therefore have the most straightforward syntax.

It doesn't address that becaseu I don't want that to be the default.
I'm considering adding syntax for relative imports so that the default
import syntax can be absolute only, rather than ambiguous.
IMO scan-upwards as the default is  worse than ambiguous.

> Also, you don't seem to have anything for the explicitly-relative
> case, which might not be strictly necessary, but I think
> I'd feel more comfortable if it were available somehow.

That's why I originally proposed single, double, triple (etc.) leading
dots.

> Thinking about the double_underscore names, it might not
> be so bad if they were a bit shorter, e.g.

I *really* don't like using double-underscore names here unless it was
for extremely uncommon cases.

There are two proposals that I can live with: my original proposal
with the leading dots counting the number of levels up, or the
triple-dot proposal with scan-up semantics.  In both cases, the
default semantics would switch to absolute for Python 3.0.

I cannot accept scan-up as default, and I don't think we can come up
with two separate clearly distinguishable non-default syntaxes to
separate scan-up and explicit relative semantics, so you have to pick
one there.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Wed Dec 17 19:24:02 2003
From: tim.one at comcast.net (Tim Peters)
Date: Wed Dec 17 19:24:03 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <3FE0DB90.9060604@erols.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>

[Edward C. Jones]
> I use Gentoo 1.4 Linux with gcc 3.2.2. The C program given below gives
> the following compiler message:
>
> silly.c: In function `silly':
> silly.c:5: warning: suggest explicit braces to avoid ambiguous `else'
>
> I doubt if there is any bug in Py_DECREF. But would it be possible to
> jiggle the Py_DECREF macros to suppress this message? In my real code,
> I, of course, just added braces.

Sounds good to me (i.e., add the braces and be done with it).  It's a silly
warning because Py_DECREF expands to a two-branch if/else, and there is no
ambiguity in reality.  It would be a refreshing change if someone suggested
that gcc change instead <wink>.

[Timothy Delany]
> If you mean you changed it to:
>
> static PyObject* silly(PyObject* obj, PyObject *args)
> {
>     if (1)
>     {
>         Py_DECREF(Py_None);
>     }
>
> then I would say that's the right thing to do - all the
> time, in all code.

[Skip Montanaro]
> Then you also agree we should bracify the Py_*REF macros, right?
> <wink>

That can't work, alas -- the macro has no idea whether curly braces are
syntactically legal in the context it's invoked in.  Timmy *can* know that,
and insert them or not accordingly.

This is what leads to the bletcherous

    do {
        the real macro guts go here
    } while(0)

form of macro definition.  Py_DECREF could be rewritten in that form -- but
I've puked enough for one day <splat>.


From guido at python.org  Wed Dec 17 19:24:13 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 19:24:20 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Wed, 17 Dec 2003 23:08:17 GMT."
	<Pine.LNX.4.58.0312172259450.898@alice> 
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz><3FE052EC.90705@iinet.net.au><Pine.LNX.4.58.0312171348310.911@alice><3FE0749D.2030705@iinet.net.au><200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>
	<200312172100.hBHL0dq22885@c-24-5-183-134.client.comcast.net>
	<brqj8i$i2k$1@sea.gmane.org> 
	<Pine.LNX.4.58.0312172259450.898@alice> 
Message-ID: <200312180024.hBI0ODP23337@c-24-5-183-134.client.comcast.net>

> So, why not this?
> 
> from ... import x

Yes, I forgot about that.  My original proposal allowed using just one
or more dots; the three-dots proposal with scan-up semantics should
also allow just that.  In terms of the Grammar file:

dotted_name: NAME ('.' NAME)* | '...' [NAME ('.' NAME)*]

BTW, it seems that for breaking up long imports, the parentheses
proposal has received universal acclaim.  So if we add either of the
dotted-from proposals to 2.4, we should also add the parenthesized
syntax: from ...... import (......), and for symmetry also without
from.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Wed Dec 17 19:34:55 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 17 19:34:53 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
References: <3FE0DB90.9060604@erols.com>
	<LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
Message-ID: <16352.63023.993326.798113@montanaro.dyndns.org>


    Tim> [Skip Montanaro]
    >> Then you also agree we should bracify the Py_*REF macros, right?
    >> <wink>

    Tim> That can't work, alas -- the macro has no idea whether curly braces
    Tim> are syntactically legal in the context it's invoked in.  

I meant the Py_*REF macros which expand to "if" statements and you'd bracify
the clauses, e.g., Py_DECREF would become:

#define Py_DECREF(op)                                   \
        if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
            --(op)->ob_refcnt != 0) {                   \
                _Py_CHECK_REFCNT(op)                    \
        } else {                                        \
                _Py_Dealloc((PyObject *)(op))           \
        }

Py_INCREF would be left alone, and the X variants would become

#define Py_XINCREF(op) if ((op) == NULL) {;} else { Py_INCREF(op) }
#define Py_XDECREF(op) if ((op) == NULL) {;} else {Py_DECREF(op) }

Skip

From zack at codesourcery.com  Wed Dec 17 19:36:21 2003
From: zack at codesourcery.com (Zack Weinberg)
Date: Wed Dec 17 19:36:41 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net> (Tim
	Peters's message of "Wed, 17 Dec 2003 19:24:02 -0500")
References: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
Message-ID: <87y8tbezcq.fsf@codesourcery.com>

"Tim Peters" <tim.one@comcast.net> writes:

> This is what leads to the bletcherous
>
>     do {
>         the real macro guts go here
>     } while(0)
>
> form of macro definition.  Py_DECREF could be rewritten in that form -- but
> I've puked enough for one day <splat>.

Should, not just could.  Yeah, it's bletcherous, but it's the only
way to be sure.  (Short of inline functions.)

zw

From guido at python.org  Wed Dec 17 19:41:00 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 19:41:08 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Wed, 17 Dec 2003 22:53:19 +0100."
	<brqj8i$i2k$1@sea.gmane.org> 
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz><3FE052EC.90705@iinet.net.au><Pine.LNX.4.58.0312171348310.911@alice><3FE0749D.2030705@iinet.net.au><200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>
	<200312172100.hBHL0dq22885@c-24-5-183-134.client.comcast.net> 
	<brqj8i$i2k$1@sea.gmane.org> 
Message-ID: <200312180041.hBI0f1G23377@c-24-5-183-134.client.comcast.net>

> > I think someone actually posted a use case: they were referring to a
> > global 3rd party module from within a package, and then later they
> > were moving the module into the package.  For both to work without
> > changes to the import statement the scan would have to go all the way
> > up.
> 
> Why should this sort of thing work "automatically"?
> When I restructure a package's modules, I don't usually assume that all
> imports will continue to work unchanged.
> 
> The main use case IMHO is to move an entire package (or self contained
> module) in a package to achive some sort of required structuring or, more
> important, resolve naming conflicts. This includes adding or re-naming some
> organizational package names do to company merging or aquisitions.

I agree that the use case is not very strong.  There really seem to be
two camps here; some people like scan-up semantics so much they want
to make it the default, others hate it and want to banish even the
bastardized version of it that is the current default ("ambiguous
import" or perhaps we can call it "search current then global").

> > If we can disregard this use case, I still like my original proposal
> > better, because it is more explicit: you have to use the right
> > number of dots, and (like Unix paths starting with ../ or ../../ etc.)
> > it is still completely explicit what you are referring to (assuming
> > the current package/directory is known).
> >
> 
> +1
> 
> I like your original proposal more than this one.

I like its preciseness, but I have to admit that the three dots are
enticing.

> There should be some solution about from-less import however, to support the
> fully-qualified name syntax. For example, I prefer writing "struct.pack"
> over writing just "pack" because it is more explicit to the reader what
> happens. Or e. g. os.path.split is another example.
> 
> Maybe just omit the dots for the name?

from ... import os.path

could be made to work (even though today, in a from-import statement,
the name after the import keyword cannot have dots in it).

> > But I expect the ... version will work just as well in practice: in
> > debugged code, the ... refers to a particular ancestor, so
> > what's above that is irrelevant; and what's below that is typically
> > only two or at most three levels deep -- I should think a group of
> > cooperating package authors who can't keep a namespace of that size
> > coordinated will have bigger problems...
> 
> I'd prefer to explicitly specify the number of steps "up" the hierarchy.
> 
> - It prevents accidentially taking a wrong module (e. g. a "global" one)
> because a module is missing.

But that would case pretty obvious error message, wouldn't it?

> - It prevents changing semantics because someone (else) adds a module with
> that name on some other level of the hierarchy.
> 
> - Explicit is better than implicit ;-)

You know, I'm torn on this one.  Maybe some more people can "vote" on
their preferences.

> > And the advantage of the ... version is that it's more conspicuous
> > than a single dot (for the most common case), and you don't have to
> > count dots.
> 
> If there are more than 2 (or maybe 3) dots in the original proposal's use, I
> think something is wrong
> with package structure - not with the syntax.

Right. :-)

> And you are normally importing modules from your own package with relative
> imports, so there should hardly be a need to "count the dots" I think.

Of course.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tdelaney at avaya.com  Wed Dec 17 20:09:43 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 17 20:09:49 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0D79@au3010avexu1.global.avaya.com>

> From: Skip Montanaro [mailto:skip@pobox.com]
> 
>     Tim> If you mean you changed it to:
> 
>     Tim>  static PyObject* silly(PyObject* obj, PyObject *args)
>     Tim>  {
>     Tim>       if (1)
>     Tim>       {
>     Tim>           Py_DECREF(Py_None);
>     Tim>       }
> 
>     Tim> then I would say that's the right thing to do - all 
> the time, in
>     Tim> all code.
> 
> Then you also agree we should bracify the Py_*REF macros, 
> right? <wink>

Maybe, maybe not. In this case, not bracifying the macros revealed what is IMO a poor coding practice. So you could argue that not having the braces there is a Good Thing(TM). However, I do prefer that macros be as self-contained as functions, since when reading them, they look exactly the same.

The problem in the original case is that the `if` clause does not contain the braces that it should have - and as a result, can lead to ambiguous code (in terms of readability). My opinion is that no matter what code you are writing in C, blocks should be delimited appropriately. That way there's no possibility of confusion.

Fortunately, I get to write the coding standards for my team, so I don't have to put up with anyone else's wrong ideas :)

Tim Delaney

From tdelaney at avaya.com  Wed Dec 17 20:14:27 2003
From: tdelaney at avaya.com (Delaney, Timothy C (Timothy))
Date: Wed Dec 17 20:14:33 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
Message-ID: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0D7E@au3010avexu1.global.avaya.com>

> From: Skip Montanaro
> 
> #define Py_DECREF(op)                                   \
>         if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA       \
>             --(op)->ob_refcnt != 0) {                   \
>                 _Py_CHECK_REFCNT(op)                    \
>         } else {                                        \
>                 _Py_Dealloc((PyObject *)(op))           \
>         }
> 
> Py_INCREF would be left alone, and the X variants would become
> 
> #define Py_XINCREF(op) if ((op) == NULL) {;} else { Py_INCREF(op) }
> #define Py_XDECREF(op) if ((op) == NULL) {;} else {Py_DECREF(op) }

Then you will probably end up with different warnings - semicolons following closing braces, etc.

As Tim Peters said, the way to deal with that is to enclose the entire bit in a do ( ... } while (0) no-op (after the compiler optimises it away) but that - as he rightly pointed out - is pretty blecherous.

#define Py_XINCREF(op) do { if ((op) != NULL) { Py_INCREF(op) } } while (0)
#define Py_XDECREF(op) do { if ((op) != NULL) { Py_DECREF(op) } } while (0)

or

#define Py_XINCREF(op) do { if ((op) != NULL) Py_INCREF(op) } while (0)
#define Py_XDECREF(op) do { if ((op) != NULL) Py_DECREF(op) } while (0)

Tim Delaney

From greg at cosc.canterbury.ac.nz  Wed Dec 17 20:57:24 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec 17 20:57:30 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>
Message-ID: <200312180157.hBI1vOC25998@oma.cosc.canterbury.ac.nz>

Jack Jansen <Jack.Jansen@cwi.nl>:

> If you have a complex package with, say, three levels you may want to
> scan up through the levels of your own package. However, if your package
> is later incorporated in a third party package I can hardly imagine that
> you want to scan in there too.

The main use I have in mind for scan-upwards is for when you're
importing something that's *not* part of your package. It's probably
top-level, but someone might want to put you and it into a package one
day, so you shouldn't hard-wire a top-level reference to it.

That's why I think scan-upwards should be the default -- because you
never know when someone might want to repackage you into a different
environment in the future.

Referring to related modules in your package is a different matter --
for that, you should probably be using explicitly relative references.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Wed Dec 17 21:02:33 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 21:02:39 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 14:57:24 +1300."
	<200312180157.hBI1vOC25998@oma.cosc.canterbury.ac.nz> 
References: <200312180157.hBI1vOC25998@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312180202.hBI22Xu23508@c-24-5-183-134.client.comcast.net>

> The main use I have in mind for scan-upwards is for when you're
> importing something that's *not* part of your package. It's probably
> top-level, but someone might want to put you and it into a package one
> day, so you shouldn't hard-wire a top-level reference to it.

You could hardwire a relative reference to it though.

> That's why I think scan-upwards should be the default -- because you
> never know when someone might want to repackage you into a different
> environment in the future.

I don't know if I want to encourage this kind of repackaging.  There
are various downsides (such as possible deployment of two differently
repackaged versions of the same code) and I think some of the burden
should be placed on the repackager.  After all, after repackaging, the
repackager is really responsible for the repackaged code.

The nice thing about absolute imports is that it's very simple to
write a tool that changes all absolute imports of a given package into
some other import, if you really want that.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Wed Dec 17 21:28:30 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:28:41 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <20031216141746.GA3145@burma.localdomain>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
Message-ID: <1071714510.27808.83.camel@anthem>

On Tue, 2003-12-16 at 09:17, Gustavo Niemeyer wrote:

> I have done this many times, so let me try to describe at least one
> legitimate usage case. 
> 
> A couple of weeks ago I wrote a software which needs a third
> party package to work. OTOH, since it's a small package, I don't
> want to force the user to go after the package, even because I'd
> have to ensure that my code always work with the available version
> of that package.
> 
> Thanks to the relative module importing mechanism, solving that is no
> harder than copying the third party package into my own package
> namespace.
> 
> This idea could probably be expressed in some other way, hacking
> sys.path or whatever, but I belive this is a fairly common pattern,
> and I vote for introducing a scheme to differ between local/global
> importing which would not break the current flexibility.

I guess the difference is when you're writing an application vs. writing
a library.  When writing an application (like Zope or Mailman), we
already manipulate sys.path to include some non-standard locations. 
Incorporating a third-party package into the application then is a
simple matter of arranging to have its top level directory somewhere on
sys.path.  In Mailman, I do this with the email package, and various
codecs.

When writing a library, that's harder because you're imposing
requirements on the downstream consumer of your library.

As an application developer, I don't have much sympathy for relative
imports.  As a library developer, I can see the point.

-Barry



From barry at python.org  Wed Dec 17 21:33:43 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:33:53 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
Message-ID: <1071714822.27808.88.camel@anthem>

On Tue, 2003-12-16 at 13:26, Guido van Rossum wrote:
> A lot people have presented a good case for relative imports.  Nobody
> has argued to keep the status quo (where imports are ambiguous about
> whether they are meant to be absolute or relative).  So I suggest that
> in 2.4, we introduce the leading dot notation for relative import,
> while still allowing relative import without a leading dot.  In
> 2.5 we can start warning about relative import without a leading dot
> (although that will undoubtedly get complaints from folks who have
> code that needs to work with 2.3 still).  In 3.0 we can retire
> ambiguous import.

I'll just note that where the current status quo trips /me/ up most is
when I accidentally have a local module with the same name as a global
module, and then I write an import statement expecting to get the
standard library module, but end up getting the local module.

That's why when I tend to think about this, I start wanting a way to
spell "definitely give me the global one, no matter what".  IOW, I feel
like I want a way to bypass relative module lookups.

I'm still catching up on this thread, but I wanted to throw this out
there...

-Barry



From barry at python.org  Wed Dec 17 21:44:38 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:44:49 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <200312180041.hBI0f1G23377@c-24-5-183-134.client.comcast.net>
References: <200312170022.hBH0Mne16539@oma.cosc.canterbury.ac.nz>
	<3FE052EC.90705@iinet.net.au><Pine.LNX.4.58.0312171348310.911@alice>
	<3FE0749D.2030705@iinet.net.au>
	<200312171721.hBHHLvq22523@c-24-5-183-134.client.comcast.net>
	<40E11434-30D1-11D8-B275-000A27B19B96@cwi.nl>
	<200312172100.hBHL0dq22885@c-24-5-183-134.client.comcast.net>
	<brqj8i$i2k$1@sea.gmane.org>
	<200312180041.hBI0f1G23377@c-24-5-183-134.client.comcast.net>
Message-ID: <1071715478.27808.90.camel@anthem>

On Wed, 2003-12-17 at 19:41, Guido van Rossum wrote:

> You know, I'm torn on this one.  Maybe some more people can "vote" on
> their preferences.

Don't we really need a PEP so we know what we're voting on? :)

-Barry



From barry at python.org  Wed Dec 17 21:49:36 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:49:46 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180202.hBI22Xu23508@c-24-5-183-134.client.comcast.net>
References: <200312180157.hBI1vOC25998@oma.cosc.canterbury.ac.nz>
	<200312180202.hBI22Xu23508@c-24-5-183-134.client.comcast.net>
Message-ID: <1071715775.27808.95.camel@anthem>

On Wed, 2003-12-17 at 21:02, Guido van Rossum wrote:

> I don't know if I want to encourage this kind of repackaging.

I concur.  I'm sure it's just me, but supporting this repacking use case
makes me very uncomfortable.  It feels like I'm loosing a solid
grounding about where and how Python finds modules named in import
statements.

-Barry



From barry at python.org  Wed Dec 17 21:52:03 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:52:10 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz>
References: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz>
Message-ID: <1071715922.27808.98.camel@anthem>

On Thu, 2003-12-18 at 07:24, Greg Ewing wrote:

> Thinking about the double_underscore names, it might not
> be so bad if they were a bit shorter, e.g.
> 
>    A: __root__.P.Q
>    B: __here__.P.Q
>       __here__.__parent__.P.Q
>    C: P.Q

I like this, although I'm not so sure about __here__.__parent__.  What I
like about this is the explicitness of where your search starts.

-Barry



From barry at python.org  Wed Dec 17 21:53:17 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:53:26 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <Pine.LNX.4.44.0312171840290.10960-100000@slab.zope.com>
References: <Pine.LNX.4.44.0312171840290.10960-100000@slab.zope.com>
Message-ID: <1071715997.27808.100.camel@anthem>

On Wed, 2003-12-17 at 18:57, Ken Manheimer wrote:

> That said, *i* much prefer the leading '.' scheme - i think the
> underscores are distinctly *not* easier to read, and i'm not bothered,
> anyway, by the "punctuality" of the leading-'.' - i think it stands
> out enough to be noticable, and the fractured-path implication of it
> conveys the relativeness...

Just to continue my contrarian ways, the leading-dot thingie was one of
the things that always bugged me about zcml. :)

-Barry



From barry at python.org  Wed Dec 17 21:59:47 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 21:59:57 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <3FDEDF30.1050904@iinet.net.au>
References: <Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com>
	<200312151842.hBFIg0K17763@c-24-5-183-134.client.comcast.net>
	<3FDEDF30.1050904@iinet.net.au>
Message-ID: <1071716386.27808.104.camel@anthem>

On Tue, 2003-12-16 at 05:32, Nick Coghlan wrote:

> If absolute imports were to be the only type allowed, then it would seem 
> that the only possible location for naming conflicts is in the _first_ 
> element.

True.

> So if I wanted to use two different third party modules, both of which 
> have unfortunately chosen the same name for the top-level package, the 
> only way to let them co-exist is to redo _all_ of the imports in one or 
> the other of them.
> 
> Whereas, if relative pathing is possible, I believe that all I have to 
> do is push them one level down in the package heirarchy (using distinct 
> names that I invent), and neither of them ever even knows about the 
> other's existence. I can get at both of them unambiguously, by using my 
> new top=level names, and neither package even knows that it is no longer 
> starting at the top of the import heirarchy.
> 
> Or is there some other solution being proposed to this problem, and I 
> just haven't understood it?

Has this ever happened to you in practice?

It seems like the way out would be to start adopting a Java-like
convention for package names.  The problem with that in current Python
is that you can't (easily) weave a package's contents from different
locations in the file system.

-Barry



From barry at python.org  Wed Dec 17 22:02:04 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 22:02:15 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <3FE08A8C.305@earthlink.net>
References: <000201c3c289$976fcae0$e841fea9@oemcomputer>
	<brims1$mml$1@sea.gmane.org> <1071500208.970.76.camel@anthem>
	<16349.53200.37613.119494@sftp.fdrake.net>
	<200312151603.hBFG3VT17409@c-24-5-183-134.client.comcast.net>
	<1071506413.970.102.camel@anthem>
	<16349.59071.26004.502201@montanaro.dyndns.org>
	<1071509846.970.105.camel@anthem>  <3FE08A8C.305@earthlink.net>
Message-ID: <1071716523.27808.107.camel@anthem>

On Wed, 2003-12-17 at 11:55, Charles Hixson wrote:
> Barry Warsaw wrote:

> >Sorry, let me rephrase.  I'd love to ban relative imports.  Personally,
> >I think all imports should be specified using fully-qualified absolute
> >package paths.
> >

> If I'm understanding this right you want to ban imports that are machine 
> independant.  That can't be right, but that's what it sounds like to me.

Nope, because "absolute package paths" are always calculated relative to
directories in sys.path when mapping a package path to a file system
path.

-Barry



From andrew-pythondev at puzzling.org  Wed Dec 17 22:04:09 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed Dec 17 22:04:15 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <1071714822.27808.88.camel@anthem>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
Message-ID: <20031218030409.GG473@frobozz>

On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:
> 
> I'll just note that where the current status quo trips /me/ up most is
> when I accidentally have a local module with the same name as a global
> module, and then I write an import statement expecting to get the
> standard library module, but end up getting the local module.
> 
> That's why when I tend to think about this, I start wanting a way to
> spell "definitely give me the global one, no matter what".  IOW, I feel
> like I want a way to bypass relative module lookups.

Alternatively, maybe what you want is a way to say "definitely give me the
standard library one, no matter what", e.g.

    from stdlib import codecs

This would allow other packages to have their own codecs module/package
without worrying about confusing import errors.  To make this backwards
compatible, though, we'd probably need the equivalent of C++'s "using
namespace" statement, with no declaration being equivalent to:

    using namespace stdlib

Or perhaps more python-like:

    __import_base__ = 'stdlib'

Python 3.0 might default to __import_base__ = '', to force people to be
explicit.

Actually, the proposed with statement could be used here, e.g.

    import stdlib
    with stdlib:
        from . import codecs, user, gc

looks reasonably clear to me, and for long package paths, it could be very
convenient:

    import zope.app.interfaces
    with zope.app.interfaces:
        from .location import ILocation
        from .foo import IFoo, IBar
        # etc

-Andrew.


From barry at python.org  Wed Dec 17 22:05:03 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 22:05:12 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0B9B@au3010avexu1.global.avaya.com>
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE0B9B@au3010avexu1.global.avaya.com>
Message-ID: <1071716702.27808.111.camel@anthem>

On Tue, 2003-12-16 at 22:20, Delaney, Timothy C (Timothy) wrote:

> That's what I've been wondering. I really don't like the proposal to
> use . in various places to signify relative and absolute imports. It
> seems non-obvious both in understandability and readability.

I agree.  That's why I like something like __root__.foo.bar and
__here__.foo.bar.  Totally explicit and will avoid deep groove digging
in the skull trying to remember if a leading dot means it should be
absolute or relative. :)

-Barry



From tim.one at comcast.net  Wed Dec 17 22:06:13 2003
From: tim.one at comcast.net (Tim Peters)
Date: Wed Dec 17 22:06:14 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <1071714822.27808.88.camel@anthem>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEKAHNAB.tim.one@comcast.net>

[Barry]
> ...
> I'll just note that where the current status quo trips /me/ up most is
> when I accidentally have a local module with the same name as a global
> module, and then I write an import statement expecting to get the
> standard library module, but end up getting the local module.

Indeed, this is common on c.l.py and the Tutor lists, with a twist:  someone
writes a module in their *current* directory, with a name like random.py,
and then "Python breaks mysteriously all over the place", because every
import of random ends up with something utterly unexpected.  For some
reason, "random.py" seems most often to be the actual name used in these
cases.

Possibly related:  I remember pissing away a day a year or so ago trying to
track down a segfault in a Zope test that happened only when I ran the test
on my McLean box.  In the end, it turned out that the test in question was
(after untangling the usual layers of Zopish indirection <wink/grrrr>)
deliberately trying to import a module that didn't exist, with an oddball
name.  It just so happened that somebody had whittled down a segfaulting
Python program and uploaded it to SourceForge, with the very same oddball
name.  I had downloaded that program into my Python build directory, and
Zope kept provoking its segfault as a side effect of (unintentionally!)
importing it.

In large part, I've come to loathe imports -- I never have the foggiest idea
how they're going to get satisfied in code I didn't write, and especially in
code that plays every trick in (and out) of the book to hide what it's
really doing with sys.path mutations, import hooks, poking stuff by magic
into sys.modules via well-hidden side effects in scattered __init__.py
files, ... it makes Java's rigid, wordy, explicit import scheme look very
attractive some days.  That you can't trick that into doing arbitrarily
unexpected things by magic at runtime isn't entirely a bad thing <wink>.


From barry at python.org  Wed Dec 17 22:08:21 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 17 22:08:28 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031218030409.GG473@frobozz>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>  <20031218030409.GG473@frobozz>
Message-ID: <1071716900.27808.114.camel@anthem>

On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:
> On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:
> > 
> > That's why when I tend to think about this, I start wanting a way to
> > spell "definitely give me the global one, no matter what".  IOW, I feel
> > like I want a way to bypass relative module lookups.
> 
> Alternatively, maybe what you want is a way to say "definitely give me the
> standard library one, no matter what", e.g.
> 
>     from stdlib import codecs

Interesting.  I see the subtle distinction.  One searches only the
standard library, the other searches every directory on sys.path.  I'm
not sure I'd have much need to restrict the search to just the standard
library.  Plus, would that include site-packages?

-Barry



From andrew-pythondev at puzzling.org  Wed Dec 17 22:16:45 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Wed Dec 17 22:16:51 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <1071716900.27808.114.camel@anthem>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem>
Message-ID: <20031218031645.GH473@frobozz>

On Wed, Dec 17, 2003 at 10:08:21PM -0500, Barry Warsaw wrote:
> On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:
> > On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:
> > > 
> > > That's why when I tend to think about this, I start wanting a way to
> > > spell "definitely give me the global one, no matter what".  IOW, I feel
> > > like I want a way to bypass relative module lookups.
> > 
> > Alternatively, maybe what you want is a way to say "definitely give me the
> > standard library one, no matter what", e.g.
> > 
> >     from stdlib import codecs
> 
> Interesting.  I see the subtle distinction.  One searches only the
> standard library, the other searches every directory on sys.path.  I'm
> not sure I'd have much need to restrict the search to just the standard
> library.  Plus, would that include site-packages?

I should have been clearer.  What I'm basically proposing is to make
site-package called "stdlib", leaving a default python install with a very
uncluttered top-level namespace, thus reducing unexpected name conflicts.

Otherwise, this idea doesn't require any modification to import semantics,
except for some magic to make stdlib the default root for imports for
backwards compatibility.  Thinking about it a bit harder, though, I'm not
sure it's as easy as I initially thought, because this would then make
importing 3rd party packages not work, i.e. currently you can do:

    import urllib
    import pygtk

without problems, because they are both top-level.  If urllib moved into a
"stdlib" package, it's hard to imagine a clean way where old code that does
those two lines would still work.

I still like the idea of having the standard library in its own package,
but I guess that will have to wait until Python 3.0.

-Andrew.


From greg at cosc.canterbury.ac.nz  Wed Dec 17 23:21:24 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec 17 23:21:30 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180020.hBI0K7T23311@c-24-5-183-134.client.comcast.net>
Message-ID: <200312180421.hBI4LNb27278@oma.cosc.canterbury.ac.nz>

Guido:

> I cannot accept scan-up as default

In that case, it's probably better not to have it at all,
and I'd go for your first proposal. One leading dot seems
to stand out pretty well to my eyes -- more is not necessary.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From tim at multitalents.net  Wed Dec 17 23:43:45 2003
From: tim at multitalents.net (Tim Rice)
Date: Wed Dec 17 23:43:52 2003
Subject: [Python-Dev] 2.3.3 portability audit
Message-ID: <Pine.UW2.4.58.0312172031350.26074@ou8.int.multitalents.net>


A few things I found in the release23-maint branch.

python-2.3/src/Doc/tools/checkargs.pm:#!/uns/bin/perl
    probably a typo.                     ^^^

While most python scripts use #!/usr/bin/env python
this one has #!/bin/env
python-2.3/src/Lib/bsddb/dbshelve.py:#!/bin/env python

Most scripts that don't use #!/usr/bin/env python
have #! /usr/local/bin/python
These 5 have #!/usr/bin/python
python-2.3/src/Lib/idlelib/idle:#!/usr/bin/python
python-2.3/src/Lib/idlelib/idle.py:#!/usr/bin/python
python-2.3/src/Lib/test/test_bz2.py:#!/usr/bin/python
python-2.3/src/Lib/test/test_optparse.py:#!/usr/bin/python
python-2.3/src/Mac/Contrib/Tabcleaner/Tabcleaner.py:#!/usr/bin/python
Intentional?

python-2.3/src/Lib/test/test_largefile.py:#!python
This may be intentional, I don't know.


python-2.3/src/Lib/test/test_pep263.py:#! -*- coding: koi8-r -*-
Is this ! supposed to be there?         ^
I think not. (could be wrong)

python-2.3/src/Doc/tools/makesec.sh:#!/bin/bash
Bash doesn't come standard on all supported platforms.
The only bashism I saw with a quick look was the last line.
rm -f lib.{aux,log}


-- 
Tim Rice				Multitalents	(707) 887-1469
tim@multitalents.net



From guido at python.org  Wed Dec 17 23:48:08 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 23:48:20 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 17:21:24 +1300."
	<200312180421.hBI4LNb27278@oma.cosc.canterbury.ac.nz> 
References: <200312180421.hBI4LNb27278@oma.cosc.canterbury.ac.nz> 
Message-ID: <200312180448.hBI4m8J23710@c-24-5-183-134.client.comcast.net>

> > I cannot accept scan-up as default
> 
> In that case, it's probably better not to have it at all,
> and I'd go for your first proposal. One leading dot seems
> to stand out pretty well to my eyes -- more is not necessary.

But you're okay with two or more dots for extra levels up, right?

(Note that this leading-dot algorithm is nearly the same as the old
MacOS pathname syntax, except they did it with colons: :foo was foo in
the current directory, ::foo was foo in the parent directory, etc.,
and foo:bar was an absolute pathname.  The only difference is that for
backwards compatibility reasons they interpreted no colons as relative
to the current directory rather than absolute.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Wed Dec 17 23:53:18 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Wed Dec 17 23:53:23 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180448.hBI4m8J23710@c-24-5-183-134.client.comcast.net>
Message-ID: <200312180453.hBI4rIg27605@oma.cosc.canterbury.ac.nz>

Guido:

> But you're okay with two or more dots for extra levels up, right?

Yes, that's fine.

> (Note that this leading-dot algorithm is nearly the same as the old
> MacOS pathname syntax

I noticed that, too. Also I think VMS did something similar
with dots in its pathnames. So there are precedents...

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From guido at python.org  Wed Dec 17 23:55:03 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 17 23:55:13 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Wed, 17 Dec 2003 21:33:43 EST."
	<1071714822.27808.88.camel@anthem> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net> 
	<1071714822.27808.88.camel@anthem> 
Message-ID: <200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>

> I'll just note that where the current status quo trips /me/ up most is
> when I accidentally have a local module with the same name as a global
> module, and then I write an import statement expecting to get the
> standard library module, but end up getting the local module.

This can be interpreted in two ways though.  You could be unfortunate
enough to create a module in your package that has the same name as a
standard module you also need to import from inside the same package
-- that's currently impossible.  But the most common variant I've seen
is what Tim Peters describes: having a module in the current directory
that accidentally hides a standard module.

The latter case does not involve packages and will not be fixed by any
import syntax -- it can only be fixed by changing the default sys.path
*not* to include the current directory, just like over the last decade
or so Unix shells have stopped putting '.' in $PATH.

But that has other downsides: it would be a lot harder to begin
writing a program broken up into modules.  Hmm, maybe the current
directory should be last?  That way you can import your own modules
except if they clash with the standard library.  Hm, but there are so
many standard library modules that it might still cause frequent
complaints from mystified beginners.

I'm beginning to agree (uselessly) with Tim: import itself is a
problem. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From greg at cosc.canterbury.ac.nz  Thu Dec 18 00:04:37 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 00:04:43 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
Message-ID: <200312180504.hBI54ba27649@oma.cosc.canterbury.ac.nz>

Guido:

> Hmm, maybe the current directory should be last?  That way you can
> import your own modules except if they clash with the standard
> library.  Hm, but there are so many standard library modules that it
> might still cause frequent complaints from mystified beginners.

Maybe in Python 3.0 the current directory should appear as a
package with some standard name. Then applications could use
relative imports to import their own modules, and absolute
ones to import standard modules, without fear of conflict.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From andrew-pythondev at puzzling.org  Thu Dec 18 00:10:21 2003
From: andrew-pythondev at puzzling.org (Andrew Bennetts)
Date: Thu Dec 18 00:10:31 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
Message-ID: <20031218051021.GI473@frobozz>

On Wed, Dec 17, 2003 at 08:55:03PM -0800, Guido van Rossum wrote:
[...]
> -- that's currently impossible.  But the most common variant I've seen
> is what Tim Peters describes: having a module in the current directory
> that accidentally hides a standard module.
> 
> The latter case does not involve packages and will not be fixed by any
> import syntax -- it can only be fixed by changing the default sys.path
> *not* to include the current directory, just like over the last decade
> or so Unix shells have stopped putting '.' in $PATH.

Or it can be fixed by not having the entire standard library at the
toplevel, but instead inside a single "stdlib" package.  That way 3rd party
packages could never conflict with the standard library (unless they were
crazy enough to call one of their own modules "stdlib"...).

This way, no matter how large the standard library grows, the potential for
module name conflicts doesn't grow.  It'd probably simplify things slightly,
too -- the site-packages directory could go away entirely, and all packages,
standard or not, would be on an equal footing.  The standard library would
be no more special than, say, Zope.  My /usr/lib/python would just contain

    stdlib/
    twisted/
    zope/
    Numeric/

etc.

The backwards compatibility problems probably means this isn't feasible
until Python 3.0, though :(

-Andrew.


From fdrake at acm.org  Thu Dec 18 00:31:20 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec 18 00:31:32 2003
Subject: [Python-Dev] 2.3.3 portability audit
In-Reply-To: <Pine.UW2.4.58.0312172031350.26074@ou8.int.multitalents.net>
References: <Pine.UW2.4.58.0312172031350.26074@ou8.int.multitalents.net>
Message-ID: <16353.15272.514696.150572@sftp.fdrake.net>


Tim Rice writes:
 > python-2.3/src/Doc/tools/checkargs.pm:#!/uns/bin/perl
 >     probably a typo.                     ^^^
...
 > python-2.3/src/Doc/tools/makesec.sh:#!/bin/bash
 > Bash doesn't come standard on all supported platforms.
 > The only bashism I saw with a quick look was the last line.
 > rm -f lib.{aux,log}

I've fixed these two on the 2.3 maintenance branch and the trunk.
Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From tim.one at comcast.net  Thu Dec 18 00:38:57 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Dec 18 00:38:59 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180448.hBI4m8J23710@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEKKHNAB.tim.one@comcast.net>

[Guido]
> ...
> (Note that this leading-dot algorithm is nearly the same as the old
> MacOS pathname syntax, except they did it with colons: :foo was foo in
> the current directory, ::foo was foo in the parent directory, etc.,
> and foo:bar was an absolute pathname.  The only difference is that for
> backwards compatibility reasons they interpreted no colons as relative
> to the current directory rather than absolute.)

More relevant to most of the world <wink> is that it's even more similar to
the pathname syntax in the Win9x/ME command.com shells:

C:\Code\python\PCbuild>cd ...\Zope

C:\Code\Zope>

I expect that basing Python syntax on obsolete, hated OSes is a smart
contrarian move <wink>>


From barry at python.org  Thu Dec 18 01:49:54 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 18 01:50:04 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
Message-ID: <1071730193.17717.4.camel@anthem>

On Wed, 2003-12-17 at 23:55, Guido van Rossum wrote:

> This can be interpreted in two ways though.  You could be unfortunate
> enough to create a module in your package that has the same name as a
> standard module you also need to import from inside the same package
> -- that's currently impossible.  

Right, and I'd like to make that possible.  The least disruptive way I
see of doing that is to introduce a magic root package from which things
can be imported from.  E.g. import __root__.re

> But the most common variant I've seen
> is what Tim Peters describes: having a module in the current directory
> that accidentally hides a standard module.

Yep, I've been bit by that too.

> The latter case does not involve packages and will not be fixed by any
> import syntax -- it can only be fixed by changing the default sys.path
> *not* to include the current directory, just like over the last decade
> or so Unix shells have stopped putting '.' in $PATH.

Yep.  I haven't had '.' in my $PATH for a bazillion years. :)

> But that has other downsides: it would be a lot harder to begin
> writing a program broken up into modules.  Hmm, maybe the current
> directory should be last?  That way you can import your own modules
> except if they clash with the standard library.  Hm, but there are so
> many standard library modules that it might still cause frequent
> complaints from mystified beginners.
> 
> I'm beginning to agree (uselessly) with Tim: import itself is a
> problem. :-)

There ya go!  That's thinking outside the box: let's deprecate import
altogether :)

-Barry



From barry at python.org  Thu Dec 18 01:52:32 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 18 01:52:43 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180504.hBI54ba27649@oma.cosc.canterbury.ac.nz>
References: <200312180504.hBI54ba27649@oma.cosc.canterbury.ac.nz>
Message-ID: <1071730352.17717.7.camel@anthem>

On Thu, 2003-12-18 at 13:04, Greg Ewing wrote:

> Maybe in Python 3.0 the current directory should appear as a
> package with some standard name. Then applications could use
> relative imports to import their own modules, and absolute
> ones to import standard modules, without fear of conflict.

In a previous thread, we already came up with __me__ as a magic name
referring to the current module.  Let's use __my__ as the prefix for
importing a relative module.

too-tired-to-wink-ly y'rs,
-Barry



From art at wiktorsadowski.com  Thu Dec 18 02:38:07 2003
From: art at wiktorsadowski.com (Wiktor Sadowski)
Date: Thu Dec 18 02:37:46 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (release candidate 1) - custom
	Py_Initialize() 
Message-ID: <028401c3c539$e1e3db40$0201010a@kret>

Thomas,consider this:
specialized_python.exe  --path/foo.px (command line)
where foo.px is zipped ctypes-venster program -modules,some
code,binaries(images etc.)
specialized_python.exe could be statically linked against C ctypes,and could
contain ctypes and venster python code (freezed)
(specialized_python.exe could also work as "regular" python.exe and
successfully run ".py" , ".pyc" files)
Now associate ".px" with specialized_python.exe and you have clickable
packed python programs.

To build such specialized_python.exe you would need your own
PyRun_*NotSoSimple*FileExFlags , new Py_Main,
modified Py_Initialize and probably new or modified import engine.
And with proposed addition specialized_python.exe could use existing Python
shared library!
I'm afraid it wouldn't be possible with Py_InitializeEx, you are proposing.

For one of our clients we created pymozilla.exe to run packed
websites(zipped html,xml,css,images and python "cgi" files)
It's pretty cool. :-)

Current Py_Initialize() is ok for distributed Python framework , but it's
a pain if you'd like to use the language and not necessary each
aspect of the framework.

Why not make Python shared library more usable?

Wiktor


From fincher.8 at osu.edu  Thu Dec 18 03:51:18 2003
From: fincher.8 at osu.edu (Jeremy Fincher)
Date: Thu Dec 18 02:53:50 2003
Subject: [Python-Dev] RE: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.304, 2.305
In-Reply-To: <011701c3c4e3$741ec620$f1bd958d@oemcomputer>
References: <011701c3c4e3$741ec620$f1bd958d@oemcomputer>
Message-ID: <200312180351.18465.fincher.8@osu.edu>

On Wednesday 17 December 2003 04:19 pm, Raymond Hettinger wrote:
> I think sorted() is better:
>
> * the context makes it clear that it's not a predicate
> * the name helps distinguish it from list.sort()
> * it fits with reversed()
> * looking at actual code using sorted(), it reads well

sorted() returns a list, and reversed() returns an iterator.  That seems like 
a recipe for confusion, *especially* because they "fit" together and do 
completely different things.

Jeremy

From martin at v.loewis.de  Wed Dec 17 16:30:23 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 18 02:57:30 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031216212312.GL17021@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
	<20031216212312.GL17021@lkcl.net>
Message-ID: <m3ptenrv2o.fsf@mira.informatik.hu-berlin.de>

Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:

>  all i can do is recommend a framework and some guidelines on what
>  conventions could be fitted over that framework.

In that case, your proposal will likely die.

I don't think the framework can provide the feature that rexec.py
originally provided. To prove that, I would need a complete proposal
how precisely what ACLs are set on what objects, and how exactly I
invoke code for restricted execution (i.e. what API do I call in what
order). As you won't provide such a proposal, and as nobody else might,
either, it is likely that this gets us nowhere.

Regards,
Martin

From tim.one at comcast.net  Thu Dec 18 03:21:28 2003
From: tim.one at comcast.net (Tim Peters)
Date: Thu Dec 18 03:21:29 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCOELHHNAB.tim.one@comcast.net>

[Guido]
> ...
> I'm beginning to agree (uselessly) with Tim: import itself is a
> problem. :-)

Not useless at all -- that's bonding!  Technically useless but humanly
essential <wink>.

It's no mystery *why* import is "a problem", but I'm not sure insight
implies a cure:  it's one of the very few places in Python where you can't
explicitly say what you intend.  Even in an inheritance hierarchy from hell,
if I really don't want to chance that please.hit_me() will get satisfied
from a class other than BlackJack, I can force it easily --

    BlackJack.hit_me(please)

There aren't any names to spell "the roots" of all the places Python may
search to satisfy an import, so no way either to be explicit about intent.
In contrast, nobody can mistake what

    import java.util.Random;

intends.  It's not a deep thing, it's just being able to name the intended
context.  I confess I'm keener on that than on growing more ways to spell
more kinds of searches.


From lkcl at lkcl.net  Thu Dec 18 04:29:22 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Thu Dec 18 04:29:36 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3ptenrv2o.fsf@mira.informatik.hu-berlin.de>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
	<20031216212312.GL17021@lkcl.net>
	<m3ptenrv2o.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031218092922.GF6324@lkcl.net>

On Wed, Dec 17, 2003 at 10:30:23PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> >  all i can do is recommend a framework and some guidelines on what
> >  conventions could be fitted over that framework.
> 

martin, to clarify: "all i can do" is the wrong preamble phrase.
"i would like to" is better.  i was trying to be ... self-denigrating,
over-careful, something like that.


> originally provided. To prove that, I would need a complete proposal
> how precisely what ACLs are set on what objects, and how exactly I
> invoke code for restricted execution (i.e. what API do I call in what
> order). 

 what api, in what order, what code is invoked in (inside the python
 executable), _how_ ACLs are set on what objects, yes.

 _what_ acls are set on what objects is conditional on whether i
 receive funding to do so, or whether some other people can be of
 significant assistance.  why?  because there's a lot of them.

 what i was trying to say above, "all i can do is recommend a framework"
 is build up to an idea of adding in a framework, similar to
 Exceptions, by which ANY generically-defined restriction system
 can be plugged in to the python language.

 the idea being that if no such a system is not plugged in,
 the performance penalty on python is insignificant, and no user-code
 is restricted.


 however, this may all turn out to be unnecessary [quote from greg
 ewing]:

> The spirit behind my suggestion was to start thinking about
> ways in which functionality could be separated out so that
> this kind of special-casing for security purposes isn't
> needed.

 so, with the correct codebase reordering, a simple capabilities
 based system can be added, the problem goes away.

 yes?

 l.


From martin at v.loewis.de  Wed Dec 17 17:59:09 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 18 04:44:03 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <3FE0DB90.9060604@erols.com>
References: <3FE0DB90.9060604@erols.com>
Message-ID: <m38ylbrqyq.fsf@mira.informatik.hu-berlin.de>

"Edward C. Jones" <edcjones@erols.com> writes:

> silly.c: In function `silly':
> silly.c:5: warning: suggest explicit braces to avoid ambiguous `else'
> 
> I doubt if there is any bug in Py_DECREF. But would it be possible to
> jiggle the Py_DECREF macros to suppress this message?

It would be possible. Can you please submit a bug report? Or, better
yet, a patch?

Regards,
Martin


From anthony at interlink.com.au  Thu Dec 18 04:50:50 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Dec 18 04:51:09 2003
Subject: [Python-Dev] 2.3.3 and beyond
Message-ID: <200312180950.hBI9ooi1005762@localhost.localdomain>

This should hopefully be no suprise to anyone, but I'm going to be
cutting 2.3.3 final in about 12-15 hours time. Once that's done, I'll
be handing the branch over to Jack so he can do his Macintosh magic.

I _had_ hoped to spend a lot of time getting various platform issues
resolved for 2.3.3 - this didn't end up happening (for reasons that
ryhme with "junkies are scum"). Once Jack's done his Mac magic, I'll
try to throw some effort at this for 2.3.4. I'm planning a 2.3.4 for
around the May timeframe, assuming Tim doesn't create more stinking
critical bugs in the meantime. (Oh, sure, he'll _claim_ the bugs were
always there, and he was just fixing them... <wink>)

Anthony
--
Anthony Baxter     <anthony@interlink.com.au>
It's never too late to have a happy childhood.

From ncoghlan at iinet.net.au  Thu Dec 18 05:42:02 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Dec 18 05:42:09 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <1071716386.27808.104.camel@anthem>
References: <Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com>	<200312151842.hBFIg0K17763@c-24-5-183-134.client.comcast.net>	<3FDEDF30.1050904@iinet.net.au>
	<1071716386.27808.104.camel@anthem>
Message-ID: <3FE1847A.7030104@iinet.net.au>

Barry Warsaw wrote:
> On Tue, 2003-12-16 at 05:32, Nick Coghlan wrote:
>> [snip description of potential naming conflicts]
> Has this ever happened to you in practice?

Well, no. But others seemed concerned about it, so I thought I'd try to 
clarify the problem - the scripts I write don't need anything more than 
the standard library and win32all.

> It seems like the way out would be to start adopting a Java-like
> convention for package names.  The problem with that in current Python
> is that you can't (easily) weave a package's contents from different
> locations in the file system.

I still like the various suggestions for _giving_ those parts of the 
system special names. That's what I was trying for, even if the names I 
chame up with weren't any good.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From ncoghlan at iinet.net.au  Thu Dec 18 05:58:42 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Thu Dec 18 05:58:48 2003
Subject: open == file considered harmful (Re: [Python-Dev]
	RE:	rexec.pyunuseable)
In-Reply-To: <200312180012.hBI0C9E23257@c-24-5-183-134.client.comcast.net>
References: <200312172250.hBHMoXt25172@oma.cosc.canterbury.ac.nz>
	<200312180012.hBI0C9E23257@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE18862.50208@iinet.net.au>

Guido van Rossum wrote:

>>It would be a lot better if we could get away from the idea
>>of a "restricted mode" in the sense of a flag somewhere that
>>a bunch of things have to take notice of in order to behave
>>securely, because that model of security is prone to springing
>>leaks -- as happened in a big way when new-style classes were
>>introduced.
> 
> 
> Right.  Restricted mode currently uses both paradigms: you only have
> access to the builtins that are given to you in the __builtins__ dict
> -- this is pure capability stuff, and IMO it works well -- and some
> builtin operations behave differently when you're in restricted mode
> -- this is the ACL stuff, and Samuele revealed serious holes in it.

What if instead of 'builtin behaves differently in restricted mode' we 
had 'restricted __builtins__ contains a DIFFERENT builtin, that happens 
to have the same name'?

That is, in addition to the ability to simply deny access to a specific 
builtin function or class, there was the ability to _replace_ one before 
giving it to the restricted code.

Regards,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From paoloinvernizzi at dmsware.com  Thu Dec 18 06:03:38 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Thu Dec 18 06:03:45 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOELHHNAB.tim.one@comcast.net>
References: <200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
	<LNBBLJKPBEHFEDALKOLCOELHHNAB.tim.one@comcast.net>
Message-ID: <brs1ia$p2b$1@sea.gmane.org>

Tim Peters wrote:

> In contrast, nobody can mistake what
> 
>     import java.util.Random;
> 
> intends.  It's not a deep thing, it's just being able to name the intended
> context.  I confess I'm keener on that than on growing more ways to spell
> more kinds of searches.

I *strongly* agree!!!

---
Paolo Invernizzi


From mwh at python.net  Thu Dec 18 08:00:25 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 18 08:00:34 2003
Subject: [Python-Dev] bug in python arm-linux?: start_new_thread fails
	after popen
In-Reply-To: <20031217151450.GB28436@unpythonic.net> (Jeff Epler's message
	of "Wed, 17 Dec 2003 09:14:50 -0600")
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
Message-ID: <2my8taffh2.fsf@starship.python.net>

Jeff Epler <jepler@unpythonic.net> writes:

> On Wed, Dec 17, 2003 at 03:01:00PM +0100, Michael Lauer wrote:
>> Hi, I have a python program which works fine on x86 but doesn't work on
>> any of my arm-linux devices (Zaurus, Ipaq, SIMpad) - all of them are
>> running Python 2.3.2 on top of glibc 2.3.2+linuxthreads on top of kernel
>> 2.4.18-rmk6-pxa3 respectively kernel 2.4.19-rmk7.
>
> Using threads and fork together seems to be a big smelly armpit in Python.
> There are also problems on redhat9, where signals in a fork+exec'd
> subprocess are blocked, for instance.

This is in no way restricted to RH9...

> This seemed to be a consequence of blocking all signals in
> thread_pthread.h:PyThread_start_new_thread().  Perhaps
> pthread_atfork()

Perhaps.

Cheers,
mwh

-- 
  I have no disaster recovery plan for black holes, I'm afraid.  Also
  please be aware that if it one looks imminent I will be out rioting
  and setting fire to McDonalds (always wanted to do that) and
  probably not reading email anyway.                     -- Dan Barlow

From barry at python.org  Thu Dec 18 08:00:29 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 18 08:00:44 2003
Subject: [Python-Dev] Re: Christmas Wishlist
In-Reply-To: <3FE1847A.7030104@iinet.net.au>
References: <Pine.LNX.4.44.0312151237080.10960-100000@slab.zope.com>
	<200312151842.hBFIg0K17763@c-24-5-183-134.client.comcast.net>
	<3FDEDF30.1050904@iinet.net.au> <1071716386.27808.104.camel@anthem>
	<3FE1847A.7030104@iinet.net.au>
Message-ID: <1071752429.17717.24.camel@anthem>

On Thu, 2003-12-18 at 05:42, Nick Coghlan wrote:
> Barry Warsaw wrote:
> > On Tue, 2003-12-16 at 05:32, Nick Coghlan wrote:
> >> [snip description of potential naming conflicts]
> > Has this ever happened to you in practice?
> 
> Well, no. But others seemed concerned about it, so I thought I'd try to 
> clarify the problem - the scripts I write don't need anything more than 
> the standard library and win32all.

I understand.  The reason I ask is because it's been A Concern for as
long as I've used Python, but in practice it's never really been an
issue.

-Barry



From skip at pobox.com  Thu Dec 18 08:10:58 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 18 08:11:00 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312180453.hBI4rIg27605@oma.cosc.canterbury.ac.nz>
References: <200312180448.hBI4m8J23710@c-24-5-183-134.client.comcast.net>
	<200312180453.hBI4rIg27605@oma.cosc.canterbury.ac.nz>
Message-ID: <16353.42850.434223.45725@montanaro.dyndns.org>


    Greg> Also I think VMS did something similar with dots in its
    Greg> pathnames. So there are precedents...

Yes, but the dots were kind of hard to miss what with the directory portion
of a path enclosed in [ ... ]

Skip

From mcherm at mcherm.com  Thu Dec 18 08:15:24 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Thu Dec 18 08:15:26 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
Message-ID: <1071753324.3fe1a86c3827e@mcherm.com>

[Michael Chermside]
> ... INSTANCES of 'file' and could access 
> f.__class__. [...] so for restricted code we return something
> like this:
> 
>     class FakeFile:
>         def __eq__(self, other):
>             return other == file
>

[Guido]
> Are you aware of the original issue, which is that as soon as you have
> a file *instance* (which might have been given to you by a very
> restrictive open() variant), you can always get to the file *class*
> using the __class__ attribute?  Access to the __class__ attribute is
> useful for all sorts of reasons.

Yes, I'm aware of that issue... in fact that's what I'm trying to
address here, but I'm obviously not doing a good job of expressing
myself. I was thinking of the restricted code having access to an
instance of 'file' named 'myFileInstance'. This instance has peculiar
behavior when you access 'myFileInstance.__class__'. Instead of
returning the class object 'file', it returns a class object like
'FakeFile' (which I *should* have made a subclass of 'type').

This whole plan is based on two misconceptions. The first is that
access to __class__ is the only way to use introspection to access
things of greater power (like the class) from an instance. It's
NOT the only way, I'm just hoping that the others can be fully
enumerated (somewhat shaky, but perhaps possible) and can be fixed
in similar ways (maybe true). The other misconception (apparently)
is that the only real USE for accessing the __class__ attribute
is for testing class membership. You said above that it's useful
"for all sorts of reasons"... I'm wondering what they are.

And yes... I *also* realize that there's not a whole lot of
difference between an instance which behaves oddly under 
introspection and one which is wrapped in a proxy. But I realize
that providing a secure proxy is somewhat difficult and I'm
wondering if the other approach is more useful.

[Guido]
> Yes, even for 3.0 this is still a dream...

True enough. So feel free to just drop the issue if you've got
better (ie, more immediate) things to spend your time on. I
wouldn't want to see you get a pie by default because there
wasn't time to complete a test suite!

-- Michael Chermside


From mwh at python.net  Thu Dec 18 08:27:58 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 18 08:28:01 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031218031645.GH473@frobozz> (Andrew Bennetts's message of
	"Thu, 18 Dec 2003 14:16:45 +1100")
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz>
Message-ID: <2mu13yfe75.fsf@starship.python.net>

Andrew Bennetts <andrew-pythondev@puzzling.org> writes:

> I still like the idea of having the standard library in its own
> package, but I guess that will have to wait until Python 3.0.

Guido tends to shoot people who suggest this, watch out :-)

Cheers,
mwh

-- 
  Lisp nearing the age of 50 is the most modern language out
  there. GC, dynamic, reflective, the best OO model extant including
  GFs, procedural macros, and the only thing old-fashioned about it 
  is that it is compiled and fast.   -- Kenny Tilton, comp.lang.python

From mwh at python.net  Thu Dec 18 08:42:25 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 18 08:42:28 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Python
	bltinmodule.c, 2.304, 2.305
In-Reply-To: <20031217210929.GX13702@epoch.metaslash.com> (Neal Norwitz's
	message of "Wed, 17 Dec 2003 16:09:29 -0500")
References: <E1AWiW7-0002WZ-00@sc8-pr-cvs1.sourceforge.net>
	<20031217210929.GX13702@epoch.metaslash.com>
Message-ID: <2mptemfdj2.fsf@starship.python.net>

Neal Norwitz <neal@metaslash.com> writes:

> On Wed, Dec 17, 2003 at 12:43:35PM -0800, rhettinger@users.sourceforge.net wrote:
>>
>> Guido grants a Christmas wish:  
>>   sorted() becomes a regular function instead of a classmethod.
>
> Would sort() be a better name?

No (<wink>).

> When I told someone who's done quite a bit of java recently about
> sorted, he asked if it returned a bool whether the list was sorted
> or not. :-)

In Python, that would be issorted (please ignore callable <wink>).
I'd expect sort() to mutate its argument.

Cheers,
mwh

-- 
  "The future" has arrived but they forgot to update the docs.
                                        -- R. David Murray, 9 May 2000

From skip at pobox.com  Thu Dec 18 08:50:12 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 18 08:50:14 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031218051021.GI473@frobozz>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
	<20031218051021.GI473@frobozz>
Message-ID: <16353.45204.728479.296988@montanaro.dyndns.org>


    Andrew> Or it can be fixed by not having the entire standard library at
    Andrew> the toplevel, but instead inside a single "stdlib" package.
    ...
    Andrew> The backwards compatibility problems probably means this isn't
    Andrew> feasible until Python 3.0, though :(

Not necessarily.  It has been proposed (more than once I think) that such a
stdlib package could be created in parallel using symlinks.  At its
simplest, I think all you'd need would be a stdlib directory, a nearly empty
__init__.py file and symlinks to all the current standard modules and
packages.  (The __init__.py file would take care of importing builtin
modules.)

I would choose a very short name for such a package ("std" seems best) even
though it might create clashes because it will be used a lot.

Skip


From mwh at python.net  Thu Dec 18 09:01:52 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 18 09:01:56 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <16353.45204.728479.296988@montanaro.dyndns.org> (Skip
	Montanaro's message of "Thu, 18 Dec 2003 07:50:12 -0600")
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
	<20031218051021.GI473@frobozz>
	<16353.45204.728479.296988@montanaro.dyndns.org>
Message-ID: <2mfzfifcmn.fsf@starship.python.net>

Skip Montanaro <skip@pobox.com> writes:

>     Andrew> Or it can be fixed by not having the entire standard library at
>     Andrew> the toplevel, but instead inside a single "stdlib" package.
>     ...
>     Andrew> The backwards compatibility problems probably means this isn't
>     Andrew> feasible until Python 3.0, though :(
>
> Not necessarily.  It has been proposed (more than once I think) that such a
> stdlib package could be created in parallel using symlinks.  At its
> simplest, I think all you'd need would be a stdlib directory, a nearly empty
> __init__.py file and symlinks to all the current standard modules and
> packages.  (The __init__.py file would take care of importing builtin
> modules.)

Are you sure that you won't get multiple copies of the same module
floating around?  E.g. will you have

from httplib import HTTPConnection as HC1
from std.httplib import HTTPConnection as HC2
HC1 is HC2

I think in the scheme sketched above this will be false, which kills
the idea stone dead.  I might be wrong, though.

Cheers,
mwh


-- 
  Counting lines is probably a good idea if you want to print it out
  and are short on paper, but I fail to see the purpose otherwise.
                                        -- Erik Naggum, comp.lang.lisp

From pf_moore at yahoo.co.uk  Thu Dec 18 09:03:26 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Thu Dec 18 09:03:13 2003
Subject: [Python-Dev] Re: Relative import
References: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz>
	<200312180020.hBI0K7T23311@c-24-5-183-134.client.comcast.net>
Message-ID: <smjiz0i9.fsf@yahoo.co.uk>

Guido van Rossum <guido@python.org> writes:

> There are two proposals that I can live with: my original proposal
> with the leading dots counting the number of levels up, or the
> triple-dot proposal with scan-up semantics.  In both cases, the
> default semantics would switch to absolute for Python 3.0.

If that's what you;d like a vote on, I prefer the triple-dot proposal. 
As far as semantics go, I have no opinion (I don't have any use for
either option) but I *really* dislike the look of the single dot. I'm
not a great fan of the triple dot, but it's the lesser of the two
evils.

I have to admit to some confusion here, though. At the moment, the
following works:

    pkg\__init__.py:
        print "importing pkg"
        import a
    pkg\a.py:
        print "importing a"
        import b
    pkg\b.py:
        print "importing b"

    >>> import pkg
    importing pkg
    importing a
    importing b

Is the proposal that any of this *stop* working? I assume not. 
Otherwise, I'm -1 on the whole thing. I see no reason to break this
usage, and good reason (I can rename pkg - before final release,
obviously - without needing to do a search-and-replace edit on
everything in the package).

I think this whole thing needs a PEP, though. If only to be completely
clear on what is and is not changing, and to record discarded options.

Paul.
-- 
This signature intentionally left blank


From paoloinvernizzi at dmsware.com  Thu Dec 18 09:19:56 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Thu Dec 18 09:20:06 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <16353.45204.728479.296988@montanaro.dyndns.org>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>	<20031216141746.GA3145@burma.localdomain>	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>	<1071714822.27808.88.camel@anthem>	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>	<20031218051021.GI473@frobozz>
	<16353.45204.728479.296988@montanaro.dyndns.org>
Message-ID: <brsd2c$gh1$1@sea.gmane.org>

Skip Montanaro wrote:

> It has been proposed (more than once I think) that such a
> stdlib package could be created in parallel using symlinks.

Symlink is one of the things that I would like to have on my XP box...

:-\

---
Paolo Invernizzi


From pf_moore at yahoo.co.uk  Thu Dec 18 09:40:30 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Thu Dec 18 09:40:48 2003
Subject: [Python-Dev] Re: Relative import
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
Message-ID: <oeu6yysh.fsf@yahoo.co.uk>

Barry Warsaw <barry@python.org> writes:

> On Tue, 2003-12-16 at 13:26, Guido van Rossum wrote:
>> A lot people have presented a good case for relative imports.  Nobody
>> has argued to keep the status quo (where imports are ambiguous about
>> whether they are meant to be absolute or relative).  So I suggest that
>> in 2.4, we introduce the leading dot notation for relative import,
>> while still allowing relative import without a leading dot.  In
>> 2.5 we can start warning about relative import without a leading dot
>> (although that will undoubtedly get complaints from folks who have
>> code that needs to work with 2.3 still).  In 3.0 we can retire
>> ambiguous import.
>
> I'll just note that where the current status quo trips /me/ up most is
> when I accidentally have a local module with the same name as a global
> module, and then I write an import statement expecting to get the
> standard library module, but end up getting the local module.
>
> That's why when I tend to think about this, I start wanting a way to
> spell "definitely give me the global one, no matter what".  IOW, I feel
> like I want a way to bypass relative module lookups.

That issue I can understand. And I agree there should be a way to
state it explicitly.

One of the issues here is that this area is a bit under-documented (I
know, I should read the source, but I don't have the time right now).

At the moment, there are two cases:

1. Import from sys.path. This is what is being called an "absolute"
   import, and is nice and easy to understand. The key issue is that
   there is no way to *force* this interpretation in the face of
   option (2) below.

2. Import from "the package". This is the under-documented bit, but if
   I understand it correctly, it's basically that from within a module
   contained in a package, sys.path is conceptually *extended* to
   include the package's __path__ (which by default contains the
   directory of the package, but which can be user-modified).

Now the big problem here is that behaviour (2) is useful. Simple
"relative" imports of one module within a package from another module
in the same package are common. Guido's (IMHO ugly) "dot" syntax
handles that, by making users explicitly request option (2), and
making the current import syntax *only* mean (1).

But none of the proposed solutions handle the __path__ variable. I
don't have any objection in principle to desupporting __path__ (heck,
it would have made thinking about PEP 302 easier, if nothing else) but
(a) it would need a deprecation warning, and (b) Guido himself offered
a use case in <http://www.python.org/doc/essays/packages.html>.

This definitely needs a PEP. If we're removing support for __path__,
the implications need to be thought through (PEP 302, the pkgutil
module, etc etc). If we're not, none of the proposals so far have
covered how __path__ gets supported in future.

A much simpler proposal, just providing an explicit way of saying
"Import from sys.path *only*" may be OK without a PEP. But even then,
I'd suspect we should have a PEP explaining *why* it has to be this
simple.

Go on, Guido. We'll be gentle if you write a PEP, and we won't set
c.l.p on you :-)

Paul.
-- 
This signature intentionally left blank


From skip at pobox.com  Thu Dec 18 09:49:39 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 18 09:49:46 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <2mfzfifcmn.fsf@starship.python.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
	<20031218051021.GI473@frobozz>
	<16353.45204.728479.296988@montanaro.dyndns.org>
	<2mfzfifcmn.fsf@starship.python.net>
Message-ID: <16353.48771.531590.7792@montanaro.dyndns.org>


    >> It has been proposed (more than once I think) that such a stdlib
    >> package could be created in parallel using symlinks....

    Michael> Are you sure that you won't get multiple copies of the same
    Michael> module floating around?  E.g. will you have

    Michael> from httplib import HTTPConnection as HC1
    Michael> from std.httplib import HTTPConnection as HC2
    Michael> HC1 is HC2

    Michael> I think in the scheme sketched above this will be false, which
    Michael> kills the idea stone dead.

Maybe the symlink idea won't work.  It seems that it would work to import
the standard modules from std/__init__.py though.  I performed a simple
experiment.  I created the std directory, added a symlink in it for the
random module and added an __init__.py file with this content:

    import sys
    import urlparse

Here's the result:

    >>> import random
    >>> from std import random as random2
    >>> random is random2
    False
    >>> import urlparse
    >>> from std import urlparse as urlparse2
    >>> urlparse is urlparse2
    True
    >>> import sys
    >>> from std import sys as sys2
    >>> sys is sys2
    True

If we added a significant number of modules to std/__init__.py, startup
would slow to a crawl.  I imagine some sort of delayed import mechanism
could be crafted to avoid this problem though.

Skip

From lists at hlabs.spb.ru  Thu Dec 18 12:52:31 2003
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Thu Dec 18 09:52:47 2003
Subject: [Python-Dev] Importing packages from command line
Message-ID: <3FE1E95F.7080209@hlabs.spb.ru>

Just an idea...

1. python -p package_name

     Equivalent to:

     import package_name

2. python -p package.zip

     Equivalent to:

     import sys
     sys.insert(0, "package.zip")
     import package

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)


From skip at pobox.com  Thu Dec 18 09:55:12 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 18 09:55:18 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <brsd2c$gh1$1@sea.gmane.org>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
	<20031218051021.GI473@frobozz>
	<16353.45204.728479.296988@montanaro.dyndns.org>
	<brsd2c$gh1$1@sea.gmane.org>
Message-ID: <16353.49104.97549.745804@montanaro.dyndns.org>


    Paolo> Symlink is one of the things that I would like to have on my XP
    Paolo> box...

As Michael Hudson suggests, symlinks probably won't work anyway.  I have
something which looks like a symlink on my Win2k box.  Windows calls it a
Shortcut in listings.  Is that not roughly the same thing as a symlink?

Skip

From fdrake at acm.org  Thu Dec 18 10:16:15 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec 18 10:16:37 2003
Subject: [Python-Dev] Doc/ tree closed on 2.3 branch
Message-ID: <16353.50367.49731.885916@sftp.fdrake.net>


Python's Doc/ tree is closed on the release23-maint branch.  Commits
may still be made on the trunk.

The final 2.3.3 documentation will appear in the development area on
python.org shortly:

    http://www.python.org/dev/doc/

The branch will be open once the 2.3.3 release has been completed.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From paoloinvernizzi at dmsware.com  Thu Dec 18 10:18:22 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Thu Dec 18 10:22:14 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <16353.49104.97549.745804@montanaro.dyndns.org>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem>
	<200312180455.hBI4t3g23741@c-24-5-183-134.client.comcast.net>
	<20031218051021.GI473@frobozz>
	<16353.45204.728479.296988@montanaro.dyndns.org>
	<brsd2c$gh1$1@sea.gmane.org>
	<16353.49104.97549.745804@montanaro.dyndns.org>
Message-ID: <3FE1C53E.1080507@dmsware.com>

Skip Montanaro wrote:

>I have
>something which looks like a symlink on my Win2k box.  Windows calls it a
>Shortcut in listings.  Is that not roughly the same thing as a symlink?
>  
>
No, the NT file system as nothing as flexible as a symlink.

If we are speaking about symlinks to files, windows shortcuts are barely 
usable only for pointing to executable for execution in a console (but 
they have a lot of redirection problems).

If you *open* a shortcut you are actually opening the shortcut itself, 
not the thing it's pointing to ;-(

Shortcuts to a directory is only usable by Explorer-like-programs, I 
don't know any usable operation with them from the console...

The only way to use shortcut in python is to turn python shortcut-aware 
in the import machinery...

---
Paolo Invernizzi

Ps... cygwin is another story....

From guido at python.org  Thu Dec 18 10:38:03 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 18 10:38:09 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
In-Reply-To: Your message of "Thu, 18 Dec 2003 20:58:42 +1000."
	<3FE18862.50208@iinet.net.au> 
References: <200312172250.hBHMoXt25172@oma.cosc.canterbury.ac.nz>
	<200312180012.hBI0C9E23257@c-24-5-183-134.client.comcast.net> 
	<3FE18862.50208@iinet.net.au> 
Message-ID: <200312181538.hBIFc3x25070@c-24-5-183-134.client.comcast.net>

> >>It would be a lot better if we could get away from the idea
> >>of a "restricted mode" in the sense of a flag somewhere that
> >>a bunch of things have to take notice of in order to behave
> >>securely, because that model of security is prone to springing
> >>leaks -- as happened in a big way when new-style classes were
> >>introduced.
> > 
> > Right.  Restricted mode currently uses both paradigms: you only have
> > access to the builtins that are given to you in the __builtins__ dict
> > -- this is pure capability stuff, and IMO it works well -- and some
> > builtin operations behave differently when you're in restricted mode
> > -- this is the ACL stuff, and Samuele revealed serious holes in it.
> 
> What if instead of 'builtin behaves differently in restricted mode' we 
> had 'restricted __builtins__ contains a DIFFERENT builtin, that happens 
> to have the same name'?
> 
> That is, in addition to the ability to simply deny access to a specific 
> builtin function or class, there was the ability to _replace_ one before 
> giving it to the restricted code.

Oh, there is, of course.

Sorry, I wasn't clear.  There are a few operations that are *not*
spelled as built-in function calls that have to be restricted.  For
example, every object supports getting obj.__dict__ -- but in
restricted mode you don't want to allow this.  The Python source
currently checks for "was I called from restricted mode" to deny such
operations.  The holes that Samuele revealed had to do with tricking
unrestricted code into invoking unsafe things with arbitrary arguments
controlled by the restricted code.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Thu Dec 18 10:45:52 2003
From: mwh at python.net (Michael Hudson)
Date: Thu Dec 18 10:47:02 2003
Subject: [Python-Dev] speeding function calls (a little)
In-Reply-To: <1067487850.24165.53.camel@localhost.localdomain>
Message-ID: <425EA205-3171-11D8-8B62-0003931DF95C@python.net>

On Thursday, Oct 30, 2003, at 05:24 Europe/Stockholm, Jeremy Hylton 
wrote:

[function calls]

> There is an optimization that depends on having no default arguments 
> (or
> keyword arguments or free variables).

Why does it depend on not having default arguments?  If you supply the 
right number of arguments (something that's obviously already checked) 
why does the function having defaults make a jot of difference?

>  It copies the arguments directly
> from the caller's frame into the callee's frame without creating an
> argument tuple.
>
> It's interesting to avoid the copy from caller to callee, but I don't
> think it's a big cost relative to everything else we're doing to set up
> a frame for calling.  (I expect the number of arguments is usually
> small.)  You would need some way to encode what variables are loaded
> from the caller stack and what variables are loaded from the current
> frame.  Either a different opcode or some kind of flag in the current
> LOAD/STORE argument.

As I think Phillip managed to convince himself recently, some kind of 
JIT functionality seems to be needed to do function calls really 
efficiently.

I wonder if libffi does enough... it would be nice if the body of 
CALL_FUNCTION could look a bit like this:

  x = POP()
PUSH(((some_cast_or_other)x)(f, stack_pointer, oparg))

Gah, this doesn't really seem to work out, on thinking about it.  Wins 
seem more likely to come from knowing with some certainly at the call 
site that you've not messed the arguments up (and so we're back to 
wanting a JIT, it seems to me).

> One other possibility for optimization is this XXX comment in
> fast_function():
> 		/* XXX Perhaps we should create a specialized
> 		   PyFrame_New() that doesn't take locals, but does
> 		   take builtins without sanity checking them.
> 		*/
> 		f = PyFrame_New(tstate, co, globals, NULL);
>
> PyFrame_New() does a fair amount of work that is unnecessary in the
> common case.

Fair amount?  I have a patch that gets ~1.5% on pystone along these 
lines, but it's a bit scary (makes a "lightweight-frame" subclass that 
assumes more about things on it's freelist, and so on).  I'm not sure 
the modest gains are worth the complexity, but I'll post it to SF...

Cheers,
mwh


From guido at python.org  Thu Dec 18 10:50:44 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 18 10:50:48 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 14:03:26 GMT."
	<smjiz0i9.fsf@yahoo.co.uk> 
References: <200312172324.hBHNOCR25328@oma.cosc.canterbury.ac.nz>
	<200312180020.hBI0K7T23311@c-24-5-183-134.client.comcast.net> 
	<smjiz0i9.fsf@yahoo.co.uk> 
Message-ID: <200312181550.hBIFoiN25162@c-24-5-183-134.client.comcast.net>

> If that's what you;d like a vote on, I prefer the triple-dot proposal. 
> As far as semantics go, I have no opinion (I don't have any use for
> either option) but I *really* dislike the look of the single dot. I'm
> not a great fan of the triple dot, but it's the lesser of the two
> evils.

Your vote doesn't help me; the triple dot proposal is semantically
different from the single dot.

> I have to admit to some confusion here, though. At the moment, the
> following works:
> 
>     pkg\__init__.py:
>         print "importing pkg"
>         import a
>     pkg\a.py:
>         print "importing a"
>         import b
>     pkg\b.py:
>         print "importing b"
> 
>     >>> import pkg
>     importing pkg
>     importing a
>     importing b
> 
> Is the proposal that any of this *stop* working? I assume not. 
> Otherwise, I'm -1 on the whole thing. I see no reason to break this
> usage, and good reason (I can rename pkg - before final release,
> obviously - without needing to do a search-and-replace edit on
> everything in the package).

The idea is indeed to break this in Python 3.0 (but not earlier for
backwards compatibility reasons).  In 3.0, you would have to (and in
2.4 perhaps you could) write this as

  from . import a

or

  from ... import a

depending on which proposal is accepted.

> I think this whole thing needs a PEP, though. If only to be completely
> clear on what is and is not changing, and to record discarded options.

Correct.  I have no time to write it though.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Thu Dec 18 10:59:00 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 18 10:59:06 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 14:40:30 GMT."
	<oeu6yysh.fsf@yahoo.co.uk> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <oeu6yysh.fsf@yahoo.co.uk> 
Message-ID: <200312181559.hBIFx0j25181@c-24-5-183-134.client.comcast.net>

[Barry]
> > That's why when I tend to think about this, I start wanting a way to
> > spell "definitely give me the global one, no matter what".  IOW, I feel
> > like I want a way to bypass relative module lookups.

[Paul Moore]
> That issue I can understand. And I agree there should be a way to
> state it explicitly.
> 
> One of the issues here is that this area is a bit under-documented (I
> know, I should read the source, but I don't have the time right now).
> 
> At the moment, there are two cases:
> 
> 1. Import from sys.path. This is what is being called an "absolute"
>    import, and is nice and easy to understand. The key issue is that
>    there is no way to *force* this interpretation in the face of
>    option (2) below.

Right.

> 2. Import from "the package". This is the under-documented bit, but if
>    I understand it correctly, it's basically that from within a module
>    contained in a package, sys.path is conceptually *extended* to
>    include the package's __path__ (which by default contains the
>    directory of the package, but which can be user-modified).

That's not a very useful way to think about it; when the module is
found on __path__ something very *different* happens than when it is
found on sys.path.  The difference is noticeable if you ask the
imported module for its __name__.  If you the import was satisfied
from __path__, then __name__ will include the package name.  If it was
satisfied from sys.path, it won't.  And __name__ is related to module
identity: all modules are stored in sys.modules using their __name__
as the key, so multiple imports of a module with the same __name__ end
up referring to the same module (and only the first import causes the
module's code to be executed).

> Now the big problem here is that behaviour (2) is useful. Simple
> "relative" imports of one module within a package from another module
> in the same package are common. Guido's (IMHO ugly) "dot" syntax
> handles that, by making users explicitly request option (2), and
> making the current import syntax *only* mean (1).

Actually, what you consider useful is considered harmless by others.
Many large packages, including Zope, have adopted a rule of always
using absolute imports, to clarify what kind of import is being used.

> But none of the proposed solutions handle the __path__ variable. I
> don't have any objection in principle to desupporting __path__ (heck,
> it would have made thinking about PEP 302 easier, if nothing else) but
> (a) it would need a deprecation warning, and (b) Guido himself offered
> a use case in <http://www.python.org/doc/essays/packages.html>.

__path__ has no bearing on the proposals, it is used for relative
imports.  I think your confusing about how the current import works
caused you to think it is relevant.

> This definitely needs a PEP. If we're removing support for __path__,
> the implications need to be thought through (PEP 302, the pkgutil
> module, etc etc). If we're not, none of the proposals so far have
> covered how __path__ gets supported in future.

We're not, but they have (by not mentioning it); but I agree that a
PEP is needed.

> A much simpler proposal, just providing an explicit way of saying
> "Import from sys.path *only*" may be OK without a PEP. But even then,
> I'd suspect we should have a PEP explaining *why* it has to be this
> simple.

This was where this thread started: someone proposed a way of spelling
"import from sys.path only".  My big problem with that is that *most*
imports are intended to be satisfied by sys.path only, so this
spelling should be the default, meaning the current spelling.

> Go on, Guido. We'll be gentle if you write a PEP, and we won't set
> c.l.p on you :-)

Alas, I have no time.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From fdrake at acm.org  Thu Dec 18 11:53:54 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Thu Dec 18 11:54:05 2003
Subject: [Python-Dev] Python 2.3.3 documentation
Message-ID: <16353.56226.174490.137050@sftp.fdrake.net>


Formatted copies of the Python 2.3.3 documentation is available at:

    ftp://ftp.python.org/pub/python/doc/2.3.3/

The website has not yet been updated.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From klm at zope.com  Thu Dec 18 12:20:30 2003
From: klm at zope.com (Ken Manheimer)
Date: Thu Dec 18 12:27:10 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <200312181550.hBIFoiN25162@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312181204540.10960-100000@slab.zope.com>

+1 for the '.' proposal, where (to verify my understanding):

  - '.' indicates "import from the current package", '..' means "import
    from the package of my package", and so forth,

  - imports where the package part lacks a leading '.' are absolute, ie
    from the top level of the path.

-1 for the '...' proposal, where '...' means a search from the current
   package up to the most superior containing package (which is
   actually a virtual package that entails the entire path).

Ie, i am for absolute being the default and explicit-relative being
available, and against ambiguous relative aka scanning the hierarchy.
(I might be convinced otherwise about the scanning deal, but i think
the more deterministic we can make imports, while still keeping them
flexible enough to do what people need, the better.)

I also like your proposal for a "no ambiguous imports" flag

  from __future__ import absolute_import

Ken
klm@zope.com


From guido at python.org  Thu Dec 18 12:50:41 2003
From: guido at python.org (Guido van Rossum)
Date: Thu Dec 18 12:50:48 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Thu, 18 Dec 2003 12:20:30 EST."
	<Pine.LNX.4.44.0312181204540.10960-100000@slab.zope.com> 
References: <Pine.LNX.4.44.0312181204540.10960-100000@slab.zope.com> 
Message-ID: <200312181750.hBIHof425371@c-24-5-183-134.client.comcast.net>

[Ken Manheimer]
> +1 for the '.' proposal, where (to verify my understanding):
> 
>   - '.' indicates "import from the current package", '..' means "import
>     from the package of my package", and so forth,
> 
>   - imports where the package part lacks a leading '.' are absolute, ie
>     from the top level of the path.
> 
> -1 for the '...' proposal, where '...' means a search from the current
>    package up to the most superior containing package (which is
>    actually a virtual package that entails the entire path).
> 
> Ie, i am for absolute being the default and explicit-relative being
> available, and against ambiguous relative aka scanning the hierarchy.
> (I might be convinced otherwise about the scanning deal, but i think
> the more deterministic we can make imports, while still keeping them
> flexible enough to do what people need, the better.)

Thanks!  Your vote is especially important for me because as far as I
can remember you proposed the scan-up rule for the very first version
of "ni" that we developed together back in 1995!

> I also like your proposal for a "no ambiguous imports" flag
> 
>   from __future__ import absolute_import

OK.  Wanna write a PEP?  Just kidding, I know you have less time than
I do for it.  So... *anybody* wanna write a PEP about this?  (The
parentheses proposal should also be folded in.)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From klm at zope.com  Thu Dec 18 13:07:11 2003
From: klm at zope.com (Ken Manheimer)
Date: Thu Dec 18 13:14:09 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <200312181750.hBIHof425371@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312181259200.3222-100000@slab.zope.com>

On Thu, 18 Dec 2003, Guido van Rossum wrote:

> Thanks!  Your vote is especially important for me because as far as I
> can remember you proposed the scan-up rule for the very first version
> of "ni" that we developed together back in 1995!

Older and*/*or wiser?-)

FWIW, i also proposed using a leading '.' (and a leading '_', when '.' 
was refused because the parser couldn't handle it) for what we're now
calling explict relative imports.

Ken
klm@zope.com



From skip at pobox.com  Thu Dec 18 16:20:22 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 18 16:21:00 2003
Subject: [Python-Dev] syncing between projects?
Message-ID: <16354.6678.463640.780682@montanaro.dyndns.org>


The python-mode.el file in the python-mode project is now at 4.41, while the
version in Python's Misc directory is still at 4.38.  Is there a way to sync
them in one cvs operation so they have the same version?

Thx,

Skip

From martin at v.loewis.de  Thu Dec 18 16:22:34 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 18 16:23:44 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <20031217233126.GA4692@unpythonic.net>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
Message-ID: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>

Jeff Epler <jepler@unpythonic.net> writes:

> To follow up on my own message, here is a program that demonstrates the
> problem I ran into with threads and signals.

Can you find out what $$ is, and what the PIDs and thread IDs of all
participating threads are?

Regards,
Martin

From martin at v.loewis.de  Thu Dec 18 16:28:50 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 18 16:29:13 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <87y8tbezcq.fsf@codesourcery.com>
References: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
	<87y8tbezcq.fsf@codesourcery.com>
Message-ID: <m3k74tu86l.fsf@mira.informatik.hu-berlin.de>

"Zack Weinberg" <zack@codesourcery.com> writes:

> > This is what leads to the bletcherous
> >
> >     do {
> >         the real macro guts go here
> >     } while(0)
> >
> > form of macro definition.  Py_DECREF could be rewritten in that form -- but
> > I've puked enough for one day <splat>.
> 
> Should, not just could.  Yeah, it's bletcherous, but it's the only
> way to be sure.  (Short of inline functions.)

Not true. As Tim explains, there is no possible application of the
macro which gets misinterpreted. We don't have a single if-statement,
we have an if-else-statement in the macro. That never leads to the
dangling else problem.

Regards,
Martin


From greg at cosc.canterbury.ac.nz  Thu Dec 18 16:33:22 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 16:33:32 2003
Subject: open == file considered harmful (Re: [Python-Dev]	RE:
	rexec.pyunuseable)
In-Reply-To: <3FE18862.50208@iinet.net.au>
Message-ID: <200312182133.hBILXMV05456@oma.cosc.canterbury.ac.nz>

Nick Coghlan <ncoghlan@iinet.net.au>:

> What if instead of 'builtin behaves differently in restricted mode' we 
> had 'restricted __builtins__ contains a DIFFERENT builtin, that happens 
> to have the same name'?

That wouldn't solve the file problem, because if the restricted
code ever got hold of a file object created by unrestricted code,
it would have access to the "real" file class which can open
any file.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From martin at v.loewis.de  Thu Dec 18 16:27:30 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 18 16:33:51 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <16352.63023.993326.798113@montanaro.dyndns.org>
References: <3FE0DB90.9060604@erols.com>
	<LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
	<16352.63023.993326.798113@montanaro.dyndns.org>
Message-ID: <m3oeu5u88t.fsf@mira.informatik.hu-berlin.de>

Skip Montanaro <skip@pobox.com> writes:

> I meant the Py_*REF macros which expand to "if" statements and you'd bracify
> the clauses, e.g., Py_DECREF would become:

That would still trigger the warning about

  if(1) Py_DECREF(foo);

as this expands to

  if(1) if(cond(foo)) {action1(foo);} else {action2(foo);}

This, in turn, is potentially unintended, as the author of that code
may have meant the else to go with the if(1), whereas, in C, it goes
with the if(cond(foo)). As Tim explains, the intended semantics is the
desired one, too, so there is really no problem in the code. As Tim
also explains, do{}while(1) would silence the warning.

Regards,
Martin

From barry at python.org  Thu Dec 18 16:36:40 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 18 16:36:45 2003
Subject: [Python-Dev] syncing between projects?
In-Reply-To: <16354.6678.463640.780682@montanaro.dyndns.org>
References: <16354.6678.463640.780682@montanaro.dyndns.org>
Message-ID: <1071783400.31232.10.camel@anthem>

On Thu, 2003-12-18 at 16:20, Skip Montanaro wrote:
> The python-mode.el file in the python-mode project is now at 4.41, while the
> version in Python's Misc directory is still at 4.38.  Is there a way to sync
> them in one cvs operation so they have the same version?

cvs com -r4.41 python-mode.el

-Barry



From gmcm at hypernet.com  Thu Dec 18 16:45:54 2003
From: gmcm at hypernet.com (Gordon McMillan)
Date: Thu Dec 18 16:46:51 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <E1AWqLl-0001zs-Uw@mail.python.org>
Message-ID: <3FE1D9C2.32355.4A1EE51A@localhost>

I am +10e6 on disambiguating imports.

I am +0 on "scan".

I am +0.01 on "explicit relative".

I can live with any syntax Guido can, so I'm just voting on 
functionality.

The use cases for "ambiguous" imports amount to:
1) I use this cool package, but I'm too lazy to keep up with
    new releases or document the release required & ensure
    that it's available & further I'm so arrogant that I don't care
    that my users might run into problems having 2 copies of
    the code or that the package developer might be offended
    by my end run.
2) I structured my package wrong, and now I want to fix
    it with the mouse.

I'm not sure I go as far as Tim, but I think perhaps the PSU
should have a regulation about writing import hooks without
a license.

-- Gordon
http://www.mcmillan-inc.com/


From greg at cosc.canterbury.ac.nz  Thu Dec 18 16:50:27 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 16:50:36 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
In-Reply-To: <1071753324.3fe1a86c3827e@mcherm.com>
Message-ID: <200312182150.hBILoRe05474@oma.cosc.canterbury.ac.nz>

Michael Chermside <mcherm@mcherm.com>:

> The other misconception (apparently) is that the only real USE for
> accessing the __class__ attribute is for testing class membership. You
> said above that it's useful "for all sorts of reasons"... I'm
> wondering what they are.

Two that come to mind:

* Looking at the class's docstring

* Finding out what methods an object has

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From martin at v.loewis.de  Thu Dec 18 16:33:52 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Thu Dec 18 16:56:04 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031218092922.GF6324@lkcl.net>
References: <20031215173156.GC25203@lkcl.net>
	<16349.63203.797449.677617@montanaro.dyndns.org>
	<20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
	<20031216212312.GL17021@lkcl.net>
	<m3ptenrv2o.fsf@mira.informatik.hu-berlin.de>
	<20031218092922.GF6324@lkcl.net>
Message-ID: <m3fzfhu7y7.fsf@mira.informatik.hu-berlin.de>

Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:

>  so, with the correct codebase reordering, a simple capabilities
>  based system can be added, the problem goes away.
> 
>  yes?

I still don't see how it could, and I believe I understand pretty well
what kind of feature you propose. Restricted execution is just
something completely different.

Regards,
Martin


From skip at pobox.com  Thu Dec 18 16:56:05 2003
From: skip at pobox.com (Skip Montanaro)
Date: Thu Dec 18 16:56:33 2003
Subject: open == file considered harmful (Re: [Python-Dev] RE:
	rexec.pyunuseable)
In-Reply-To: <200312182150.hBILoRe05474@oma.cosc.canterbury.ac.nz>
References: <1071753324.3fe1a86c3827e@mcherm.com>
	<200312182150.hBILoRe05474@oma.cosc.canterbury.ac.nz>
Message-ID: <16354.8821.435209.523359@montanaro.dyndns.org>


    >> You said above that it's useful "for all sorts of reasons"... I'm
    >> wondering what they are.

    Greg> Two that come to mind:
    ...

And another is breaking out of rexec hell to do devious things on the system
where the code is running.

Skip

From greg at cosc.canterbury.ac.nz  Thu Dec 18 16:57:39 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 16:57:59 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <smjiz0i9.fsf@yahoo.co.uk>
Message-ID: <200312182157.hBILvd005491@oma.cosc.canterbury.ac.nz>

Paul Moore <pf_moore@yahoo.co.uk>:

> If that's what you'd like a vote on, I prefer the triple-dot
> proposal.

How would you spell relative references to parent packages
using the triple-dot version? Remember we've dropped the
idea of search-upwards, so that would have to be made
explicit somehow.

>     pkg\__init__.py:
>         print "importing pkg"
>         import a
>     pkg\a.py:
>         print "importing a"
>         import b
>     pkg\b.py:
>         print "importing b"
> 
>     >>> import pkg
>     importing pkg
>     importing a
>     importing b
> 
> Is the proposal that any of this *stop* working?

I think the proposal is for it to keep working for backwards
compatibility now, but to stop working in Python 3.0. Is
that correct?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Thu Dec 18 16:59:26 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 16:59:49 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <brsd2c$gh1$1@sea.gmane.org>
Message-ID: <200312182159.hBILxQu05504@oma.cosc.canterbury.ac.nz>

Paolo Invernizzi <paoloinvernizzi@dmsware.com>:

> Skip Montanaro wrote:
> 
> > It has been proposed (more than once I think) that such a
> > stdlib package could be created in parallel using symlinks.
> 
> Symlink is one of the things that I would like to have on my XP box...

Also, wouldn't that give two ways of getting each of the
modules, with all the attendand Bad Things?

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Thu Dec 18 17:07:33 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 17:07:41 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <oeu6yysh.fsf@yahoo.co.uk>
Message-ID: <200312182207.hBIM7Xn05611@oma.cosc.canterbury.ac.nz>

Paul Moore <pf_moore@yahoo.co.uk>:

> But none of the proposed solutions handle the __path__ variable. I
> don't have any objection in principle to desupporting __path__

I'll be worried if __path__ is likely to disappear as a
result of all this. I'm using it in what will be the next
version of my Python GUI library, to switch in different
implementations of submodules depending on the platform.

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From greg at cosc.canterbury.ac.nz  Thu Dec 18 17:13:03 2003
From: greg at cosc.canterbury.ac.nz (Greg Ewing)
Date: Thu Dec 18 17:13:13 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <16353.49104.97549.745804@montanaro.dyndns.org>
Message-ID: <200312182213.hBIMD3x05633@oma.cosc.canterbury.ac.nz>

Skip Montanaro <skip@pobox.com>:

> I have something which looks like a symlink on my Win2k box.  Windows
> calls it a Shortcut in listings.  Is that not roughly the same thing
> as a symlink?

Only if it's transparent to the system calls which operate on files
(e.g. open()ing it opens the file that it refers to, etc.)  I don't
believe that's the case with Windows shortcuts.

Macintosh "aliases" have the same problem. (And now with MacOS X, we
have both aliases *and* symlinks in the same system, just to add to
the confusion...)

Greg Ewing, Computer Science Dept, +--------------------------------------+
University of Canterbury,	   | A citizen of NewZealandCorp, a	  |
Christchurch, New Zealand	   | wholly-owned subsidiary of USA Inc.  |
greg@cosc.canterbury.ac.nz	   +--------------------------------------+

From barry at python.org  Thu Dec 18 17:15:05 2003
From: barry at python.org (Barry Warsaw)
Date: Thu Dec 18 17:15:11 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <200312182157.hBILvd005491@oma.cosc.canterbury.ac.nz>
References: <200312182157.hBILvd005491@oma.cosc.canterbury.ac.nz>
Message-ID: <1071785704.31232.22.camel@anthem>

On Fri, 2003-12-19 at 05:57, Greg Ewing wrote:

> How would you spell relative references to parent packages
> using the triple-dot version? 

Does anybody have an example, in released code, of where this kind of
repackaging relative imports is being used?  I'd like to look at some
real-world examples.

Thanks,
-Barry



From lkcl at lkcl.net  Thu Dec 18 17:57:48 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Thu Dec 18 17:57:48 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <m3fzfhu7y7.fsf@mira.informatik.hu-berlin.de>
References: <20031215191023.GB26055@lkcl.net>
	<1071524194.15985.660.camel@localhost.localdomain>
	<20031215233639.GE26055@lkcl.net>
	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>
	<20031216195525.GJ17021@lkcl.net>
	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>
	<20031216212312.GL17021@lkcl.net>
	<m3ptenrv2o.fsf@mira.informatik.hu-berlin.de>
	<20031218092922.GF6324@lkcl.net>
	<m3fzfhu7y7.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031218225748.GE2234@lkcl.net>

On Thu, Dec 18, 2003 at 10:33:52PM +0100, Martin v. L?wis wrote:
> Luke Kenneth Casson Leighton <lkcl@lkcl.net> writes:
> 
> >  so, with the correct codebase reordering, a simple capabilities
> >  based system can be added, the problem goes away.
> > 
> >  yes?
> 
> I still don't see how it could, 

 neither do i: i believe that the best person you should address
 that concern to is to greg ewing, who raised this point:

	 "The spirit behind my suggestion was to start thinking about
	 ways in which functionality could be separated out so that
	 this kind of special-casing for security purposes isn't
	 needed."
	
 in other words, i think he means that by restructuring the
 python codebase [and libraries? and its design?] it _may_ be
 possible to avoid a need for adding ACLs at all.

 and i believe he may be thinking along the lines of being able
 to permanently _remove_ at run-time, under python-programmer-control,
 access to __class__ from objects.

 somehow.


> and I believe I understand pretty well
> what kind of feature you propose. Restricted execution is just
> something completely different.
 
 if you believe that, then i have evidently not explained 
 myself clearly enough.

 i'll give this one more shot.  i'll be honest with you.
 if it's not obvious enough at the end of this message, i have
 to give up and stop, and say i tried but did not succeed in
 explaining this clearly enough to you, which is no failing
 on your part, but on mine.

 
 anyway.


 first i should outline the pseudo-code modifications needed
 to Python/ceval.c which are necessary, then after that i will
 outline what ACLs are needed.
 
 what i _can_ say that i haven't gone into detail on [in this
 message] is _how_ the acls are obtained from the function
 object. e.g. in GetACLforFunction.

 but it is worth mentioning that _how_ the acls are _associated_
 with the function object (or any other object) is an
 implementation-specific issue NOT a design / specification
 issue.


 static PyObject *
 call_function(PyObject ***pp_stack, int oparg)
 {
 	
	 PyObject **pfunc = (*pp_stack) - n - 1;
     PyObject *func = *pfunc;


	/* caller function name on stack below?? probably not,
	   but the principle is at least demonstrable */
	PyObject *calling_func = GetCallerFunction(*pp_stack) - n - 2;
	PyACL *acl_for_func = GetACLforFunction(func);

	...
	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
	                             PERMS_EXECUTE)
	{
		return Exception("access denied");
	}

	...

 }

 
 also, to satisfy the requirements of "capabilities", a specific check
 to make it look like the object doesn't exist is added.

 it's probably possible to have a look-up table on a per-opcode basis
 to look up the permissions to be checked against, which alleviates the
 need to put a call to check_permissions in every darn switch statement.

 note the check for ACL being NULL: if it's NULL, that's assumed
 to mean "anything goes".

 it also means that there's not much performance hit involved unless
 you actually add permissions.



 case STORE_ATTR:
	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
					PERMS_CAN_SEE)
	{
		return Exception("attrib doesn't exist");
	}
	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
					PERMS_WRITE)
	{
		return Exception("access denied");
	}
 case DELETE_ATTR:
	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
					PERMS_CAN_SEE)
	{
		return Exception("attrib doesn't exist");
	}
	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
					PERMS_WRITE+PERMS_DELETE)
	{
		return Exception("access denied");
	}
 case LOAD_ATTR:

	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
					PERMS_CAN_SEE)
	{
		return Exception("attrib doesn't exist");
	}
	if (acl_for_func != NULL && !check_permissions(calling_func, acl_for_func,
					PERMS_READ)
	{
		return Exception("access denied");
	}


 basically, as long as code is only executed through eval_frame(),
 then it's likely to be the only place where access control checks
 are needed to be added.


 
 basically, therefore, once calls to restrict operations are added,
 it's a matter of dropping the ACLs in at the right places.

 _that's_ how you achieve the same job as the rexec.py code.

 you place an acl on __builtins__ to restrict "all functions", DENY,
 EVERYTHING.

 you place an acl on open() to restrict "all functions", DENY,
 FILE_WRITE.

 you place an acl on open.__class__ "all functions", DENY, CAN_SEE+FILE_WRITE

 in fact you might even be able to get away with putting an acl
 on EVERY function right at the top-level.

 but that, as described previously, means that it'd be necessary
 to get the "ACL-inheritance" to work properly, in order to avoid
 having to add a DENY FILE_WRITE acl to every single file operation
 capable of writing, for example.


 so like i said, i believe it to be a relatively simple job to spec
 out how to add the means _by which_ ACLs can be usefully evaluated.

 what i CAN'T tell you is exactly WHERE every single ACL [needed to
 achieve the same results as rexec] have to be added [such that
 they will be usefully evaluated]: i simply don't
 have enough knowledge of the python codebase to do that on my own.

 [unless someone was happy to pay me for long enough to find out,
  of course].


 is this making sense at all?

 in some ways, the longer this is left, the harder it is going to
 be to retrospectively bolt on.

 there's an adage that says security cannot be easily added in, it
 has to be designed in from the start.

 fortunately, i think there are a lot of smart people about :)

 sincerely,

 l.


From janssen at parc.com  Thu Dec 18 18:54:49 2003
From: janssen at parc.com (Bill Janssen)
Date: Thu Dec 18 18:55:12 2003
Subject: [Python-Dev] Re: Relative import 
In-Reply-To: Your message of "Thu, 18 Dec 2003 14:15:05 PST."
	<1071785704.31232.22.camel@anthem> 
Message-ID: <03Dec18.155458pst."58611"@synergy1.parc.xerox.com>

> Does anybody have an example, in released code, of where this kind of
> repackaging relative imports is being used?  I'd like to look at some
> real-world examples.

I'm citing an example from the Jython compile of the Plucker
distiller, for one.  That's available as part of the Plucker package
from www.plkr.org -- look at the Makefile in parser/python.  But I'd
guess most real examples are proprietary code (I can think of a couple
right now).

Bill

From jepler at unpythonic.net  Thu Dec 18 20:23:55 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Thu Dec 18 20:23:44 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031219012347.GB4692@unpythonic.net>

On Thu, Dec 18, 2003 at 10:22:34PM +0100, Martin v. L?wis wrote:
> Can you find out what $$ is, and what the PIDs and thread IDs of all
> participating threads are?

I'm not sure what all information I should try to gather for you.  Let me
know if you think this is enough to file a bug report with...  I changed
the example to make it clearer that it's the subprocess ignoring the
signal that is the problem, not anything in Python that is taking time
to notice the death of a child process.

Typical output (of course, pids and ppids change from run to run.  I
didn't find gettid(), I think thread.get_ident() is not what you were
asking for):
	not threaded
	os.getpid() -> 6444
	os.getppid() -> 6332
	thread.get_ident() -> 1074152064
	shell process 6445
	thread ppid 6444
	Elapsed time 0.00640296936035

	subthread
	os.getpid() -> 6444
	os.getppid() -> 6332
	thread.get_ident() -> 1082547504
	shell process 6447
	thread ppid 6444
	kill failed
	Elapsed time 1.01621508598

	main thread
	os.getpid() -> 6444
	os.getppid() -> 6332
	thread.get_ident() -> 1074152064
	shell process 6449
	thread ppid 6444
	Elapsed time 0.00641894340515


:r threadbug.py
import thread, time, os

def doit():
	print "os.getpid() ->", os.getpid()
	print "os.getppid() ->", os.getppid()
	print "thread.get_ident() ->", thread.get_ident()
        t = time.time()
        os.system("echo shell process $$; echo thread ppid $PPID; kill $$; echo kill failed; sleep 1")
        t1 = time.time()
        print "Elapsed time", t1-t
	print

print
print "not threaded"
doit()
print "subthread"
thread.start_new(doit, ())
time.sleep(2)
print "main thread"
doit()




From mal at egenix.com  Fri Dec 19 03:30:27 2003
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Dec 19 03:30:27 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
 unicodeobject.c, 2.204, 2.205
In-Reply-To: <E1AX9vq-0000UC-00@sc8-pr-cvs1.sourceforge.net>
References: <E1AX9vq-0000UC-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <3FE2B723.9060108@egenix.com>

perky@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory sc8-pr-cvs1:/tmp/cvs-serv1651/Objects
> 
> Modified Files:
> 	unicodeobject.c 
> Log Message:
> SF #859573: Reduce compiler warnings on gcc 3.2 and above.
>
> Index: unicodeobject.c
> ***************
> *** 2204,2208 ****
>   
>       /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
> !     if (size == 1 && *(unsigned char*)s < 256) {
>   	Py_UNICODE r = *(unsigned char*)s;
>   	return PyUnicode_FromUnicode(&r, 1);
> --- 2212,2216 ----
>   
>       /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
> !     if (size == 1) {
>   	Py_UNICODE r = *(unsigned char*)s;
>   	return PyUnicode_FromUnicode(&r, 1);

This "fix" doesn't look right. Please check.

> ***************
> *** 2406,2409 ****
> --- 2414,2421 ----
>   			else if (*p<1000)
>   			    repsize += 2+3+1;
> + #ifndef Py_UNICODE_WIDE
> + 			else
> + 			    repsize += 2+4+1;
> + #else
>   			else if (*p<10000)
>   			    repsize += 2+4+1;
> ***************
> *** 2414,2417 ****
> --- 2426,2430 ----
>   			else
>   			    repsize += 2+7+1;
> + #endif
>   		    }
>   		    requiredsize = respos+repsize+(endp-collend);

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 19 2003)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From perky at i18n.org  Fri Dec 19 04:23:09 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Fri Dec 19 04:23:20 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
	unicodeobject.c, 2.204, 2.205
In-Reply-To: <3FE2B723.9060108@egenix.com>
References: <E1AX9vq-0000UC-00@sc8-pr-cvs1.sourceforge.net>
	<3FE2B723.9060108@egenix.com>
Message-ID: <20031219092309.GA81703@i18n.org>

On Fri, Dec 19, 2003 at 09:30:27AM +0100, M.-A. Lemburg wrote:
> perky@users.sourceforge.net wrote:
> >Update of /cvsroot/python/python/dist/src/Objects
> >In directory sc8-pr-cvs1:/tmp/cvs-serv1651/Objects
> >
> >Modified Files:
> >	unicodeobject.c 
> >Log Message:
> >SF #859573: Reduce compiler warnings on gcc 3.2 and above.
> >
> >Index: unicodeobject.c
> >***************
> >*** 2204,2208 ****
> >  
> >      /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
> >!     if (size == 1 && *(unsigned char*)s < 256) {
> >  	Py_UNICODE r = *(unsigned char*)s;
> >  	return PyUnicode_FromUnicode(&r, 1);
> >--- 2212,2216 ----
> >  
> >      /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
> >!     if (size == 1) {
> >  	Py_UNICODE r = *(unsigned char*)s;
> >  	return PyUnicode_FromUnicode(&r, 1);
> 
> This "fix" doesn't look right. Please check.
> 

gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I./Include  -DPy_BUILD_CORE -o Objects/unicodeobject.o Objects/unicodeobject.c
Objects/unicodeobject.c: In function `PyUnicodeUCS2_DecodeLatin1':
Objects/unicodeobject.c:2214: warning: comparison is always true due to limited range of data type

AFAIK, *(unsigned char*)s is always smaller than 256.
Also decoding latin-1 can be done by just casting it into Py_UNICODE.

I'm sorry but can you explain more?


Hye-Shik

From tismer at tismer.com  Fri Dec 19 04:32:37 2003
From: tismer at tismer.com (Christian Tismer)
Date: Fri Dec 19 04:31:08 2003
Subject: [Python-Dev] Last chance! (was: Does anybody really use
	frame->f_tstate ?)
In-Reply-To: <3FDE6D35.3090100@tismer.com>
References: <3FDE6D35.3090100@tismer.com>
Message-ID: <3FE2C5B5.8080208@tismer.com>

Dear Python community,

since I didn't get *any* reply to this request,
either the request was bad or there is really
nobody using f_tstate in a way that makes it
urgent to keep.
I will wait a few hours and then make the change
to Stackless, and I'd like to propose to do the
same to the Python core.

Christian Tismer wrote:
> Hi colleagues,
> 
> this is my second attempt to get rid of the f_tstate field
> in frames. I need to find every user of this field.
> 
> What am I talking about?
> Well, Python always has a thread state variable which
> is a unique handle to the current thread. This variable
> is accessed in many places, and there exists a fast macro
> to get at it.
> Every executing Python frame also gets a copy on creation.
> In some cases, this frame->f_tstate field is used,
> in other cases the current tstate variable is used.
> 
> If this sounds foreign to you, please stop reading here.
> 
> -------------------------------------------------------------
> 
> I would like to get rid of the frame->f_tstate, and I'm trying
> to find out if there is a need for it. I don't need it,
> for Stackless, it is the opposite, it disturbs.
> 
> There was a small thread about this in June this year, where
> Guido convinced me that it is possible to create a traceback
> on a frame that comes from a different thread.
> 
> http://mail.python.org/pipermail/python-dev/2003-June/036254.html
> 
> Ok, this is in fact possible, although I don't think
> anybody has a need for this.
> 
> My question to all extension writers is this:
> If you use frame->f_tstate at all, do you use it just
> because it is handy, or do you want to use it for
> some other purpose?
> 
> One purpose could be that you really want to create a traceback
> on a different than the current thread. I have never seen this,
> but who knows, so that's why I'm asking the Python world.
> 
> In most cases, a traceback will be created on a frame
> that is currently processd or just has been processed.
> Accessing a frame of a different thread that is being processed
> might make sense for special debugger cases.
> 
> My proposal is
> --------------
> 
> a) change semantics of PytraceBack_Here to use the current tstate.
> 
> b) if such a special purpose exists, create a new function for it.
> 
> c) if urgent, different needs exist to keep f_tstate,
>    then let's forget about this proposal.
> 
> Especially for Stackless, I'd be keen of getting rid of this.
> 
> thanks for input -- chris


-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From fredrik at pythonware.com  Fri Dec 19 05:03:41 2003
From: fredrik at pythonware.com (Fredrik Lundh)
Date: Fri Dec 19 05:03:53 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
References: <E1AX9vq-0000UC-00@sc8-pr-cvs1.sourceforge.net><3FE2B723.9060108@egenix.com>
	<20031219092309.GA81703@i18n.org>
Message-ID: <bruie3$m3b$1@sea.gmane.org>

Hye-Shik Chang wrote:

> AFAIK, *(unsigned char*)s is always smaller than 256.

except when it isn't.  see the ANSI C spec for details.

</F>




From mal at egenix.com  Fri Dec 19 05:18:28 2003
From: mal at egenix.com (M.-A. Lemburg)
Date: Fri Dec 19 05:18:42 2003
Subject: [Python-Dev] Re: [Python-checkins]
	python/dist/src/Objects	unicodeobject.c, 2.204, 2.205
In-Reply-To: <20031219092309.GA81703@i18n.org>
References: <E1AX9vq-0000UC-00@sc8-pr-cvs1.sourceforge.net>	<3FE2B723.9060108@egenix.com>
	<20031219092309.GA81703@i18n.org>
Message-ID: <3FE2D074.90000@egenix.com>

Hye-Shik Chang wrote:
> On Fri, Dec 19, 2003 at 09:30:27AM +0100, M.-A. Lemburg wrote:
> 
>>perky@users.sourceforge.net wrote:
>>
>>>Update of /cvsroot/python/python/dist/src/Objects
>>>In directory sc8-pr-cvs1:/tmp/cvs-serv1651/Objects
>>>
>>>Modified Files:
>>>	unicodeobject.c 
>>>Log Message:
>>>SF #859573: Reduce compiler warnings on gcc 3.2 and above.
>>>
>>>Index: unicodeobject.c
>>>***************
>>>*** 2204,2208 ****
>>> 
>>>     /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
>>>!     if (size == 1 && *(unsigned char*)s < 256) {
>>> 	Py_UNICODE r = *(unsigned char*)s;
>>> 	return PyUnicode_FromUnicode(&r, 1);
>>>--- 2212,2216 ----
>>> 
>>>     /* Latin-1 is equivalent to the first 256 ordinals in Unicode. */
>>>!     if (size == 1) {
>>> 	Py_UNICODE r = *(unsigned char*)s;
>>> 	return PyUnicode_FromUnicode(&r, 1);
>>
>>This "fix" doesn't look right. Please check.
>>
> 
> gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I./Include  -DPy_BUILD_CORE -o Objects/unicodeobject.o Objects/unicodeobject.c
> Objects/unicodeobject.c: In function `PyUnicodeUCS2_DecodeLatin1':
> Objects/unicodeobject.c:2214: warning: comparison is always true due to limited range of data type
> 
> AFAIK, *(unsigned char*)s is always smaller than 256.
> Also decoding latin-1 can be done by just casting it into Py_UNICODE.

You are right. I was thinking that there was some reason we
needed this to get Unicode working on Crays, but looking at
the CVS log, this was probably just the result of adjusting
Martin's single character sharing code to work for Latin-1
rather than just ASCII characters.

-- 
Marc-Andre Lemburg
eGenix.com

Professional Python Services directly from the Source  (#1, Dec 19 2003)
 >>> Python/Zope Consulting and Support ...        http://www.egenix.com/
 >>> mxODBC.Zope.Database.Adapter ...             http://zope.egenix.com/
 >>> mxODBC, mxDateTime, mxTextTools ...        http://python.egenix.com/
________________________________________________________________________

::: Try mxODBC.Zope.DA for Windows,Linux,Solaris,FreeBSD for free ! ::::

From 3gc6p6iqz at aol.com  Fri Dec 19 11:23:03 2003
From: 3gc6p6iqz at aol.com (Nichole Villalobos)
Date: Fri Dec 19 06:22:00 2003
Subject: [Python-Dev] Python-announce-list-request ocb orzmsmz fur
Message-ID: <u84js932ps-3-vy4-c@ta1rtmien>

Hi Python-announce-list-request ,

I have a picture online now. I just want someone to know me before they 
see me. I just feel better that way. (but believe me you wont be 
disappointed) Well i am 22 years old. I have a very out going personality. I 
love to meet new people, i am on the varsity cheerleading squad. I like 
the little romantic stuff and love to be swept off my feet and 
suprised. I am currently looking for a relationship. All around if i were to 
use 3 words to describe me id say Outgoing, sexy, and spontaneous :) And 
if u wanna chat or get to know me. If u really like what u see. youll 
do more than just send me a note.

Talk to you soon I hope... :)

Carolynn
ps. my friend Maria is on with me as well.

http://seeingnoone.com/confirm/?oc=52211091













I do not wish to go on any blind dates at all, now or in the future:
http://seeingnoone.com/remove/?oc=1


m393nnm3m3rem wv
From perky at i18n.org  Fri Dec 19 06:59:52 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Fri Dec 19 06:59:59 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <bruie3$m3b$1@sea.gmane.org>
References: <20031219092309.GA81703@i18n.org> <bruie3$m3b$1@sea.gmane.org>
Message-ID: <20031219115952.GA84061@i18n.org>

On Fri, Dec 19, 2003 at 11:03:41AM +0100, Fredrik Lundh wrote:
> Hye-Shik Chang wrote:
> 
> > AFAIK, *(unsigned char*)s is always smaller than 256.
> 
> except when it isn't.  see the ANSI C spec for details.
> 

Ah. I found. I'm very surprised for that. Thank you! :-)
BTW, do we really support architectures with 9bits-sized char?


Hye-Shik

From mwh at python.net  Fri Dec 19 07:10:11 2003
From: mwh at python.net (Michael Hudson)
Date: Fri Dec 19 07:10:15 2003
Subject: [Python-checkins] Re: [Python-Dev] Re:
	python/dist/src/Objectsunicodeobject.c, 2.204, 2.205
In-Reply-To: <20031219115952.GA84061@i18n.org> (Hye-Shik Chang's message of
	"Fri, 19 Dec 2003 20:59:52 +0900")
References: <20031219092309.GA81703@i18n.org> <bruie3$m3b$1@sea.gmane.org>
	<20031219115952.GA84061@i18n.org>
Message-ID: <2mbrq5f1p8.fsf@starship.python.net>

Hye-Shik Chang <perky@i18n.org> writes:

> On Fri, Dec 19, 2003 at 11:03:41AM +0100, Fredrik Lundh wrote:
>> Hye-Shik Chang wrote:
>> 
>> > AFAIK, *(unsigned char*)s is always smaller than 256.
>> 
>> except when it isn't.  see the ANSI C spec for details.
>> 
>
> Ah. I found. I'm very surprised for that. Thank you! :-)
> BTW, do we really support architectures with 9bits-sized char?

On some kinds of Cray that Python has been built on in the past, I
think the smallest addressable unit of memory is 64 bits.  So, not
quite 96, but getting on that way.  I don't think we want to make the
lives of people porting to such architectures any harder than it
already is...

Cheers,
mwh

-- 
  Need to Know is usually an interesting UK digest of things that
  happened last week or might happen next week. [...] This week,
  nothing happened, and we don't care.
                           -- NTK Now, 2000-12-29, http://www.ntk.net/

From ncoghlan at iinet.net.au  Fri Dec 19 07:54:16 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri Dec 19 07:54:22 2003
Subject: [Python-checkins] Re: [Python-Dev]
	Re:	python/dist/src/Objectsunicodeobject.c, 2.204, 2.205
In-Reply-To: <2mbrq5f1p8.fsf@starship.python.net>
References: <20031219092309.GA81703@i18n.org>
	<bruie3$m3b$1@sea.gmane.org>	<20031219115952.GA84061@i18n.org>
	<2mbrq5f1p8.fsf@starship.python.net>
Message-ID: <3FE2F4F8.80507@iinet.net.au>

Michael Hudson wrote:
> Hye-Shik Chang <perky@i18n.org> writes:
[...]
>>BTW, do we really support architectures with 9bits-sized char?
[...]
> I don't think we want to make the
> lives of people porting to such architectures any harder than it
> already is...

TI make chips where the smallest addressable unit is 16-bits and 
sizeof(char) == sizeof(int) == 16 bits == 1 byte due to the way the C 
standard is written.

I don't think Python is ported to any such chip at present (the one I 
use is a DSP, and I would seriously question the sanity of anyone who 
tried to run Python on one of these critters), but it's a possibility 
that shouldn't be ignored. Porting to such a machine would be rather 
entertaining (sizeof() gets a _lot_ of work in the code for that DSP).

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From skip at pobox.com  Fri Dec 19 08:03:00 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 19 08:03:05 2003
Subject: [Python-checkins] Re: [Python-Dev]
	Re:     python/dist/src/Objectsunicodeobject.c, 2.204, 2.205
In-Reply-To: <3FE2F4F8.80507@iinet.net.au>
References: <20031219092309.GA81703@i18n.org> <bruie3$m3b$1@sea.gmane.org>
	<20031219115952.GA84061@i18n.org>
	<2mbrq5f1p8.fsf@starship.python.net> <3FE2F4F8.80507@iinet.net.au>
Message-ID: <16354.63236.775441.319110@montanaro.dyndns.org>

    Nick> Michael Hudson wrote:
    >> Hye-Shik Chang <perky@i18n.org> writes:
    Nick> [...]
    >>> BTW, do we really support architectures with 9bits-sized char?
    Nick> [...]
    >> I don't think we want to make the lives of people porting to such
    >> architectures any harder than it already is...

    Nick> TI make chips where the smallest addressable unit is 16-bits and
    Nick> sizeof(char) == sizeof(int) == 16 bits == 1 byte due to the way
    Nick> the C standard is written.

It seems to me the right thing to do is to cook up a test in the configure
script which checks the number of bits in an unsigned char and sets a cpp
macro which the code in question then uses to compile the fast case for
8-bit chars and the slow case otherwise.

Skip

From ncoghlan at iinet.net.au  Fri Dec 19 08:07:38 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Fri Dec 19 08:08:50 2003
Subject: [Python-Dev] rexec.py unuseable
In-Reply-To: <20031218225748.GE2234@lkcl.net>
References: <20031215191023.GB26055@lkcl.net>	<1071524194.15985.660.camel@localhost.localdomain>	<20031215233639.GE26055@lkcl.net>	<m3zndsft6u.fsf@mira.informatik.hu-berlin.de>	<20031216195525.GJ17021@lkcl.net>	<m3fzfkfrkf.fsf@mira.informatik.hu-berlin.de>	<20031216212312.GL17021@lkcl.net>	<m3ptenrv2o.fsf@mira.informatik.hu-berlin.de>	<20031218092922.GF6324@lkcl.net>	<m3fzfhu7y7.fsf@mira.informatik.hu-berlin.de>
	<20031218225748.GE2234@lkcl.net>
Message-ID: <3FE2F81A.7090802@iinet.net.au>

Luke Kenneth Casson Leighton wrote:
>  in some ways, the longer this is left, the harder it is going to
>  be to retrospectively bolt on.
> 
>  there's an adage that says security cannot be easily added in, it
>  has to be designed in from the start.

This is very true, but it hurts an ACL-based approach even worse than it 
hurts a capabilities based one.

To get capabilities to work, the question is: how do we construct an 
environment where 'builtins' and all other objects passed to code in 
that environment have been suitably restricted to prevent malicious code 
from causing damage.

The original objects, which are never made available to the untrusted 
code, don't need to care about trust issues - they just keep working as 
they always have.

To get ACL's to work, _everything_ in Python has to care about trust 
issues, as they have to know that they should be checking for the 
existence of an ACL.

I can't even begin to imagine how those ACL's might be managed 
effectively, but I can imagine constructing a special execution 
environment which only allowed 'safe' objects to be passed in.

A 'safe' object would be one of the restricted builtins, or objects able 
to be constructed using only that restricted set of builtins. The major 
issue comes in dealing with Python's introspection capabilities without 
making them completely useless (then again, perhaps 'restricted, with 
almost no introspection' would be an improvement over 'no restricted mode'.

Anyway, despite either approach being Python 3.0 material, the 
capability method at least seems conceptually possible - deleting 
entries out of Python namespace dictionaries is a fairly straightforward 
activity, as is substituting a new implementation for the old 'unsafe' 
implementation when we want to switch to 'restricted' mode. Whereas 
handling ACL's would be a completely new approach that spreads its 
tentacles through much of the CPython source code.

For code, capabilities just make more sense - if they can't use it, 
don't even let them know it's there.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From mcherm at mcherm.com  Fri Dec 19 08:11:00 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Fri Dec 19 08:11:05 2003
Subject: [Python-Dev] Importing packages from command line
Message-ID: <1071839460.3fe2f8e4dbbad@mcherm.com>

Dmitry Vasiliev writes:
> Just an idea...
> 
> 1. python -p package_name
> 
>      Equivalent to:
> 
>      import package_name

(1) Sourceforge is a great place to request feature enhancements. 
    Suggestions made on this mailing list is likely to be forgotten
    sooner or later.

(2) Can you explain WHY you would want this feature? Is there some
    use case? I would prefer NOT to have this, because right now
    if I'm reading code and it uses "package_name.someFunction()"
    I can scan upward for "package_name" to find out what it's
    using. With command line imports, I couldn't do that. So unless
    you've got a good reason for it, I personally wouldn't want
    this feature.


-- Michael Chermside

From skip at pobox.com  Fri Dec 19 08:20:52 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 19 08:20:49 2003
Subject: [Python-checkins] Re: [Python-Dev]
	Re:     python/dist/src/Objectsunicodeobject.c, 2.204, 2.205
In-Reply-To: <16354.63236.775441.319110@montanaro.dyndns.org>
References: <20031219092309.GA81703@i18n.org> <bruie3$m3b$1@sea.gmane.org>
	<20031219115952.GA84061@i18n.org>
	<2mbrq5f1p8.fsf@starship.python.net> <3FE2F4F8.80507@iinet.net.au>
	<16354.63236.775441.319110@montanaro.dyndns.org>
Message-ID: <16354.64308.552430.331674@montanaro.dyndns.org>


    Skip> It seems to me the right thing to do is to cook up a test in the
    Skip> configure script which checks the number of bits in an unsigned
    Skip> char ...

Better yet, let's use CHAR_BIT:

#if defined(CHAR_BIT) && CHAR_BIT == 8
    ... fast case ...
#else
    ... slow case ...
#endif

From ark at acm.org  Fri Dec 19 09:00:07 2003
From: ark at acm.org (Andrew Koenig)
Date: Fri Dec 19 09:00:02 2003
Subject: [Python-Dev] 2.3.3 released?
Message-ID: <007d01c3c638$69074c20$6402a8c0@arkdesktop>

It appears that the main python.org page is announcing the release of 2.3.3
final, but the release page http://www.python.org/2.3.3/ does not show a
place to download the Windows executable.

Have I caught it in mid-transition?


From fdrake at acm.org  Fri Dec 19 09:09:57 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec 19 09:10:04 2003
Subject: [Python-Dev] 2.3.3 released?
In-Reply-To: <007d01c3c638$69074c20$6402a8c0@arkdesktop>
References: <007d01c3c638$69074c20$6402a8c0@arkdesktop>
Message-ID: <16355.1717.391064.71007@sftp.fdrake.net>


Andrew Koenig writes:
 > It appears that the main python.org page is announcing the release of 2.3.3
 > final, but the release page http://www.python.org/2.3.3/ does not show a
 > place to download the Windows executable.

The Windows installer will be added when it's ready, then the general
announcement will be sent.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From skip at pobox.com  Fri Dec 19 09:14:59 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 19 09:15:00 2003
Subject: [Python-Dev] 2.3.3 released?
In-Reply-To: <007d01c3c638$69074c20$6402a8c0@arkdesktop>
References: <007d01c3c638$69074c20$6402a8c0@arkdesktop>
Message-ID: <16355.2019.340005.818253@montanaro.dyndns.org>


    Andrew> [2.3.3 release teasers]

    Andrew> Have I caught it in mid-transition?

Yes, I believe so.  I saw a bunch of checkins related to that, but it
appears all the bits have not yet been pushed into place.

Skip

From fdrake at acm.org  Fri Dec 19 09:21:46 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Fri Dec 19 09:21:53 2003
Subject: [Python-Dev] 2.3.3 released?
In-Reply-To: <16355.2019.340005.818253@montanaro.dyndns.org>
References: <007d01c3c638$69074c20$6402a8c0@arkdesktop>
	<16355.2019.340005.818253@montanaro.dyndns.org>
Message-ID: <16355.2426.281597.99584@sftp.fdrake.net>


Skip Montanaro writes:
 > Yes, I believe so.  I saw a bunch of checkins related to that, but it
 > appears all the bits have not yet been pushed into place.

I made the changes to the doc/ area on python.org last night, and
everything is up on the site.  I think the 2.3.3/ area is updated as
well, the Windows installer just isn't ready yet.  It'll be there when
it is.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From theller at python.net  Fri Dec 19 09:40:30 2003
From: theller at python.net (Thomas Heller)
Date: Fri Dec 19 09:40:36 2003
Subject: [Python-Dev] 2.3.3 released?
In-Reply-To: <16355.2426.281597.99584@sftp.fdrake.net> (Fred L. Drake, Jr.'s
	message of "Fri, 19 Dec 2003 09:21:46 -0500")
References: <007d01c3c638$69074c20$6402a8c0@arkdesktop>
	<16355.2019.340005.818253@montanaro.dyndns.org>
	<16355.2426.281597.99584@sftp.fdrake.net>
Message-ID: <oeu498gx.fsf@python.net>

"Fred L. Drake, Jr." <fdrake@acm.org> writes:

> Skip Montanaro writes:
>  > Yes, I believe so.  I saw a bunch of checkins related to that, but it
>  > appears all the bits have not yet been pushed into place.
>
> I made the changes to the doc/ area on python.org last night, and
> everything is up on the site.  I think the 2.3.3/ area is updated as
> well, the Windows installer just isn't ready yet.  It'll be there when
> it is.

Hm, I was waiting for Anthony in IRC #python-dev to upload the windows
installer.  Sorry if I misunderstood something...

Thomas


From anthony at interlink.com.au  Fri Dec 19 10:29:23 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Fri Dec 19 10:29:51 2003
Subject: [Python-Dev] 2.3.3 released? 
In-Reply-To: <007d01c3c638$69074c20$6402a8c0@arkdesktop> 
Message-ID: <200312191529.hBJFTN8Y009795@localhost.localdomain>


>>> "Andrew Koenig" wrote
> It appears that the main python.org page is announcing the release of 2.3.3
> final, but the release page http://www.python.org/2.3.3/ does not show a
> place to download the Windows executable.
> 
> Have I caught it in mid-transition?

Yep, timezones make it tricky to manage this stuff sometimes. I'll send the
release announcement in the morning when I've checked it's all good.

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From guido at python.org  Fri Dec 19 10:31:42 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec 19 10:31:51 2003
Subject: [Python-Dev] Last chance! (was: Does anybody really use
	frame->f_tstate ?)
In-Reply-To: Your message of "Fri, 19 Dec 2003 10:32:37 +0100."
	<3FE2C5B5.8080208@tismer.com> 
References: <3FDE6D35.3090100@tismer.com>  <3FE2C5B5.8080208@tismer.com> 
Message-ID: <200312191531.hBJFVgw27430@c-24-5-183-134.client.comcast.net>

> since I didn't get *any* reply to this request,
> either the request was bad or there is really
> nobody using f_tstate in a way that makes it
> urgent to keep.
> I will wait a few hours and then make the change
> to Stackless, and I'd like to propose to do the
> same to the Python core.

I saved the message, but haven't had the time yet to think things
through.

I *did* notice at least one case where using f_tstate might actually
be a mistake: theoretically it's possible that two or more threads
alternate calling next() on a generator (if they wrap it in a critical
section); AFAICT the f_tstate is never updated.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Fri Dec 19 10:39:45 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec 19 10:39:49 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: Your message of "Fri, 19 Dec 2003 20:59:52 +0900."
	<20031219115952.GA84061@i18n.org> 
References: <20031219092309.GA81703@i18n.org> <bruie3$m3b$1@sea.gmane.org>  
	<20031219115952.GA84061@i18n.org> 
Message-ID: <200312191539.hBJFdjC27495@c-24-5-183-134.client.comcast.net>

> > > AFAIK, *(unsigned char*)s is always smaller than 256.
> > 
> > except when it isn't.  see the ANSI C spec for details.
> 
> Ah. I found. I'm very surprised for that. Thank you! :-)
> BTW, do we really support architectures with 9bits-sized char?

I would expect that a lot of our code assumes 8-bit characters, and I
personally wouldn't mind if Python was limited to such platforms.
They aren't very important for attracting new users, and certainly
they don't seem to be a growing kind of platform...  (Probably because
so much other software makes the same assumption. :-)

So IMO your fix is fine.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Fri Dec 19 10:40:13 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec 19 10:40:19 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <20031219115952.GA84061@i18n.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCCEFEHOAB.tim.one@comcast.net>

[Hye-Shik Chang]
> BTW, do we really support architectures with 9bits-sized char?

I don't think so.  There are assumptions that a char is 8 bits scattered
throughout Python's code, not so much in the context of using characters
*as* characters, but more indirectly by assuming that the number of *bits*
in an object of a non-char type T can be computed as sizeof(T)*8.

Skip's idea of making config smarter about this is a good one, but instead
of trying to "fix stuff" for a case that's probably never going to arise,
and that can't really be tested anyway until it does, I'd add a block like
this everywhere we know we're relying on 8-bit char:

#ifdef HAS_FUNNY_SIZE_CHAR
#error "The following code needs rework when a char isn't 8 bits"
#endif
/* A comment explaining why the following code needs rework
 * when a char isn't 8 bits.
 */

Crays are a red herring here.  It's true that some Cray *hardware* can't
address anything smaller than 64 bits, and that's also true of some other
architectures.  char is nevertheless 8 bits on all such 64-bit boxes I know
of (and since I worked in a 64-bit world for 15 years, I know about most of
them <wink>).  On Crays, this is achieved (albeit at major expense) in
software:  by *software* convention, a pointer-to-char stores the byte
offset in the *most*-significant 3 bits of a pointer, and long-winded
generated coded picks that part at runtime, loading or storing 8 bytes at a
time (the HW can't do less than that), shifting and masking and or'ing to
give the illusion of byte addressing for char.  Some Alphas do something
similar, but that HW's loads and stores simply ignore the last 3 bits of a
memory address, and the CPU has special-purpose instructions to help
generated code do the subsequent extraction and insertion of 8-bit chunks
efficiently and succinctly.


From tismer at tismer.com  Fri Dec 19 12:49:15 2003
From: tismer at tismer.com (Christian Tismer)
Date: Fri Dec 19 12:47:42 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <200312191531.hBJFVgw27430@c-24-5-183-134.client.comcast.net>
References: <3FDE6D35.3090100@tismer.com> <3FE2C5B5.8080208@tismer.com>
	<200312191531.hBJFVgw27430@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE33A1B.5030903@tismer.com>

Guido van Rossum wrote:

...

> I saved the message, but haven't had the time yet to think things
> through.
> 
> I *did* notice at least one case where using f_tstate might actually
> be a mistake: theoretically it's possible that two or more threads
> alternate calling next() on a generator (if they wrap it in a critical
> section); AFAICT the f_tstate is never updated.

Right, f_tstate is never updated.
I think there is another inconsistent
situation, which can be created easily.
If a generator is run by a different thread
than it's creator, then the frame is run in that
other thread. eval_fame correctly uses tstate,
but if tracing is activated, call_trace uses
frame->f_tstate for no obvious reason, which
will probably mess up the tracing flags of the wrong
thread.

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From blink at msn.com  Sat Dec 20 05:49:02 2003
From: blink at msn.com (Reuben Crenshaw)
Date: Fri Dec 19 15:55:28 2003
Subject: [Python-Dev] You tried diets and workouts,
	add this and be amazed!! bpvwvczpyztptp
Message-ID: <i-v0p-$x$wj84-arg$9@qun.c.3n5w>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031220/9fee7bad/attachment.html
From Jack.Jansen at cwi.nl  Fri Dec 19 17:57:02 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Dec 19 17:57:07 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <200312191539.hBJFdjC27495@c-24-5-183-134.client.comcast.net>
References: <20031219092309.GA81703@i18n.org> <bruie3$m3b$1@sea.gmane.org>
	<20031219115952.GA84061@i18n.org>
	<200312191539.hBJFdjC27495@c-24-5-183-134.client.comcast.net>
Message-ID: <A8A0978E-3276-11D8-BE55-000A27B19B96@cwi.nl>


On 19-dec-03, at 16:39, Guido van Rossum wrote:

>>>> AFAIK, *(unsigned char*)s is always smaller than 256.
>>>
>>> except when it isn't.  see the ANSI C spec for details.
>>
>> Ah. I found. I'm very surprised for that. Thank you! :-)
>> BTW, do we really support architectures with 9bits-sized char?
>
> I would expect that a lot of our code assumes 8-bit characters, and I
> personally wouldn't mind if Python was limited to such platforms.

Then there is a lot of code that could be tuned for this. Since I'm
using gcc 3.3 (which came with OSX 10.3) I get lots of warnings about
comparisons that are always true due to the size of the operands.
I looked at a couple of these and I think they were all char-related.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From jack at performancedrivers.com  Fri Dec 19 16:30:31 2003
From: jack at performancedrivers.com (Jack Diederich)
Date: Fri Dec 19 22:03:52 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031218031645.GH473@frobozz>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz>
Message-ID: <20031219213031.GD1246@wopr>

On Thu, Dec 18, 2003 at 02:16:45PM +1100, Andrew Bennetts wrote:
> On Wed, Dec 17, 2003 at 10:08:21PM -0500, Barry Warsaw wrote:
> > On Wed, 2003-12-17 at 22:04, Andrew Bennetts wrote:
> > > On Wed, Dec 17, 2003 at 09:33:43PM -0500, Barry Warsaw wrote:
> > > > 
> > > > That's why when I tend to think about this, I start wanting a way to
> > > > spell "definitely give me the global one, no matter what".  IOW, I feel
> > > > like I want a way to bypass relative module lookups.
> > > 
> > > Alternatively, maybe what you want is a way to say "definitely give me the
> > > standard library one, no matter what", e.g.
> > > 
> > >     from stdlib import codecs
> > 
> > Interesting.  I see the subtle distinction.  One searches only the
> > standard library, the other searches every directory on sys.path.  I'm
> > not sure I'd have much need to restrict the search to just the standard
> > library.  Plus, would that include site-packages?
> 
> I should have been clearer.  What I'm basically proposing is to make
> site-package called "stdlib", leaving a default python install with a very
> uncluttered top-level namespace, thus reducing unexpected name conflicts.

I'm at least a little confused with what people want, as far as I can tell
import semantics have three things
 * what module to import
 * what name to give that import
 * where to look for it

The 'from' keyword seems to be getting overloaded to mean both what module and
where to find it, which the 'stdlib' tries to resolve.  Most of the
discussion has been about how to improve 'where to look'

I like syntax that reads most important left-to-right, so what about
from MODULE import NAMES as RENAME searching HOW
or
import NAMES as RENAME from MODULE searching HOW

searching could be 'asbolute' 'relative' or any of the other suggested
words.  If you wanted to get fancy it could be a list, if it isn't a list
people who truly care could cascade try/except on ImportError.

-jackdied

ps, I like the second version better but it is less like the current version,
which is a drawback.

From barry at python.org  Fri Dec 19 23:13:15 2003
From: barry at python.org (Barry Warsaw)
Date: Fri Dec 19 23:13:26 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031219213031.GD1246@wopr>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz>
	<20031219213031.GD1246@wopr>
Message-ID: <1071893594.1718.166.camel@anthem>

On Fri, 2003-12-19 at 16:30, Jack Diederich wrote:

> I'm at least a little confused with what people want, as far as I can tell
> import semantics have three things
>  * what module to import
>  * what name to give that import
>  * where to look for it
> 
> The 'from' keyword seems to be getting overloaded to mean both what module and
> where to find it, which the 'stdlib' tries to resolve.  Most of the
> discussion has been about how to improve 'where to look'

Very interesting observation.

> I like syntax that reads most important left-to-right, so what about
> from MODULE import NAMES as RENAME searching HOW
> or
> import NAMES as RENAME from MODULE searching HOW
> 
> searching could be 'asbolute' 'relative' or any of the other suggested
> words.  If you wanted to get fancy it could be a list, if it isn't a list
> people who truly care could cascade try/except on ImportError.

I want to think about that more, but at first glance, it has some
appeal.  Neat!

-Barry



From tim.one at comcast.net  Fri Dec 19 23:32:45 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec 19 23:32:51 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <1071893594.1718.166.camel@anthem>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEJAHOAB.tim.one@comcast.net>

[Jack Diederich]
>> ...
>> I like syntax that reads most important left-to-right, so what about
>> from MODULE import NAMES as RENAME searching HOW
>> or
>> import NAMES as RENAME from MODULE searching HOW
>>
>> searching could be 'asbolute' 'relative' or any of the other
>> suggested words.  If you wanted to get fancy it could be a list, if
>> it isn't a list people who truly care could cascade try/except on
>> ImportError.

[Barry Warsaw]
> I want to think about that more, but at first glance, it has some
> appeal.  Neat!

Ditto.  I think Jack's on to something there.  I particularly like the
second form, as it's an ongoing irritation to me that some imports begin
with "from" and others with "import".  In the current "from" form, the most
important thing to me is almost never the module I'm importing the thing
from, but (of course) the thing I'm importing.

Going from

    import math

to

    import sin from math

(or vice versa) is also an easier edit than from

    import math

to

    from math import sin

(because the former doesn't require transposition).

Adding a distinct clause to influence search details seems downright
Pythonic.  Maybe *too* Pythonic <wink>.


From tim.one at comcast.net  Sat Dec 20 00:22:22 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 00:22:27 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <200312191539.hBJFdjC27495@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEJDHOAB.tim.one@comcast.net>

[Guido]
> I would expect that a lot of our code assumes 8-bit characters, and I
> personally wouldn't mind if Python was limited to such platforms.
> They aren't very important for attracting new users, and certainly
> they don't seem to be a growing kind of platform...  (Probably because
> so much other software makes the same assumption. :-)

Fine by me too.

The first mainframe I used was a Univac 1108.  There were a *lot* of
competing HW architectures at that time, and manufacturers didn't agree
about character size any more than they agreed about floating-point format
or semantics, or the natural size of "a word".  Univac was forward-looking,
though:  they didn't want their hardware to become obsolete if a different
character size than the one they preferred clicked, so a control bit in the
CPU could be set to treat their 36-bit words as either 6 6-bit characters,
or as 4 9-bit characters.  It worked!  We're *still* equally comfortable
with 6-bit bytes as with 9-bit bytes <wink>.

I was betting on 6-bit bytes at the time, because that also worked well with
CDC's 60-bit words.  FORTRAN didn't even admit to the existence of lower
case at the time, so 64 characters was way more than enough for anything
anyone really needed to say to a computer.

half-the-bits-in-these-new-fangled-bytes-are-just-wasted-ly y'rs  - tim


From whisper at oz.net  Sat Dec 20 00:32:26 2003
From: whisper at oz.net (David LeBlanc)
Date: Sat Dec 20 00:51:36 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEJDHOAB.tim.one@comcast.net>
Message-ID: <LAEHLEAOBBLKAALGAPAFAEGECLAA.whisper@oz.net>

> [Guido]
> > I would expect that a lot of our code assumes 8-bit characters, and I
> > personally wouldn't mind if Python was limited to such platforms.
> > They aren't very important for attracting new users, and certainly
> > they don't seem to be a growing kind of platform...  (Probably because
> > so much other software makes the same assumption. :-)
>
> Fine by me too.
>
> The first mainframe I used was a Univac 1108.  There were a *lot* of
> competing HW architectures at that time, and manufacturers didn't agree
> about character size any more than they agreed about floating-point format
> or semantics, or the natural size of "a word".  Univac was
> forward-looking,
> though:  they didn't want their hardware to become obsolete if a different
> character size than the one they preferred clicked, so a control
> bit in the
> CPU could be set to treat their 36-bit words as either 6 6-bit characters,
> or as 4 9-bit characters.  It worked!  We're *still* equally comfortable
> with 6-bit bytes as with 9-bit bytes <wink>.
>
> I was betting on 6-bit bytes at the time, because that also
> worked well with
> CDC's 60-bit words.  FORTRAN didn't even admit to the existence of lower
> case at the time, so 64 characters was way more than enough for anything
> anyone really needed to say to a computer.
>
> half-the-bits-in-these-new-fangled-bytes-are-just-wasted-ly y'rs  - tim
>

I would think the lesson to be learned from this is that one should not lock
the software into any particular number of bits per character. The coming
flood of 64 bit machines could make 16 bit unicode attractive. It's an ever
more global world and "we" should keep in mind that in the next decade most
of the world's programming is going to be done in India and China if
American corporations have their way.

soon-to-be-looking-for-the-carton-the-mainframe-came-in-to-live-ingly-y'rs

Dave LeBlanc
Seattle, WA USA


From zack at codesourcery.com  Fri Dec 19 13:08:52 2003
From: zack at codesourcery.com (Zack Weinberg)
Date: Sat Dec 20 02:35:53 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <m3k74tu86l.fsf@mira.informatik.hu-berlin.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "18 Dec 2003 22:28:50
	+0100")
References: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
	<87y8tbezcq.fsf@codesourcery.com>
	<m3k74tu86l.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87wu8sy91n.fsf@egil.codesourcery.com>

martin@v.loewis.de (Martin v. L?wis) writes:

> "Zack Weinberg" <zack@codesourcery.com> writes:
>
>> > This is what leads to the bletcherous
>> >
>> >     do {
>> >         the real macro guts go here
>> >     } while(0)
>> >
>> > form of macro definition.  Py_DECREF could be rewritten in that form -- but
>> > I've puked enough for one day <splat>.
>> 
>> Should, not just could.  Yeah, it's bletcherous, but it's the only
>> way to be sure.  (Short of inline functions.)
>
> Not true. As Tim explains, there is no possible application of the
> macro which gets misinterpreted. We don't have a single if-statement,
> we have an if-else-statement in the macro. That never leads to the
> dangling else problem.

It does, however, lead to a different problem:

  if (condition)
    Py_DECREF (foo);
  else               // oops, syntax error here
    ...

In general I consider it appropriate to wrap _all_ macros that don't
return a value in do { ... } while (0), because that way you don't
have to worry about which of them truly need it.

zw

From arigo at tunes.org  Sat Dec 20 03:43:57 2003
From: arigo at tunes.org (Armin Rigo)
Date: Sat Dec 20 03:48:49 2003
Subject: [Python-Dev] Re: [pypy-dev] Last chance! (was: Does anybody really
	use frame->f_tstate ?)
In-Reply-To: <3FE2C5B5.8080208@tismer.com>
References: <3FDE6D35.3090100@tismer.com> <3FE2C5B5.8080208@tismer.com>
Message-ID: <20031220084357.GA25998@vicky.ecs.soton.ac.uk>

Hello Christian,

On Fri, Dec 19, 2003 at 10:32:37AM +0100, Christian Tismer wrote:
> since I didn't get *any* reply to this request,
> either the request was bad or there is really
> nobody using f_tstate in a way that makes it
> urgent to keep.

Sorry for the delay, we were kind of busy this week here...

There are references to f_tstate in Psyco but nothing critical.  Using
PyThreadState_GET() instead should be fine.


A bientot,

Armin.


From ncoghlan at iinet.net.au  Sat Dec 20 04:09:48 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sat Dec 20 04:11:46 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCEFEHOAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCCEFEHOAB.tim.one@comcast.net>
Message-ID: <3FE411DC.3010401@iinet.net.au>

Tim Peters wrote:
> [Hye-Shik Chang]
> 
>>BTW, do we really support architectures with 9bits-sized char?
[...]
> Skip's idea of making config smarter about this is a good one, but instead
> of trying to "fix stuff" for a case that's probably never going to arise,
> and that can't really be tested anyway until it does, I'd add a block like
> this everywhere we know we're relying on 8-bit char:
> 
> #ifdef HAS_FUNNY_SIZE_CHAR
> #error "The following code needs rework when a char isn't 8 bits"
> #endif
> /* A comment explaining why the following code needs rework
>  * when a char isn't 8 bits.
>  */

This would probably be appropriate for those TI DSP's I mentioned. While 
they genuinely have a 16-bit char type, they're also intended for use as 
co-processors, rather than as the main controller for an application. 
That is, a more standard CPU should be used to handle the general 
application programming, while the DSP is used for the serious number 
crunching (that's what it is made for, after all).

Anyone who _thinks_ they want to run Python on the DSP core almost 
certainly needs to have a long hard think about their system design.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From martin at v.loewis.de  Sat Dec 20 05:15:21 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Dec 20 05:17:51 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <20031219012347.GB4692@unpythonic.net>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
Message-ID: <m3smjfztfq.fsf@mira.informatik.hu-berlin.de>

Jeff Epler <jepler@unpythonic.net> writes:

> > Can you find out what $$ is, and what the PIDs and thread IDs of all
> > participating threads are?
> 
> I'm not sure what all information I should try to gather for you.  Let me
> know if you think this is enough to file a bug report with...  I changed
> the example to make it clearer that it's the subprocess ignoring the
> signal that is the problem, not anything in Python that is taking time
> to notice the death of a child process.

That is an important observation; signals that are blocked in the
parent process will be blocked in the child process as well.

I'm not sure what to do about this: We apparently *want* the signals
blocked in the thread, but we don't want them to be blocked in the
process invoked through system(). Proposals are welcome.

Regards,
Martin

From martin at v.loewis.de  Sat Dec 20 05:23:43 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Dec 20 05:24:56 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LAEHLEAOBBLKAALGAPAFAEGECLAA.whisper@oz.net>
References: <LAEHLEAOBBLKAALGAPAFAEGECLAA.whisper@oz.net>
Message-ID: <m3oeu3zt1s.fsf@mira.informatik.hu-berlin.de>

"David LeBlanc" <whisper@oz.net> writes:

> I would think the lesson to be learned from this is that one should
> not lock the software into any particular number of bits per
> character. The coming flood of 64 bit machines could make 16 bit
> unicode attractive.

You are talking about an entirely different issue here. This thread is
about the number of bits in a "char", which, in C, is the same thing
as a "byte". The number of bits for a "character" is independent.

16-bit Unicode is attractive already, although it is dying, to make
place for 32-bit Unicode. However, new 64-bit architectures will make
sure they support an 8-bit data type, and compiler vendors will make
sure that "char" maps to that 8-bit data type (most likely, they also
will make char signed by default). There is just too much software
that breaks if you could not address bytes anymore. Primarily, the
entire networking interfaces would break down, which is a risk that
new architectures are unlikely to take.

> It's an ever more global world and "we" should keep in mind that in
> the next decade most of the world's programming is going to be done
> in India and China if American corporations have their way.

Certainly, but unrelated to the issue at hand.

Regards,
Martin

From martin at v.loewis.de  Sat Dec 20 05:31:18 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sat Dec 20 05:40:20 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <87wu8sy91n.fsf@egil.codesourcery.com>
References: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
	<87y8tbezcq.fsf@codesourcery.com>
	<m3k74tu86l.fsf@mira.informatik.hu-berlin.de>
	<87wu8sy91n.fsf@egil.codesourcery.com>
Message-ID: <m3k74rzsp5.fsf@mira.informatik.hu-berlin.de>

"Zack Weinberg" <zack@codesourcery.com> writes:

> It does, however, lead to a different problem:
> 
>   if (condition)
>     Py_DECREF (foo);
>   else               // oops, syntax error here

How so? This expands to

if(condition)
  if(cond2)action1;
  else action2;
else
  ...

This is perfectly well-formed C, and it groups as my
indentation suggests.

There is no real problem with this macro. It is just gcc complaining
falsely.

> In general I consider it appropriate to wrap _all_ macros that don't
> return a value in do { ... } while (0), because that way you don't
> have to worry about which of them truly need it.

Hmm. Yes, this could be done - but it is unfortunate that this needs
to be done only to silence gcc.

Regards,
Martin


From lkcl at lkcl.net  Sat Dec 20 06:21:20 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Sat Dec 20 06:21:00 2003
Subject: [Python-Dev] Capabilities - published interfaces
Message-ID: <20031220112120.GE1933@lkcl.net>

following on, with a change of subject, from the rexec / acls
discussions, i believe that i have finally worked out why acls
are not the preferred way forward, leaving speed disadvantages
aside.

ACLs need to be applied subtly and explicitly on every important
type of object.  if they're not designed in at the time they are a
PAIN to retro-fit.

ACLs need to be applied subtly because if they are applied unsubtly
you end up with more gates than you do green pastures - more
roundabouts than roads (viz, milton keynes).

ACLs need to be applied explicitly because of their nature: you
have something (it could be the right to "see" something) and you
try do _do_ something and are explicitly told whether it worked or
not (or are explicitly fooled into believing that something cannot
be "seen).

from a code security review perspective, it's actually easier to
review ACL-restricted code than it is capabilities-based code, but
i'll expand on that later.


Capabilities are like the difference between python 0.0 and python 2.5.
python 0.0 is "capable" of doing certain things - however python 2.5
is "capable" of doing a heck of a lot more, simply because the
functionality in python 0.0 doesn't _exist_.

The principle behind Capabilities is therefore to simply _not give_
python 2.5 users _any_ means to use, view, see, certain functionality
i.e. make it look like it doesn't exist.

in other words [and this is the bit that i could _really_ have done
with, two days ago], when a python user accessess __globals__,
__builtins__, __locals__, an object's __class__, they find that
the functionality they may be looking for literally _isn't there_.

it's as if someone had recompiled python 2.5 #ifdeffing out a whole
boat-load of code and forgot to include some modules.


the advantage of Capabilities over ACLs is that there's no chance of
making a mistake on the ACLs, because access to something that, as
far as the user is concerned, doesn't _exist_, is just a non-issue.

the other advantage is speed.


the _dis_advantage of Capabilities over ACLs comes when an attempt to
combine unrestricted Capability code with restricted Capability code
in the same codebase is made, and a security audit is called.

the nature of restricted Capabilities is such that it is "by omission"
that you achieve security.

how the heck are you going to do a security audit looking for things
that are omitted, rather than included, especially in amongst a full
set of code that has the full functionality?

by contrast, with ACLs, you can check everything (in a regimented
fashion) and if it don't have an ACL, you can at least ask _why_ it don't
have an ACL.


so, with that explanation in mind, i'm going to write up a separate
message outlining some of the ways in which i believe the capabilities
restriction system that is already in python can be improved.

l.

From lkcl at lkcl.net  Sat Dec 20 07:19:42 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Sat Dec 20 07:19:23 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220112120.GE1933@lkcl.net>
References: <20031220112120.GE1933@lkcl.net>
Message-ID: <20031220121942.GF1933@lkcl.net>

capabilities: the means to make code disappear.

thing is that there is a lot of code.

some of the boundaries have already been highlighted where
the "Capabilities" have not sufficiently been restricted.

object.__class__ is a good example.

Access to __class__
-------------------

the obvious solution to fix that particular problem is to
turn the access to __class__ around.

the problem is that by creating objects, as many __class__es
are created as there are objects.  considering restricting
access to all of those __class__es is totally daunting and
impractical.

... but look at it closely: they're all the same function name,
so it's not necessarily the individual __class__es that could
be restricted but the method by which __class__ is accessed.

if you "feed" access to __class__ via a single function, you have
a bottleneck again which can be replaced.

so, the proposal is simple: create an __builtin__ function called
__get_class__ through which access to __class__ is made.

it takes, as a first argument, the class itself.

obj.__class__ is redirected to it (__builtin__.__get_class__(obj))
to maintain code compatibility.

then it becomes possible to over-ride the __get_class__ function and
to "vet" it in a restricted execution mode.

again, it's worth reiterating here that if it wasn't for the fact
that python is run "initiated" in "unrestricted" mode and "jumps down"
to "restricted" execution, this wouldn't be a problem.

the reason for that is because if you ran python initiated in a
file-restricted mode from startup, no unrestricted file objects with
write permission would be created, such that there would be zero risk
of anyone _getting_ an unrestricted file object, through which the
file.__class__.__new__(file.__class__, ....) method could be called.


anyway. if that is not practical (for whatever reason), then you leave
it alone, and consider the function __new__ instead, which i believe
is the _real_ reason why access to __class__ is considered bad.

again, turning it round by creating a function __builtin__.__new__()
is a way forward because that function can be restricted.


Access to __builtins__ and __globals__ from the stack.
-----------------------------------------------------

i don't know if this is the case, but it's worth mentioning, as
background, to avoid potential confusion.

there's the concept of globals, locals and there's __builtins__
in the python language.

in the _implementation_ of the python language, there are c-variables
which may be global, may be local: possible conceptual confusion between
the implementation and the language need to be avoided!

my question is:

	_is_ it possible to make a function call in python
	such that that function's "view" of what the globals,
	locals and __builtins__ are are COMPLETELY different
	from what the CALLER of that function's view of globals,
	locals and __builtins__?

in other words, is globals() global in the python implementation,
or is globals() copied on the function-call stack frame in the
python implementation? [or other]

what i am getting at is that it needs to be possible, if capabilities
are to work, to be able to make a transition in the python
language from a full-featured mode in one function call into a
totally restricted mode in another function call, and to be able
to come back again.

_without_ the entry into the restricted mode having any damaging
consequences on the full-featured mode.

i note, for example, in the rexec.py code, that only self.locals
is modified in the runcode() example RestrictedConsole class.

why is only self.locals['__builtins__'] restricted, not self.globals?

am i missing something here?

surely the globals should be modified too?

what happens if self.globals is reassigned?




Access to interfaces
--------------------

one of the things that is strangely lacking in python is the ability
to restrict access to python objects, a la public, protected and
private from c++.

perl users find this to be utterly incomprehensible and reprehensible,
especially the bits where conventions are obeyed - and followed! - about
putting underscores in front of function names.

from a restricted execution perspective, this is not really okay.

what i propose is that two additional per-class functions (or lists)
be added (two more functions later) which can ONLY be called from
__init__: they are called __set_public_interface__() and
__set_protected_interface__().

these functions take, as arguments, a list - or simpler yet,
two variables are added which can only be initialised in __init__:
__public_interface__ and __protected_interface__.

basically they consist of lists of functions that can be accessed
publicly and by inheriting classes respectively.

the reason for not specifying a list of private functions is because
it then becomes necessary to explicitly search through that list
excluding things from external access, which is not the way
"Capabilities" are done.

if the lists are None, there are no restrictions.

if the lists are empty, no functions are accessible - which is
different.

what i also propose is that it be possible to make a copy of a
class and then _remove_ items from these two lists.

it must _not_ be possible to either add to or to replace the list
items or the lists themselves.


there's quite a lot here.

i hope you find it useful.

l.


From aahz at pythoncraft.com  Sat Dec 20 09:08:43 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sat Dec 20 09:08:55 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220121942.GF1933@lkcl.net>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
Message-ID: <20031220140842.GA5627@panix.com>

On Sat, Dec 20, 2003, Luke Kenneth Casson Leighton wrote:
>
> one of the things that is strangely lacking in python is the ability
> to restrict access to python objects, a la public, protected and
> private from c++.
>
> perl users find this to be utterly incomprehensible and reprehensible,
> especially the bits where conventions are obeyed - and followed! -
> about putting underscores in front of function names.
>
> from a restricted execution perspective, this is not really okay.

Yup.  That really is the fundamental issue.  Python has been deliberately
designed to let programmers do whatever they want; it's a "let the
grownups play" perspective.  That's part of the reason restricted
execution hasn't received more attention: it's fundamentally opposed to
Python's design.  Another major reason is because restricted execution
from within Python can never completely solve the problem unless we
devote vast amounts of effort.  Consider the following two snippets of
code:

    100 ** 100 ** 100
    [None] * (10 ** 10)

The first chews up CPU; the second chews up memory.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From lkcl at lkcl.net  Sat Dec 20 09:39:10 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Sat Dec 20 09:38:47 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220140842.GA5627@panix.com>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<20031220140842.GA5627@panix.com>
Message-ID: <20031220143910.GA19472@lkcl.net>

On Sat, Dec 20, 2003 at 09:08:43AM -0500, Aahz wrote:
> On Sat, Dec 20, 2003, Luke Kenneth Casson Leighton wrote:
> >
> > one of the things that is strangely lacking in python is the ability
> > to restrict access to python objects, a la public, protected and
> > private from c++.
> >
> > perl users find this to be utterly incomprehensible and reprehensible,
> > especially the bits where conventions are obeyed - and followed! -
> > about putting underscores in front of function names.
> >
> > from a restricted execution perspective, this is not really okay.
> 
> Yup.  That really is the fundamental issue.  Python has been deliberately
> designed to let programmers do whatever they want; it's a "let the
> grownups play" perspective.  

 as python moves into a more mainstream acceptance, it becomes more
 of an issue to let the kiddies bash themselves with rubber hammers.

 [i mention this because a few years ago, a six year old bashed his dad
  over the head with a hammer after watching tom and jerry beat the
  stuffing out of each other on telly.]


> Python's design.  Another major reason is because restricted execution
> from within Python can never completely solve the problem unless we
> devote vast amounts of effort.  Consider the following two snippets of
> code:
> 
>     100 ** 100 ** 100
>     [None] * (10 ** 10)
> 
> The first chews up CPU; the second chews up memory.

 i infer from these two observations that some of the requirements
 of restricted execution could be to conserve CPU and memory?

 l.


From aahz at pythoncraft.com  Sat Dec 20 10:16:29 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sat Dec 20 10:16:33 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220143910.GA19472@lkcl.net>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<20031220140842.GA5627@panix.com> <20031220143910.GA19472@lkcl.net>
Message-ID: <20031220151629.GA24651@panix.com>

On Sat, Dec 20, 2003, Luke Kenneth Casson Leighton wrote:
> On Sat, Dec 20, 2003 at 09:08:43AM -0500, Aahz wrote:
>> On Sat, Dec 20, 2003, Luke Kenneth Casson Leighton wrote:
>>>
>>> one of the things that is strangely lacking in python is the ability
>>> to restrict access to python objects, a la public, protected and
>>> private from c++.
>>>
>>> from a restricted execution perspective, this is not really okay.
>> 
>> Yup.  That really is the fundamental issue.  Python has been deliberately
>> designed to let programmers do whatever they want; it's a "let the
>> grownups play" perspective.  
> 
>  as python moves into a more mainstream acceptance, it becomes more
>  of an issue to let the kiddies bash themselves with rubber hammers.

That's an assertion.  I think to a certain extent you'll need to prove
your assertion.

>> Another major reason is because restricted execution from within
>> Python can never completely solve the problem unless we devote vast
>> amounts of effort.  Consider the following two snippets of code:
>>
>>     100 ** 100 ** 100
>>     [None] * (10 ** 10)
>> 
>> The first chews up CPU; the second chews up memory.
> 
>  i infer from these two observations that some of the requirements
>  of restricted execution could be to conserve CPU and memory?

Exactly.  From my observations of these discussions, there are
essentially only two reasons for restricted execution:

* To simplify things by reducing the potential solution space

* To protect a system against a hostile attacker

There are already many ways to achieve the first objective in Python
without restricted execution.  For the second, there's really no way to
succeed without help from the operating system, unless you run a server
with a limited set of capabilities that does *not* include random code.
There are already plenty of servers written in Python....

Supposedly there's a middle ground of untrusted but non-hostile code,
but what's the point of providing support for that?
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From lkcl at lkcl.net  Sat Dec 20 10:30:22 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Sat Dec 20 10:30:00 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220151629.GA24651@panix.com>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<20031220140842.GA5627@panix.com> <20031220143910.GA19472@lkcl.net>
	<20031220151629.GA24651@panix.com>
Message-ID: <20031220153022.GB19472@lkcl.net>

On Sat, Dec 20, 2003 at 10:16:29AM -0500, Aahz wrote:

> >  as python moves into a more mainstream acceptance, it becomes more
> >  of an issue to let the kiddies bash themselves with rubber hammers.
> 
> That's an assertion.  I think to a certain extent you'll need to prove
> your assertion.
 
  'm a bit worried that such proofs would involve painting
  large metal hammers and covering them in plastic to make them
  look less real.

  the resultant cracked skills could possibly land someone in jail.

> Supposedly there's a middle ground of untrusted but non-hostile code,
> but what's the point of providing support for that?

 the example that i gave that was because i wanted to offer a subset
 of python functionality to end-users such that they could run
 DNS lookups, pings, check a web page existed, telnet to a box,
 run commands and check the output.

 so it's running a user's python code on a server where there is a
 networked host being analysed by the user.

 what i _didn't_ want to happen was for that user to run code on
 the server that could damage the server and interfere with the
 100 or so _other_ programs running to analyse 100 _other_ hosts.

 the whole point of using python was to avoid having to write an new
 programming language.

 what i came up with was very very useful.

 to some extent, i didn't care about things like __class__ because
 1) the users weren't that bright.
 2) the user's weren't that hostile.


 rexec fitted the requirements perfectly - and it still does: it's
 just been disabled and also changed into something that stops even
 the library functions from writing to log files.
 i couldn't even use the MySQLdb module which was kinda critical to
 the database-driven backend.

 l.

From mwh at python.net  Sat Dec 20 10:51:34 2003
From: mwh at python.net (Michael Hudson)
Date: Sat Dec 20 10:51:39 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <m3smjfztfq.fsf@mira.informatik.hu-berlin.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "20 Dec 2003 11:15:21
	+0100")
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
Message-ID: <2mwu8rebcp.fsf@starship.python.net>

martin@v.loewis.de (Martin v. L?wis) writes:

> Jeff Epler <jepler@unpythonic.net> writes:
>
>> > Can you find out what $$ is, and what the PIDs and thread IDs of all
>> > participating threads are?
>> 
>> I'm not sure what all information I should try to gather for you.  Let me
>> know if you think this is enough to file a bug report with...  I changed
>> the example to make it clearer that it's the subprocess ignoring the
>> signal that is the problem, not anything in Python that is taking time
>> to notice the death of a child process.
>
> That is an important observation; signals that are blocked in the
> parent process will be blocked in the child process as well.
>
> I'm not sure what to do about this: We apparently *want* the signals
> blocked in the thread, but we don't want them to be blocked in the
> process invoked through system(). Proposals are welcome.

Does pthread_atfork() help?

Cheers,
mwh

-- 
  We've had a lot of problems going from glibc 2.0 to glibc 2.1.
  People claim binary compatibility.  Except for functions they
  don't like.                       -- Peter Van Eynde, comp.lang.lisp

From aahz at pythoncraft.com  Sat Dec 20 10:55:48 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sat Dec 20 10:55:53 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220153022.GB19472@lkcl.net>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<20031220140842.GA5627@panix.com> <20031220143910.GA19472@lkcl.net>
	<20031220151629.GA24651@panix.com>
	<20031220153022.GB19472@lkcl.net>
Message-ID: <20031220155548.GA7420@panix.com>

On Sat, Dec 20, 2003, Luke Kenneth Casson Leighton wrote:
> On Sat, Dec 20, 2003 at 10:16:29AM -0500, Aahz wrote:
>>
>> Supposedly there's a middle ground of untrusted but non-hostile code,
>> but what's the point of providing support for that?
> 
>  the example that i gave that was because i wanted to offer a subset
>  of python functionality to end-users such that they could run
>  DNS lookups, pings, check a web page existed, telnet to a box,
>  run commands and check the output.
> 
>  to some extent, i didn't care about things like __class__ because
>  1) the users weren't that bright.
>  2) the user's weren't that hostile.

Yup.  By "what's the point?" I didn't mean that there were no use cases;
the problem is that such cases are not frequent enough to justify the
effort.

>  rexec fitted the requirements perfectly - and it still does: it's
>  just been disabled and also changed into something that stops even
>  the library functions from writing to log files.
>  i couldn't even use the MySQLdb module which was kinda critical to
>  the database-driven backend.

Well, you're free to maintain rexec as a separate project (or borrow
from the still-maintained Zope system).  But anything shipped as part of
Python can't afford to assume your points 1) and 2).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From lkcl at lkcl.net  Sat Dec 20 11:35:51 2003
From: lkcl at lkcl.net (Luke Kenneth Casson Leighton)
Date: Sat Dec 20 11:35:28 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220155548.GA7420@panix.com>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<20031220140842.GA5627@panix.com> <20031220143910.GA19472@lkcl.net>
	<20031220151629.GA24651@panix.com>
	<20031220153022.GB19472@lkcl.net> <20031220155548.GA7420@panix.com>
Message-ID: <20031220163551.GC19472@lkcl.net>

On Sat, Dec 20, 2003 at 10:55:48AM -0500, Aahz wrote:

> >  to some extent, i didn't care about things like __class__ because
> >  1) the users weren't that bright.
> >  2) the user's weren't that hostile.
> 
> Yup.  By "what's the point?" I didn't mean that there were no use cases;
> the problem is that such cases are not frequent enough to justify the
> effort.
 
 ... which is why i made some recommendations to add in the concept
 of run-time-defineable public and protected class interfaces.

 such a concept 1) fits with the principle of capabilities 2)
 is an enhancement that goes beyond the small requirements of
 restricted execution 3) offers a means through which rexec
 can be implemented.


> >  rexec fitted the requirements perfectly - and it still does: it's
> >  just been disabled and also changed into something that stops even
> >  the library functions from writing to log files.
> >  i couldn't even use the MySQLdb module which was kinda critical to
> >  the database-driven backend.
> 
> Well, you're free to maintain rexec as a separate project (or borrow
> from the still-maintained Zope system).  But anything shipped as part of
> Python can't afford to assume your points 1) and 2).

 i appreciate that.  so it turns into a wishlist: a class named
 RExecDontUseThisItsBrokenForMostPeople
 and a class named RExec which simply pulls that exception.

 l.


From aahz at pythoncraft.com  Sat Dec 20 11:39:31 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sat Dec 20 11:39:35 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220163551.GC19472@lkcl.net>
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<20031220140842.GA5627@panix.com> <20031220143910.GA19472@lkcl.net>
	<20031220151629.GA24651@panix.com>
	<20031220153022.GB19472@lkcl.net> <20031220155548.GA7420@panix.com>
	<20031220163551.GC19472@lkcl.net>
Message-ID: <20031220163931.GA20862@panix.com>

On Sat, Dec 20, 2003, Luke Kenneth Casson Leighton wrote:
> On Sat, Dec 20, 2003 at 10:55:48AM -0500, Aahz wrote:
>>Luke: 
>>>
>>>  to some extent, i didn't care about things like __class__ because
>>>  1) the users weren't that bright.
>>>  2) the user's weren't that hostile.
>> 
>> Yup.  By "what's the point?" I didn't mean that there were no use cases;
>> the problem is that such cases are not frequent enough to justify the
>> effort.
>  
>  ... which is why i made some recommendations to add in the concept
>  of run-time-defineable public and protected class interfaces.
> 
>  such a concept 1) fits with the principle of capabilities 2)
>  is an enhancement that goes beyond the small requirements of
>  restricted execution 3) offers a means through which rexec
>  can be implemented.

Sure.  *If* we decide to implement restricted execution.  As I made
clear with my comments about CPU and memory usage, there's still no
consensus that adding a restricted execution environment to Python
represents a good use of community resources.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From guido at python.org  Fri Dec 19 13:15:24 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Dec 20 12:15:31 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: Your message of "Fri, 19 Dec 2003 18:49:15 +0100."
	<3FE33A1B.5030903@tismer.com> 
References: <3FDE6D35.3090100@tismer.com> <3FE2C5B5.8080208@tismer.com>
	<200312191531.hBJFVgw27430@c-24-5-183-134.client.comcast.net> 
	<3FE33A1B.5030903@tismer.com> 
Message-ID: <200312191815.hBJIFP528036@c-24-5-183-134.client.comcast.net>

> > I *did* notice at least one case where using f_tstate might actually
> > be a mistake: theoretically it's possible that two or more threads
> > alternate calling next() on a generator (if they wrap it in a critical
> > section); AFAICT the f_tstate is never updated.
> 
> Right, f_tstate is never updated.
> I think there is another inconsistent
> situation, which can be created easily.
> If a generator is run by a different thread
> than it's creator, then the frame is run in that
> other thread. eval_fame correctly uses tstate,
> but if tracing is activated, call_trace uses
> frame->f_tstate for no obvious reason, which
> will probably mess up the tracing flags of the wrong
> thread.

Right.

Could you dig through CVS logs to find out when f_tstate was first
introduced?  Maybe there's a clue about why there.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From skip at pobox.com  Sat Dec 20 11:27:34 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Dec 20 12:18:27 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEJDHOAB.tim.one@comcast.net>
References: <200312191539.hBJFdjC27495@c-24-5-183-134.client.comcast.net>
	<LNBBLJKPBEHFEDALKOLCIEJDHOAB.tim.one@comcast.net>
Message-ID: <16356.30838.750692.236147@montanaro.dyndns.org>


    Tim> [Guido]
    >> I would expect that a lot of our code assumes 8-bit characters, and I
    >> personally wouldn't mind if Python was limited to such platforms.

    Tim> Fine by me too.

Then how about adding 

#if UCHAR_MAX != 255
#error "Python's source code currently assumes 8-bit characters."
#endif

right after the HAVE_LIMITS_H test?

Skip

From python at rcn.com  Sat Dec 20 13:47:56 2003
From: python at rcn.com (Raymond Hettinger)
Date: Sat Dec 20 13:48:26 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <200312191815.hBJIFP528036@c-24-5-183-134.client.comcast.net>
Message-ID: <000201c3c729$c8ffcde0$e841fea9@oemcomputer>

> > > I *did* notice at least one case where using f_tstate might
actually
> > > be a mistake: theoretically it's possible that two or more threads
> > > alternate calling next() on a generator (if they wrap it in a
critical
> > > section); AFAICT the f_tstate is never updated.
> >
> > Right, f_tstate is never updated.
> > I think there is another inconsistent
> > situation, which can be created easily.
> > If a generator is run by a different thread
> > than it's creator, then the frame is run in that
> > other thread. eval_fame correctly uses tstate,
> > but if tracing is activated, call_trace uses
> > frame->f_tstate for no obvious reason, which
> > will probably mess up the tracing flags of the wrong
> > thread.
> 
> Right.
> 
> Could you dig through CVS logs to find out when f_tstate was first
> introduced?  Maybe there's a clue about why there.

Here is a head start on the research.

The ceval.c use of tstate goes back to the introduction of generators in
2001.

The use in traceback.c and  sysmodule.c goes back to 1997 when
per-thread globals were factored into a structure to support separate
thread-state management.

Prior to that, the history is more diffuse and harder to follow.  



Raymond Hettinger


---------------------------------------------------------------------




Ceval.c
--------
2.252        (nascheme 21-Jun-01):      PyThreadState *tstate =

2.252        (nascheme 21-Jun-01):       * necessarily their creator. */
2.255        (tim_one  23-Jun-01):      Py_XINCREF(tstate->frame);
2.252        (nascheme 21-Jun-01):      assert(f->f_back == NULL);
2.252        (nascheme 21-Jun-01):      f->f_back = tstate->frame;
2.252        (nascheme 21-Jun-01):

revision 2.255
date: 2001/06/23 05:47:56;  author: tim_one;  state: Exp;  lines: +2 -2
gen_iternext():  Don't assume that the current thread state's frame is
not NULL.  I don't think it can be NULL from Python code, but if using
generators via the C API I expect a NULL frame is possible.

revision 2.252
date: 2001/06/21 02:41:10;  author: nascheme;  state: Exp;  lines: +27
-14
Try to avoid creating reference cycles involving generators.  Only keep
a
reference to f_back when its really needed.  Do a little whitespace
normalization as well.  This whole file is a big war between tabs and
spaces
but now is probably not the time to reindent everything.

traceback.c
------------
2.22         (guido    05-May-97):      PyThreadState *tstate =
frame->f_tstate;
2.22         (guido    05-May-97):      tracebackobject *oldtb =
(tracebackobjec

revision 2.22
date: 1997/05/05 20:56:15;  author: guido;  state: Exp;  lines: +6 -30
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

The story for their life a globals pre-dates what I can


sysmodule.c
------------
2.45         (guido    02-Aug-97):      PyThreadState *tstate =
PyThreadState_Ge
2.45         (guido    02-Aug-97):      PyObject *sd =
tstate->interp->sysdict;
2.57         (guido    05-Oct-99):      if (sd == NULL)

2.41         (guido    05-May-97):      PyThreadState *tstate;
2.41         (guido    05-May-97):      tstate = PyThreadState_Get();
2.41         (guido    05-May-97):      return Py_BuildValue(
2.41         (guido    05-May-97):              "(OOO)",
2.41         (guido    05-May-97):              tstate->exc_type != NULL
? tstat
2.41         (guido    05-May-97):              tstate->exc_value !=
NULL ? tsta
2.41         (guido    05-May-97):              tstate->exc_traceback !=
NULL ?
2.41         (guido    05-May-97):
tstate->exc_traceback :
2.41         (guido    05-May-97): }
2.41         (guido    05-May-97):

revision 2.41
date: 1997/05/05 20:56:11;  author: guido;  state: Exp;  lines: +30 -9
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.


Details of the 2.41 checkin
----------------------------
Index: sysmodule.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/sysmodule.c,v
retrieving revision 2.40
retrieving revision 2.41
diff -c -r2.40 -r2.41
*** sysmodule.c	29 Apr 1997 20:42:30 -0000	2.40
--- sysmodule.c	5 May 1997 20:56:11 -0000	2.41
***************
*** 48,56 ****
  
  #include "osdefs.h"
  
- PyObject *_PySys_TraceFunc, *_PySys_ProfileFunc;
- int _PySys_CheckInterval = 10;
- 
  #if HAVE_UNISTD_H
  #include <unistd.h>
  #endif
--- 48,53 ----
***************
*** 98,103 ****
--- 95,119 ----
  }
  
  static PyObject *
+ sys_exc_info(self, args)
+ 	PyObject *self;
+ 	PyObject *args;
+ {
+ 	PyThreadState *tstate;
+ 	if (!PyArg_Parse(args, ""))
+ 		return NULL;
+ 	tstate = PyThreadState_Get();
+ 	if (tstate == NULL)
+ 		Py_FatalError("sys.exc_info(): no thread state");
+ 	return Py_BuildValue(
+ 		"(OOO)",
+ 		tstate->exc_type != NULL ? tstate->exc_type : Py_None,
+ 		tstate->exc_value != NULL ? tstate->exc_value : Py_None,
+ 		tstate->exc_traceback != NULL ?
+ 			tstate->exc_traceback : Py_None);
+ }
+ 
+ static PyObject *
  sys_exit(self, args)
  	PyObject *self;
  	PyObject *args;
***************
*** 112,123 ****
  	PyObject *self;
  	PyObject *args;
  {
  	if (args == Py_None)
  		args = NULL;
  	else
  		Py_XINCREF(args);
! 	Py_XDECREF(_PySys_TraceFunc);
! 	_PySys_TraceFunc = args;
  	Py_INCREF(Py_None);
  	return Py_None;
  }
--- 128,140 ----
  	PyObject *self;
  	PyObject *args;
  {
+ 	PyThreadState *tstate = PyThreadState_Get();
  	if (args == Py_None)
  		args = NULL;
  	else
  		Py_XINCREF(args);
! 	Py_XDECREF(tstate->sys_tracefunc);
! 	tstate->sys_tracefunc = args;
  	Py_INCREF(Py_None);
  	return Py_None;
  }
***************
*** 127,138 ****
  	PyObject *self;
  	PyObject *args;
  {
  	if (args == Py_None)
  		args = NULL;
  	else
  		Py_XINCREF(args);
! 	Py_XDECREF(_PySys_ProfileFunc);
! 	_PySys_ProfileFunc = args;
  	Py_INCREF(Py_None);
  	return Py_None;
  }
--- 144,156 ----
  	PyObject *self;
  	PyObject *args;
  {
+ 	PyThreadState *tstate = PyThreadState_Get();
  	if (args == Py_None)
  		args = NULL;
  	else
  		Py_XINCREF(args);
! 	Py_XDECREF(tstate->sys_profilefunc);
! 	tstate->sys_profilefunc = args;
  	Py_INCREF(Py_None);
  	return Py_None;
  }
***************
*** 142,148 ****
  	PyObject *self;
  	PyObject *args;
  {
! 	if (!PyArg_ParseTuple(args, "i", &_PySys_CheckInterval))
  		return NULL;
  	Py_INCREF(Py_None);
  	return Py_None;
--- 160,167 ----
  	PyObject *self;
  	PyObject *args;
  {
! 	PyThreadState *tstate = PyThreadState_Get();
! 	if (!PyArg_ParseTuple(args, "i", &tstate->sys_checkinterval))
  		return NULL;
  	Py_INCREF(Py_None);
  	return Py_None;
***************
*** 202,207 ****
--- 221,227 ----
  
  static PyMethodDef sys_methods[] = {
  	/* Might as well keep this in alphabetic order */
+ 	{"exc_info",	sys_exc_info, 0},
  	{"exit",	sys_exit, 0},
  #ifdef COUNT_ALLOCS
  	{"getcounts",	sys_getcounts, 0},
***************
*** 232,238 ****
  	if (list == NULL)
  		return NULL;
  	for (i = 0; _PyImport_Inittab[i].name != NULL; i++) {
! 		PyObject *name =
PyString_FromString(_PyImport_Inittab[i].name);
  		if (name == NULL)
  			break;
  		PyList_Append(list, name);
--- 252,259 ----
  	if (list == NULL)
  		return NULL;
  	for (i = 0; _PyImport_Inittab[i].name != NULL; i++) {
! 		PyObject *name = PyString_FromString(
! 			_PyImport_Inittab[i].name);
  		if (name == NULL)
  			break;
  		PyList_Append(list, name);


From raymond.hettinger at verizon.net  Fri Dec 19 13:13:15 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sat Dec 20 14:00:11 2003
Subject: [Python-Dev] Liceinses in the Docs
Message-ID: <006f01c3c65b$c61b3020$e841fea9@oemcomputer>

I would like to add an section to the Library Reference to fulfill
license requirements for code included in Python.
 
For example, the license for underlying the Mersenne Twister code is
already embedded in the source but it requests to be included in the
docs as well.
 
Any thoughts or objections?
 
 
 
Raymond Hettinger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031219/fb5e107d/attachment.html
From zack at codesourcery.com  Sat Dec 20 14:02:27 2003
From: zack at codesourcery.com (Zack Weinberg)
Date: Sat Dec 20 14:02:34 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <m3k74rzsp5.fsf@mira.informatik.hu-berlin.de> (
	=?iso-8859-1?q?Martin_v._L=F6wis's_message_of?= "20 Dec 2003 11:31:18
	+0100")
References: <LNBBLJKPBEHFEDALKOLCCEJDHNAB.tim.one@comcast.net>
	<87y8tbezcq.fsf@codesourcery.com>
	<m3k74tu86l.fsf@mira.informatik.hu-berlin.de>
	<87wu8sy91n.fsf@egil.codesourcery.com>
	<m3k74rzsp5.fsf@mira.informatik.hu-berlin.de>
Message-ID: <87r7yzuxbw.fsf@egil.codesourcery.com>

martin@v.loewis.de (Martin v. L?wis) writes:

> "Zack Weinberg" <zack@codesourcery.com> writes:
>
>> It does, however, lead to a different problem:
>> 
>>   if (condition)
>>     Py_DECREF (foo);
>>   else               // oops, syntax error here
>
> How so? This expands to
>
> if(condition)
>   if(cond2)action1;
>   else action2;
> else

No, it expands to

 if(condition)
  if(cond2) action1;
  else action2;;
 else

-- note the extra semicolon, which provokes a syntax error.

zw

From fdrake at acm.org  Sat Dec 20 14:20:06 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sat Dec 20 14:20:15 2003
Subject: [Python-Dev] Liceinses in the Docs
In-Reply-To: <006f01c3c65b$c61b3020$e841fea9@oemcomputer>
References: <006f01c3c65b$c61b3020$e841fea9@oemcomputer>
Message-ID: <16356.41190.525408.280943@sftp.fdrake.net>


Raymond Hettinger writes:
 > I would like to add an section to the Library Reference to fulfill
 > license requirements for code included in Python.
 >  
 > For example, the license for underlying the Mersenne Twister code is
 > already embedded in the source but it requests to be included in the
 > docs as well.
 >  
 > Any thoughts or objections?

It should be added to Doc/commontex/license.tex with an explanation of
what it pertains to.

Thanks!


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From pf_moore at yahoo.co.uk  Sat Dec 20 07:02:20 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Sat Dec 20 14:23:26 2003
Subject: [Python-Dev] Re: Relative import
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz>
	<20031219213031.GD1246@wopr>
Message-ID: <vfobzohf.fsf@yahoo.co.uk>

Jack Diederich <jack@performancedrivers.com> writes:

> I like syntax that reads most important left-to-right, so what about
> from MODULE import NAMES as RENAME searching HOW
> or
> import NAMES as RENAME from MODULE searching HOW

I like this. It feels much more Pythonic than the various suggestions
around magic dots in the name. It's explicit, easy to understand, and
easy to extend (extra possibilities for HOW could be added with little
difficulty).

Paul.
-- 
This signature intentionally left blank


From tim.one at comcast.net  Sat Dec 20 14:35:34 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 14:35:42 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <87r7yzuxbw.fsf@egil.codesourcery.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEMMHOAB.tim.one@comcast.net>

[Zack Weinberg]
>>> It does, however, lead to a different problem:
>>>
>>>   if (condition)
>>>     Py_DECREF (foo);
>>>   else               // oops, syntax error here

[Martin v. L?wis]
>> How so? This expands to
>>
>> if(condition)
>>   if(cond2)action1;
>>   else action2;
>> else

[Zack]
> No, it expands to
>
>  if(condition)
>   if(cond2) action1;
>   else action2;;
>  else
>
> -- note the extra semicolon, which provokes a syntax error.

Zack, which version of Python are you using?  The current definition of
Py_DECREF doesn't end with a semicolon:

#define Py_DECREF(op)                               \
        if (_Py_DEC_REFTOTAL  _Py_REF_DEBUG_COMMA   \
            --(op)->ob_refcnt != 0)                 \
		    _Py_CHECK_REFCNT(op)                \
        else                                        \
                _Py_Dealloc((PyObject *)(op))

The expansion of _Py_CHECK_REFCNT(op) always ends with a semicolon (and may
be nothing but a semicolon, depending on build type), but that's internal to
the stuff Py_DECREF generates, and the leading underscore in
_Py_CHECK_REFCNT's name makes it off-limits (by convention) for direct use
by user-written code (_Py_CHECK_REFCNT is internal to Python's
implementation, and Python internals guarantee to use it correctly).

The expansion of Py_DECREF's else branch never ends with a semicolon, and
Martin's explanation is correct.

Same deal with Py_INCREF, Py_XINCREF, and Py_XDECREF.

As a pragmatic matter, it's very unusual to need to do

    if (whatever)
        Py_DECREF(op);

The functionality is common enough when "whatever" is "op != NULL", but
accomplishing "decref if not NULL" is Py_XDECREF's purpose -- there's no
need to code your own if-block in that case, and it's bad style (because
non-idiomatic) to do so.


From tim.one at comcast.net  Sat Dec 20 14:41:47 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 14:41:51 2003
Subject: [Python-Dev] Liceinses in the Docs
In-Reply-To: <006f01c3c65b$c61b3020$e841fea9@oemcomputer>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEMOHOAB.tim.one@comcast.net>

[Raymond Hettinger]
> I would like to add an section to the Library Reference to fulfill
> license requirements for code included in Python.
>
> For example, the license for underlying the Mersenne Twister code is
> already embedded in the source but it requests to be included in the
> docs as well.
>
> Any thoughts or objections?

+1.  Even in cases where it's not legally required, I think it's plain good
manners to acknowledge external work Python incorporates, from bsddb to
zlib.


From zack at codesourcery.com  Sat Dec 20 14:42:51 2003
From: zack at codesourcery.com (Zack Weinberg)
Date: Sat Dec 20 14:42:56 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEMMHOAB.tim.one@comcast.net> (Tim
	Peters's message of "Sat, 20 Dec 2003 14:35:34 -0500")
References: <LNBBLJKPBEHFEDALKOLCEEMMHOAB.tim.one@comcast.net>
Message-ID: <87n09nuvgk.fsf@egil.codesourcery.com>

"Tim Peters" <tim.one@comcast.net> writes:

> [Zack Weinberg]
>>>> It does, however, lead to a different problem:
>>>>
>>>>   if (condition)
>>>>     Py_DECREF (foo);
>>>>   else               // oops, syntax error here
>
> [Martin v. L?wis]
>>> How so? This expands to
>>>
>>> if(condition)
>>>   if(cond2)action1;
>>>   else action2;
>>> else
>
> [Zack]
>> No, it expands to
>>
>>  if(condition)
>>   if(cond2) action1;
>>   else action2;;
>>  else
>>
>> -- note the extra semicolon, which provokes a syntax error.
>
> Zack, which version of Python are you using?  The current definition of
> Py_DECREF doesn't end with a semicolon:

I thought you might come back at me with that.  Yes, you can avoid the
syntax error by carefully leaving off the semicolon here.  I don't
consider this to be good style, though.  It's easy to forget, and it's
easy for someone to come along later and be confused and check in an
'obvious' patch that adds a semicolon - and it might not even break
anything... then.

I'm really not speaking about the python source code in specific.  I'm
speaking about general style principles, and I really do think it is
good style *always* to wrap complex macros in do { } while (0) whether
or not they need it.

zw

From tim.one at comcast.net  Sat Dec 20 15:06:51 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 15:06:56 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <16356.30838.750692.236147@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEMPHOAB.tim.one@comcast.net>

[Skip Montanaro]
> Then how about adding
>
> #if UCHAR_MAX != 255
> #error "Python's source code currently assumes 8-bit characters."
> #endif
>
> right after the HAVE_LIMITS_H test?

I wouldn't object.  It should probably then also have

#ifndef UCHAR_MAX
#error ...
#endif

right before it, and stringobject.c's

#if !defined(HAVE_LIMITS_H) && !defined(UCHAR_MAX)
#define UCHAR_MAX 255
#endif

should go away.

From anthony at python.org  Fri Dec 19 21:12:09 2003
From: anthony at python.org (Anthony Baxter)
Date: Sat Dec 20 15:12:30 2003
Subject: [Python-Dev] RELEASED Python 2.3.3 (final)
Message-ID: <200312200212.hBK2C9Ni011271@localhost.localdomain>


On behalf of the Python development team and the Python community, I'm
happy to announce the release of Python 2.3.3 (final).

Python 2.3.3 is a bug-fix release of Python 2.3. A couple of serious
bugs related to weakrefs and the cyclic garbage collector have been 
fixed, along with a number of bugs in the standard library. See the 
release notes on the web page for more details.

For more information on Python 2.3.3, including download links for
various platforms, release notes, and known issues, please see

    http://www.python.org/2.3.3

Highlights of this new release include:

  - A couple of serious bugs in the interactions of weakrefs and 
    cyclic GC have been squashed.
  - At shutdown, the second call to the cyclic garbage collector has
    been removed. This caused more problems than it solved.
  - The xml.parsers.expat module now provides Expat 1.95.7.
  - Bug #823328: urllib2's HTTP Digest Auth support works again.
  - See http://www.python.org/2.3.3/NEWS.html for other bugfixes.

Highlights of the previous major Python release (2.3) are available     
from the Python 2.3 page, at                                            

    http://www.python.org/2.3/highlights.html

Enjoy the new release,
Anthony

Anthony Baxter
anthony@python.org
Python 2.3.x Release Manager
(on behalf of the entire python-dev team)

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 226 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20031220/1b7d245a/attachment.bin
From tismer at tismer.com  Fri Dec 19 14:49:38 2003
From: tismer at tismer.com (Christian Tismer)
Date: Sat Dec 20 15:51:56 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <200312191815.hBJIFP528036@c-24-5-183-134.client.comcast.net>
References: <3FDE6D35.3090100@tismer.com> <3FE2C5B5.8080208@tismer.com>
	<200312191531.hBJFVgw27430@c-24-5-183-134.client.comcast.net>
	<3FE33A1B.5030903@tismer.com>
	<200312191815.hBJIFP528036@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE35652.5000709@tismer.com>

Guido van Rossum wrote:

...

> Could you dig through CVS logs to find out when f_tstate was first
> introduced?  Maybe there's a clue about why there.

Yeehaa! Quite some ride...

Well, f_tstate was introduced here, in frameobject.c:
-----------------------------------------------------
Revision : 2.29
Date : 1997/5/5 20:55:52
Author : 'guido'
State : 'Exp'
Lines : +13 -2
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

For frameobject.h, it was this version:
---------------------------------------
Revision : 2.23
Date : 1997/5/5 20:55:43
Author : 'guido'
State : 'Exp'
Lines : +3 -1
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

Especially interesting:
There is no single hint mentioning f_tstate.
But, the diff shows

	PyObject *f_exc_type, *f_exc_value, *f_exc_traceback;
	PyThreadState *f_tstate;

added at the same time.

This was Python 1.5, I remember very well, the time
where threading became the standard for Python.

ceval.c was at this revision:
-----------------------------
Revision : 2.118
Date : 1997/5/5 20:55:58
Author : 'guido'
State : 'Exp'
Lines : +263 -151
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

I fetched ceval 2.117 and 2.118 and compared them.
This was exactly the time when the thread state was introduced.
There were exactly two references to f_tstate:

The most interesting diff for this question:

  PyObject *
  PyEval_SaveThread()
  {
  #ifdef WITH_THREAD
  	if (interpreter_lock) {
-		PyObject *res;
-		res = (PyObject *)current_frame;
-		current_frame = NULL;
+		PyThreadState *tstate = PyThreadState_Swap(NULL);
+		PyObject *res = tstate ? (PyObject *) (tstate->frame) : NULL;
  		release_lock(interpreter_lock);
  		return res;
  	}
  #endif
  	return NULL;
  }

  void
  PyEval_RestoreThread(x)
  	PyObject *x;
  {
  #ifdef WITH_THREAD
  	if (interpreter_lock) {
  		int err;
  		err = errno;
  		acquire_lock(interpreter_lock, 1);
  		errno = err;
-		current_frame = (PyFrameObject *)x;
+		PyThreadState_Swap(x ? ((PyFrameObject *)x)->f_tstate : NULL);
  	}
  #endif
  }

PyEval_SaveThread has changed a bit meanwhile,
no longer return the frame, but tstate.
semantics are similar.

The two functions clearly show hwo the transition from
a global current_frame variable was made to the
tstate mechanism. f_tstate was the source for switching
the thread state.

The current ceval version looks like this:

void
PyEval_RestoreThread(PyThreadState *tstate)
{
	if (tstate == NULL)
		Py_FatalError("PyEval_RestoreThread: NULL tstate");
#ifdef WITH_THREAD
	if (interpreter_lock) {
		int err = errno;
		PyThread_acquire_lock(interpreter_lock, 1);
		errno = err;
	}
#endif
	PyThreadState_Swap(tstate);
}

That means, tstate is not fetched from the frame, but
given as a parameter.
I checked all places where this function gets called,
the tstate parameter never comes from a frame.

As a conclusion, f_tstate appears to be an intermediate
step during the move to threading. It was an easy place
to keep the thread state. Its use was reduced to
very few later on.

call_trace introduced today's slightly false behavior
exactly in that 2.118 version. It looks very much like
an optimization, since there was no macro for fetching
tstate at that time.
This was perfectly fine, until generators were introduced.
Until then, a frame was always bound to its thread.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From tim.one at comcast.net  Sat Dec 20 16:07:15 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 16:07:21 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <87n09nuvgk.fsf@egil.codesourcery.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCCENFHOAB.tim.one@comcast.net>

[Zack Weinberg]
> I thought you might come back at me with that.  Yes, you can avoid the
> syntax error by carefully leaving off the semicolon here.  I don't
> consider this to be good style, though.  It's easy to forget, and it's
> easy for someone to come along later and be confused and check in an
> 'obvious' patch that adds a semicolon - and it might not even break
> anything... then.

Zack, you have to be kidding.  Martin was clearly talking about the
expansion of Py_DECREF specifically, not "macros in general", as also was
the poster who started this thread.

> I'm really not speaking about the python source code in specific.

That's now obvious <wink>.

> I'm speaking about general style principles, and I really do think it is
> good style *always* to wrap complex macros in do { } while (0) whether
> or not they need it.

Py_DECREF doesn't need it; there may be some point to it if doing so shuts
up gcc nuisance complaints, and we can be sure that the extra complexity of
the expanded code doesn't provoke optimization bugs in any of the C
compilers used to compile Python.  Writing non-loops *as* loops is a good
way to tickle endcase bugs in sophisticated optimizers -- it's not without
risk, and there are several thousand instances of Py_DECREF and Py_XDECREF
just in the core.  Also because it's used a lot, in a debug build such a
change would generate a ton of useless extra instructions (at least MSVC
really does generate code to create a zero, test it against zero, and jump
back to the top of "the loop" if zero doesn't equal zero; I expect that most
C compilers would).


From zack at codesourcery.com  Sat Dec 20 16:17:50 2003
From: zack at codesourcery.com (Zack Weinberg)
Date: Sat Dec 20 16:17:56 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <LNBBLJKPBEHFEDALKOLCCENFHOAB.tim.one@comcast.net> (Tim
	Peters's message of "Sat, 20 Dec 2003 16:07:15 -0500")
References: <LNBBLJKPBEHFEDALKOLCCENFHOAB.tim.one@comcast.net>
Message-ID: <87iskbur29.fsf@egil.codesourcery.com>

"Tim Peters" <tim.one@comcast.net> writes:

> [Zack Weinberg]
>> I thought you might come back at me with that.  Yes, you can avoid the
>> syntax error by carefully leaving off the semicolon here.  I don't
>> consider this to be good style, though.  It's easy to forget, and it's
>> easy for someone to come along later and be confused and check in an
>> 'obvious' patch that adds a semicolon - and it might not even break
>> anything... then.
>
> Zack, you have to be kidding.  Martin was clearly talking about the
> expansion of Py_DECREF specifically, not "macros in general", as also was
> the poster who started this thread.

I, however, was never speaking about Py_DECREF specifically.  I
thought this was clear from previous statements but I apologize for
the confusion.

>> I'm speaking about general style principles, and I really do think it is
>> good style *always* to wrap complex macros in do { } while (0) whether
>> or not they need it.
>
> Py_DECREF doesn't need it; there may be some point to it if doing so shuts
> up gcc nuisance complaints, and we can be sure that the extra complexity of
> the expanded code doesn't provoke optimization bugs in any of the C
> compilers used to compile Python.  Writing non-loops *as* loops is a good
> way to tickle endcase bugs in sophisticated optimizers -- it's not without
> risk, and there are several thousand instances of Py_DECREF and Py_XDECREF
> just in the core.  Also because it's used a lot, in a debug build such a
> change would generate a ton of useless extra instructions (at least MSVC
> really does generate code to create a zero, test it against zero, and jump
> back to the top of "the loop" if zero doesn't equal zero; I expect that most
> C compilers would).

I doubt both statements just because this is a standard C idiom.
Recognizing it and generating no extra code falls naturally out 
of basic constant-condition detection, which I would expect to
be present in any modern compiler, - and done even in "debug
builds" because it can't hurt and in fact usually makes the
compiler run faster (less junk intermediat code to process).
I especially doubt that this construct will tickle bugs in
sophisticated optimizers -- it would be harder to get it 
wrong than to get it right.

I am not disputing that MSVC does this silly thing you describe, but
MSVC has a long track record of bugs and poor code generation.  GCC
does not generate junk code even at -O0.

zw

From tim.one at comcast.net  Sat Dec 20 17:13:34 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 17:13:39 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <87iskbur29.fsf@egil.codesourcery.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCAENKHOAB.tim.one@comcast.net>

[Zack Weinberg]
> I doubt both statements just because this is a standard C idiom.

I worked in compiler development for 15 years, so I'll have to trust my
experience on this.  The union of MSVC and gcc accounts for the vast bulk of
Python installations, but only a fraction of the total number of compilers
in use.  Many optimizers know nothing at all about C specifically, as
they're a common back end for many front-end intermediate code generators,
for many languages.  It's unusual for a compiler to do any sort of
optimization unless optimizations are specifically requested.  I don't have
any gripe with gcc doing some regardless, but gcc isn't the universe.

> Recognizing it and generating no extra code falls naturally out
> of basic constant-condition detection,

No, changing the flow graph is a serious and delicate program
transformation.  A C compiler front end doing it more-than-less
"mindlessly", as part of special-casing the goofy idioms C encourages, has a
fine shot at never screwing up, because it's doing it then before flow graph
construction.  Not all compilers do it that way.

> which I would expect to be present in any modern compiler,

In one way or another, yes, at least optimizing compilers.  Not all
compilers optimize, though.  A possibly important example for Python in the
near future is that the "standard edition" retail .NET compilers have no
optimization capability, and neither does the freely downloadable .NET C/C++
compiler.

> ...
> I am not disputing that MSVC does this silly thing you describe, but
> MSVC has a long track record of bugs

That doesn't match our experience.  I don't believe Python code has ever
tickled an MSVC codegen or optimization bug, neither any in gcc-for-x86.
We've certainly tickled codegen and optimization bugs in gcc for other
platforms, and *many* such bugs in vendor compilers (SGI's native compilers
probably hold the record for sheer quantity of codegen and optimization bugs
when compiling Python).

At Dragon Systems we were using C++ rather than C, and I think MSVC and gcc
were blamed for about an equal number of codegen and optimization bugs.

> and poor code generation.

This also doesn't match our experience.  Release-build Python compiled under
MSVC for x86 has been faster than release-build Python compiled under gcc
for x86 each time I've seen them compared, and the same was true for
Dragon's software.  Maybe gcc is getting better on x86; I don't know; for a
long time it was at a systematic disadvantage because it was originally
designed for RISC-ish architectures with a large, uniform register set, and
x86 is about as from that model as things get.  MSVC had many years'
advantage in dealing with x86's many architecural quirks.

> GCC does not generate junk code even at -O0.

It certainly generates needless code even at -O0.  If it didn't, there would
be no reason to have a higher optimization level than -O0 <wink>.


From skip at pobox.com  Sat Dec 20 20:07:32 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sat Dec 20 20:08:21 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEMPHOAB.tim.one@comcast.net>
References: <16356.30838.750692.236147@montanaro.dyndns.org>
	<LNBBLJKPBEHFEDALKOLCMEMPHOAB.tim.one@comcast.net>
Message-ID: <16356.62036.22726.310077@montanaro.dyndns.org>


    >> #if UCHAR_MAX != 255
    >> #error "Python's source code currently assumes 8-bit characters."
    >> #endif
    >> 
    >> right after the HAVE_LIMITS_H test?

    Tim> I wouldn't object.  It should probably then also have

    Tim> #ifndef UCHAR_MAX
    Tim> #error ...
    Tim> #endif

Isn't that supposed to always be defined in limits.h or is UCHAR_MAX not a
standard macro?

    Tim> right before it, and stringobject.c's

    Tim> #if !defined(HAVE_LIMITS_H) && !defined(UCHAR_MAX)
    Tim> #define UCHAR_MAX 255
    Tim> #endif

    Tim> should go away.

Sounds like a plan.  I modify my local source and see if it affects
anything.

Skip

From tim.one at comcast.net  Sat Dec 20 21:52:03 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sat Dec 20 21:52:09 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <16356.62036.22726.310077@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net>

[Skip Montanaro]
> Isn't that supposed to always be defined in limits.h or is UCHAR_MAX
> not a standard macro?

Yes, it's supposed to be there.  OTOH, so is limits.h, i.e. the
HAVE_LIMITS_H test shouldn't be necessary either.  So they're just sanity
checks.  But you're right, if UCHAR_MAX isn' defined, I was thinking the
preprocessor would expand

#if UCHAR_MAX != 255

to

#if != 255

and then the error message would be incomprehensible.  But unknown names in
preprocessor conditionals actually get replaced by 0, and

#if 0 != 255

does just as well.

> Sounds like a plan.  I modify my local source and see if it affects
> anything.

It should work fine.


From anthony at interlink.com.au  Sat Dec 20 22:14:03 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Sat Dec 20 22:14:41 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Include
	patchlevel.h, 2.74.4.9, 2.74.4.10 
In-Reply-To: <E1AXofx-0005rH-00@sc8-pr-cvs1.sourceforge.net> 
Message-ID: <200312210314.hBL3E3ZU017703@localhost.localdomain>


>>> fdrake@users.sourceforge.net wrote
> Update of /cvsroot/python/python/dist/src/Include
> In directory sc8-pr-cvs1:/tmp/cvs-serv22509
> 
> Modified Files:
>       Tag: release23-maint
> 	patchlevel.h 
> Log Message:
> Post-release version bumpage.

Argh! Nonononononono! Please leave the branch at 2.3.3 for Jack to cut 
the Mac versions!

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From fdrake at acm.org  Sun Dec 21 00:12:00 2003
From: fdrake at acm.org (Fred L. Drake, Jr.)
Date: Sun Dec 21 00:12:16 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Include
	patchlevel.h, 2.74.4.9, 2.74.4.10 
In-Reply-To: <200312210314.hBL3E3ZU017703@localhost.localdomain>
References: <E1AXofx-0005rH-00@sc8-pr-cvs1.sourceforge.net>
	<200312210314.hBL3E3ZU017703@localhost.localdomain>
Message-ID: <16357.11168.459937.182871@sftp.fdrake.net>


Anthony Baxter writes:
 > Argh! Nonononononono! Please leave the branch at 2.3.3 for Jack to cut 
 > the Mac versions!

Oops, sorry; forgot about that.  It was triggered by a change that
Raymond made to the docs, properly removing material that leaked from
the 2.4 tutorial.

I've outdated the change to correct the version numbers.  Not sure
what the right thing to do about the docs is; let's just leave it
alone, since it was a valuable correction.


  -Fred

-- 
Fred L. Drake, Jr.  <fdrake at acm.org>
PythonLabs at Zope Corporation

From tismer at tismer.com  Sun Dec 21 03:45:30 2003
From: tismer at tismer.com (Christian Tismer)
Date: Sun Dec 21 04:08:19 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <200312191815.hBJIFP528036@c-24-5-183-134.client.comcast.net>
References: <3FDE6D35.3090100@tismer.com> <3FE2C5B5.8080208@tismer.com>
	<200312191531.hBJFVgw27430@c-24-5-183-134.client.comcast.net>
	<3FE33A1B.5030903@tismer.com>
	<200312191815.hBJIFP528036@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE55DAA.2020808@tismer.com>

<this message didn't make it>

Guido van Rossum wrote:

...

> Could you dig through CVS logs to find out when f_tstate was first
> introduced?  Maybe there's a clue about why there.

Yeehaa! Quite some ride...

Well, f_tstate was introduced here, in frameobject.c:
-----------------------------------------------------
Revision : 2.29
Date : 1997/5/5 20:55:52
Author : 'guido'
State : 'Exp'
Lines : +13 -2
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

For frameobject.h, it was this version:
---------------------------------------
Revision : 2.23
Date : 1997/5/5 20:55:43
Author : 'guido'
State : 'Exp'
Lines : +3 -1
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

Especially interesting:
There is no single hint mentioning f_tstate.
But, the diff shows

	PyObject *f_exc_type, *f_exc_value, *f_exc_traceback;
	PyThreadState *f_tstate;

added at the same time.

This was Python 1.5, I remember very well, the time
where threading became the standard for Python.

ceval.c was at this revision:
-----------------------------
Revision : 2.118
Date : 1997/5/5 20:55:58
Author : 'guido'
State : 'Exp'
Lines : +263 -151
Description :
Massive changes for separate thread state management.
All per-thread globals are moved into a struct which is manipulated
separately.

I fetched ceval 2.117 and 2.118 and compared them.
This was exactly the time when the thread state was introduced.
There were exactly two references to f_tstate:

The most interesting diff for this question:

  PyObject *
  PyEval_SaveThread()
  {
  #ifdef WITH_THREAD
  	if (interpreter_lock) {
-		PyObject *res;
-		res = (PyObject *)current_frame;
-		current_frame = NULL;
+		PyThreadState *tstate = PyThreadState_Swap(NULL);
+		PyObject *res = tstate ? (PyObject *) (tstate->frame) : NULL;
  		release_lock(interpreter_lock);
  		return res;
  	}
  #endif
  	return NULL;
  }

  void
  PyEval_RestoreThread(x)
  	PyObject *x;
  {
  #ifdef WITH_THREAD
  	if (interpreter_lock) {
  		int err;
  		err = errno;
  		acquire_lock(interpreter_lock, 1);
  		errno = err;
-		current_frame = (PyFrameObject *)x;
+		PyThreadState_Swap(x ? ((PyFrameObject *)x)->f_tstate : NULL);
  	}
  #endif
  }

PyEval_SaveThread has changed a bit meanwhile,
no longer return the frame, but tstate.
semantics are similar.

The two functions clearly show hwo the transition from
a global current_frame variable was made to the
tstate mechanism. f_tstate was the source for switching
the thread state.

The current ceval version looks like this:

void
PyEval_RestoreThread(PyThreadState *tstate)
{
	if (tstate == NULL)
		Py_FatalError("PyEval_RestoreThread: NULL tstate");
#ifdef WITH_THREAD
	if (interpreter_lock) {
		int err = errno;
		PyThread_acquire_lock(interpreter_lock, 1);
		errno = err;
	}
#endif
	PyThreadState_Swap(tstate);
}

That means, tstate is not fetched from the frame, but
given as a parameter.
I checked all places where this function gets called,
the tstate parameter never comes from a frame.

As a conclusion, f_tstate appears to be an intermediate
step during the move to threading. It was an easy place
to keep the thread state. Its use was reduced to
very few later on.

call_trace introduced today's slightly false behavior
exactly in that 2.118 version. It looks very much like
an optimization, since there was no macro for fetching
tstate at that time.
This was perfectly fine, until generators were introduced.
Until then, a frame was always bound to its thread.

cheers - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/






From tismer at tismer.com  Sun Dec 21 03:41:54 2003
From: tismer at tismer.com (Christian Tismer)
Date: Sun Dec 21 04:08:32 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <000201c3c729$c8ffcde0$e841fea9@oemcomputer>
References: <000201c3c729$c8ffcde0$e841fea9@oemcomputer>
Message-ID: <3FE55CD2.9040707@tismer.com>

Raymond Hettinger wrote:

...

> Here is a head start on the research.
> 
> The ceval.c use of tstate goes back to the introduction of generators in
> 2001.
> 
> The use in traceback.c and  sysmodule.c goes back to 1997 when
> per-thread globals were factored into a structure to support separate
> thread-state management.
> 
> Prior to that, the history is more diffuse and harder to follow.  

Thanks a lot.
I did a similar research on Friday, but for some reason it
was not accepted by python.org, as it appears.
(why, only 4 k, yours was 10?)

I will try to send it again.

My summary: f_tstate not needed!

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From martin at v.loewis.de  Sun Dec 21 05:27:45 2003
From: martin at v.loewis.de (Martin v. =?iso-8859-15?q?L=F6wis?=)
Date: Sun Dec 21 05:27:59 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <2mwu8rebcp.fsf@starship.python.net>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
Message-ID: <m3smjea2ji.fsf@mira.informatik.hu-berlin.de>

Michael Hudson <mwh@python.net> writes:

> > I'm not sure what to do about this: We apparently *want* the signals
> > blocked in the thread, but we don't want them to be blocked in the
> > process invoked through system(). Proposals are welcome.
> 
> Does pthread_atfork() help?

Most likely. system(3) is specified as being implemented through
fork()/exec(), so an atfork handler should be invoked in any compliant
implementation. We could install a child handler, which unblocks the
signals we don't want to be blocked.

Now, the question is: What signals precisely we don't want to be
blocked? I think the answer is "All signals that have not explicitly
been blocked by the application".

OTOH, we already have PyOS_AfterFork, which could be used instead of
pthread_atfork. Jeff, would you like to add some code there, to set
all signal handlers into default for which Handlers lists that the
default handling should occur?

Regards,
Martin


From ncoghlan at iinet.net.au  Sun Dec 21 08:16:58 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Sun Dec 21 08:17:03 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031219213031.GD1246@wopr>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>	<20031216141746.GA3145@burma.localdomain>	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>	<1071714822.27808.88.camel@anthem>
	<20031218030409.GG473@frobozz>	<1071716900.27808.114.camel@anthem>
	<20031218031645.GH473@frobozz> <20031219213031.GD1246@wopr>
Message-ID: <3FE59D4A.8050403@iinet.net.au>

Jack Diederich wrote:
> I like syntax that reads most important left-to-right, so what about
> from MODULE import NAMES as RENAME searching HOW
> or
> import NAMES as RENAME from MODULE searching HOW

This would work really well for any uses cases I can imagine having. It 
also has the following benefits:
1. No magic punctuation
2. People who don't need it, just never use a 'searching' clause
3. People encountering it for the first time have a clear flag letting 
them know something fancy is going on
4. These people have a word they can look up in the Python docs

I hadn't even thought of point 4 prior to considering Jack's proposal. 
If I encounter "import .fred.mary" in some python I didn't write, how 
the heck do I find out what it means?

[...]
> ps, I like the second version better but it is less like the current version,
> which is a drawback.

To head a little further down this sidetrack, the from version of 
'import' has puzzled me for a while:

A. from mypkg import foo
B. import mypkg.foo as foo

What, if any, are the semantic differences between A & B? Is it only 
that A2 works, and B2 doesn't?

A2. from mymodule import *
B2. import mymodule.*

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From aahz at pythoncraft.com  Sun Dec 21 12:30:15 2003
From: aahz at pythoncraft.com (Aahz)
Date: Sun Dec 21 12:30:18 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: <200312181750.hBIHof425371@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312181204540.10960-100000@slab.zope.com>
	<200312181750.hBIHof425371@c-24-5-183-134.client.comcast.net>
Message-ID: <20031221173014.GA20633@panix.com>

On Thu, Dec 18, 2003, Guido van Rossum wrote:
>
> OK.  Wanna write a PEP?  Just kidding, I know you have less time than
> I do for it.  So... *anybody* wanna write a PEP about this?  (The
> parentheses proposal should also be folded in.)

Since nobody else has volunteered, I guess I'll write the PEP.  Expect
it sometime in the next couple of weeks.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From tim.one at comcast.net  Sun Dec 21 13:29:06 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 21 13:29:11 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <3FE55CD2.9040707@tismer.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEAMHPAB.tim.one@comcast.net>

[Christian Tismer]
> I did a similar research on Friday, but for some reason it
> was not accepted by python.org, as it appears.
> (why, only 4 k, yours was 10?)

Your original was accepted:

   http://mail.python.org/pipermail/python-dev/2003-December/041287.html

I think there may have been a long delay in sending it out, though, perhaps
related to the python.org domain transfer, or to hardware fiddling recently
done at mail.python.org's physical site.  Whichever, it's in the archive,
and I got it via email too.

> My summary: f_tstate not needed!

More, it seems conceptually flawed, albeit subtly.  Python's runtime is, in
effect, simulating thread-local storage by hand, and the current value of
_PyThreadState_Current always points to a PyThreadState struct holding the
currently-executing thread's *conceptual* thread-local storage.  If we were
able to use honest-to-goodness TLS instead, it wouldn't have been possible
(well, not w/o ugly hacks) for a frame to use the values of these guys
associated with any thread other than the one currently executing.  Or, IOW,
the existence of f_tstate creates a possibility for inter-thread mixups.

Still, the possibility to switch threads across generator resumptions seems
darned hard to view as "a feature".  I'd advise people not to rely on it
<wink>.


From skip at pobox.com  Sun Dec 21 15:03:54 2003
From: skip at pobox.com (Skip Montanaro)
Date: Sun Dec 21 15:03:59 2003
Subject: [Python-Dev] make test failing in a spectacular way in
	test_codeccallbacks
Message-ID: <16357.64682.286170.38911@montanaro.dyndns.org>


Fresh cvs up (no local changes) on Mac OS X 10.2.8 yields this while running
test_codeccallbacks (non-framework build, configured only using
--with-pydebug):

    malloc[20582]: protecting edges
    malloc[20582]: recording stacks using standard recorder
    malloc[20582]: enabling scribbling to detect mods to free blocks
    malloc[20582]: checks heap after 1th operation and each 1000 operations
    malloc[20582]: environment variables that can be set for debug:
    - MallocGuardEdges to add 2 guard pages for each large block
    - MallocDoNotProtectPrelude to disable protection (when previous flag set)
    - MallocDoNotProtectPostlude to disable protection (when previous flag set)
    - MallocStackLogging to record all stacks.  Tools like leaks can then be applied
    - MallocStackLoggingNoCompact to record all stacks.  Needed for malloc_history
    - MallocScribble to detect writing on free blocks: 0x55 is written upon free
    - MallocCheckHeapStart <n> to check the heap from time to time after <n> operations 
    - MallocHelp - this help!
    MallocCheckHeap: PASSED check at 1th operation
    MallocCheckHeap: PASSED check at 1001th operation
    MallocCheckHeap: PASSED check at 2001th operation
    MallocCheckHeap: PASSED check at 3001th operation
    test_backslashescape (__main__.CodecCallbackTest) ... ok
    test_badandgoodbackslashreplaceexceptions (__main__.CodecCallbackTest) ... ok
    test_badandgoodignoreexceptions (__main__.CodecCallbackTest) ... ok
    test_badandgoodreplaceexceptions (__main__.CodecCallbackTest) ... ok
    test_badandgoodstrictexceptions (__main__.CodecCallbackTest) ... ok
    test_badandgoodxmlcharrefreplaceexceptions (__main__.CodecCallbackTest) ... ok
    test_badhandlerresults (__main__.CodecCallbackTest) ... ok
    test_badregistercall (__main__.CodecCallbackTest) ... ok
    test_bug828737 (__main__.CodecCallbackTest) ... ok
    test_callbacks (__main__.CodecCallbackTest) ... ok
    test_charmapencode (__main__.CodecCallbackTest) ... ok
    test_decodehelper (__main__.CodecCallbackTest) ... ok
    MallocCheckHeap: PASSED check at 4001th operation
    test_encodehelper (__main__.CodecCallbackTest) ... ok
    Debug memory block at address p=0x7cb138:
        7032 bytes originally requested
        The 4 pad bytes at p-4 are FORBIDDENBYTE, as expected.
        The 4 pad bytes at tail=0x7cccb0 are not all FORBIDDENBYTE (0xfb):
            at tail+0: 0x33 *** OUCH
            at tail+1: 0x35 *** OUCH
            at tail+2: 0x34 *** OUCH
            at tail+3: 0x3b *** OUCH
        The block was made by call #639840562 to debug malloc/realloc.
        Data at p: 00 00 00 00 00 00 00 00 ... 33 35 34 3b 26 23 31 32
    Fatal Python error: bad trailing pad byte
    Abort trap

The "Data at p" hex bytes map to ['3', '5', '4', ';', '&', '#', '1', '2'].
The Malloc messages are all from Apple's malloc() pkg with a bunch of debug
environment variables set.

I haven't dug into this any further.  I applied a new OS X security update
yesterday.  That's the only recent change to the system I'm aware of.  GCC
is

    [GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin

as reported by the Python interpreter at startup.

Anyone else seen this?

Skip

From raymond.hettinger at verizon.net  Sun Dec 21 17:15:03 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Sun Dec 21 17:15:32 2003
Subject: [Python-Dev] Vinay Sajip
Message-ID: <000601c3c80f$e2a5fee0$0dae2c81@oemcomputer>

FYI, 
 
Vinay Sajip has been granted CVS commit privileges for the purpose of
maintaining and extending his logging package.
 
 
Raymond Hettinger
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031221/67ed5d5c/attachment.html
From jepler at unpythonic.net  Sun Dec 21 18:07:04 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Sun Dec 21 18:07:09 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
Message-ID: <20031221230655.GA4417@unpythonic.net>

On Sun, Dec 21, 2003 at 11:27:45AM +0100, Martin v. L?wis wrote:
> OTOH, we already have PyOS_AfterFork, which could be used instead of
> pthread_atfork. Jeff, would you like to add some code there, to set
> all signal handlers into default for which Handlers lists that the
> default handling should occur?

When using pthread_atfork, os.system never triggers my code.  However,
reimplementing os.system in terms of os.fork+os.execv, it does.  I don't
know if this is right or wrong according to pthread, but since it doesn't
work on my platform the question is academic for me.

Wouldn't the PyOS_AfterFork approach also require python to provide its
own versions of any POSIX APIs that would typically be implemented in
terms of fork (system(), popen(), and spawn*() come to mind)?

Jeff

Index: Python/thread_pthread.h
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/thread_pthread.h,v
retrieving revision 2.48
diff -u -r2.48 thread_pthread.h
--- Python/thread_pthread.h	19 Nov 2003 22:52:22 -0000	2.48
+++ Python/thread_pthread.h	21 Dec 2003 23:03:52 -0000
@@ -143,6 +143,17 @@
  * Initialization.
  */
 
+static void PyThread__fork_child(void) {
+	/* Mask all signals in the current thread before creating the new
+	 * thread.  This causes the new thread to start with all signals
+	 * blocked.
+	 */
+	sigset_t childmask;
+	sigfillset(&childmask);
+	SET_THREAD_SIGMASK(SIG_UNBLOCK, &childmask, NULL);
+	fprintf(stderr, "PyThread__fork_child()\n"); fflush(stderr);
+}
+
 #ifdef _HAVE_BSDI
 static
 void _noop(void)
@@ -157,6 +168,7 @@
 	pthread_t thread1;
 	pthread_create(&thread1, NULL, (void *) _noop, &dummy);
 	pthread_join(thread1, NULL);
+	pthread_atfork(NULL, NULL, PyThread__fork_child);
 }
 
 #else /* !_HAVE_BSDI */
@@ -167,6 +179,7 @@
 #if defined(_AIX) && defined(__GNUC__)
 	pthread_init();
 #endif
+	pthread_atfork(NULL, NULL, PyThread__fork_child);
 }
 
 #endif /* !_HAVE_BSDI */

From guido at python.org  Sun Dec 21 19:03:50 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 21 19:03:57 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Sun, 21 Dec 2003 12:30:15 EST."
	<20031221173014.GA20633@panix.com> 
References: <Pine.LNX.4.44.0312181204540.10960-100000@slab.zope.com>
	<200312181750.hBIHof425371@c-24-5-183-134.client.comcast.net> 
	<20031221173014.GA20633@panix.com> 
Message-ID: <200312220003.hBM03ph08816@c-24-5-183-134.client.comcast.net>

> Since nobody else has volunteered, I guess I'll write the PEP.  Expect
> it sometime in the next couple of weeks.

Thanks, Aahz!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tismer at tismer.com  Sun Dec 21 19:23:51 2003
From: tismer at tismer.com (Christian Tismer)
Date: Sun Dec 21 19:23:37 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEAMHPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEAMHPAB.tim.one@comcast.net>
Message-ID: <3FE63997.4000600@tismer.com>

Tim Peters wrote:

...

> Your original was accepted:
> 
>    http://mail.python.org/pipermail/python-dev/2003-December/041287.html

Yup, I wasn't aware of the DNS change.

>>My summary: f_tstate not needed!
> 
> More, it seems conceptually flawed, albeit subtly.

...

Very much agreed. I always felt that f_tstate was not quite ok.
You spelled it and supplied more reason.

> Still, the possibility to switch threads across generator resumptions seems
> darned hard to view as "a feature".  I'd advise people not to rely on it
> <wink>.

Well, this might only work for generators if it is guaranteed
that the thread switch doesn't happen while the generator
is executing. I think this will give an exception. But most
probably the result will be hard to predict.

Other for my "tasklet" tiny threads. Their first purpose is
to have concurrent small tasks without interaction (or if
with interaction, then via channels and blocking). A tasklet
being interrupted by a thread switch will not be auto-schedulable
until the thread is resumed, but other tasklets might be, and
they don't need to care about which real thread is actually in
charge. This is one of my reasons why I want to get rid of
f_tstate -- it does not really apply to most of my frames.
Being tied to a thread is a property of the tasklet, and it
does not apply all the time, only when I have to keep C state.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From jcarlson at uci.edu  Sun Dec 21 19:41:45 2003
From: jcarlson at uci.edu (Josiah Carlson)
Date: Sun Dec 21 19:43:37 2003
Subject: [Python-Dev] Got None.  Maybe Some?
In-Reply-To: <200312220003.hBM03ph08816@c-24-5-183-134.client.comcast.net>
References: <20031221173014.GA20633@panix.com>
	<200312220003.hBM03ph08816@c-24-5-183-134.client.comcast.net>
Message-ID: <20031221163032.10E2.JCARLSON@uci.edu>

Hey,

I'm a new lurker (hoping to change that) here on the python-dev list,
and I thought I would post a message about a new PEP I recently
submitted entitled "A Case for Some".

The PEP talks about how there exists None, that is false, and that is
smaller than any other object ( min(None, a) -> None ).  However, there
doesn't exist an equivalent true value that is larger than any other
number, like ( max(Some, a) -> Some ).

Personally, I think the introduction of Some offers a nice compliment to
None, especially because it allows algorithms that require an
initialization of infinity to not require large numbers.  Some would be
sufficient.  It also has the side benefit of being funny.

I just wanted to get the discussion going.  How does everyone feel about
Some?

Thanks,
 - Josiah Carlson


From guido at python.org  Sun Dec 21 19:49:54 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 21 19:50:02 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: Your message of "Sun, 21 Dec 2003 16:41:45 PST."
	<20031221163032.10E2.JCARLSON@uci.edu> 
References: <20031221173014.GA20633@panix.com>
	<200312220003.hBM03ph08816@c-24-5-183-134.client.comcast.net> 
	<20031221163032.10E2.JCARLSON@uci.edu> 
Message-ID: <200312220049.hBM0nsU09360@c-24-5-183-134.client.comcast.net>

> I'm a new lurker (hoping to change that) here on the python-dev list,
> and I thought I would post a message about a new PEP I recently
> submitted entitled "A Case for Some".
> 
> The PEP talks about how there exists None, that is false, and that is
> smaller than any other object ( min(None, a) -> None ).  However, there
> doesn't exist an equivalent true value that is larger than any other
> number, like ( max(Some, a) -> Some ).
> 
> Personally, I think the introduction of Some offers a nice compliment to
> None, especially because it allows algorithms that require an
> initialization of infinity to not require large numbers.  Some would be
> sufficient.  It also has the side benefit of being funny.
> 
> I just wanted to get the discussion going.  How does everyone feel about
> Some?

-1.  Depending on your algorithm you can already use sys.maxint, or a
large float (e.g. 1e300), or you can write 1e1000 (or 1e300**2) which
on most systems evaluates to a floating point Infinity.  And "Some"
sounds like a really strange spelling of "Infinity".  There's also a
PEP (PEP 754) to add a proper way to spell floating point Infinity.
Or you could reverse signs and use min(None, -a).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Sun Dec 21 20:07:59 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 21 20:08:03 2003
Subject: [Python-Dev] make test failing in a spectacular way
	intest_codeccallbacks
In-Reply-To: <16357.64682.286170.38911@montanaro.dyndns.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCAECJHPAB.tim.one@comcast.net>

[Skip]
> Fresh cvs up (no local changes) on Mac OS X 10.2.8 yields this while
> running test_codeccallbacks (non-framework build, configured only
> using --with-pydebug):
> ...
> Anyone else seen this?

Just tried on Windows.  test_codeccallbacks crashes here in release or debug
builds.  Here from a debug build:

C:\Code\python\PCbuild>python_d ../lib/test/test_codeccallbacks.py
test_backslashescape (__main__.CodecCallbackTest) ... ok
test_badandgoodbackslashreplaceexceptions (__main__.CodecCallbackTest) ...
ok
test_badandgoodignoreexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodreplaceexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodstrictexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodxmlcharrefreplaceexceptions (__main__.CodecCallbackTest) ...
ok
test_badhandlerresults (__main__.CodecCallbackTest) ... ok
test_badregistercall (__main__.CodecCallbackTest) ... ok
test_bug828737 (__main__.CodecCallbackTest) ... ok
test_callbacks (__main__.CodecCallbackTest) ... ok
test_charmapencode (__main__.CodecCallbackTest) ... ok
test_decodehelper (__main__.CodecCallbackTest) ... ok
test_encodehelper (__main__.CodecCallbackTest) ... ok
test_longstrings (__main__.CodecCallbackTest) ...
C:\Code\python\PCbuild>python_d ../lib/test/test_codeccallbacks.py
test_backslashescape (__main__.CodecCallbackTest) ... ok
test_badandgoodbackslashreplaceexceptions (__main__.CodecCallbackTest) ...
ok
test_badandgoodignoreexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodreplaceexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodstrictexceptions (__main__.CodecCallbackTest) ... ok
test_badandgoodxmlcharrefreplaceexceptions (__main__.CodecCallbackTest) ...
ok
test_badhandlerresults (__main__.CodecCallbackTest) ... ok
test_badregistercall (__main__.CodecCallbackTest) ... ok
test_bug828737 (__main__.CodecCallbackTest) ... ok
test_callbacks (__main__.CodecCallbackTest) ... ok
test_charmapencode (__main__.CodecCallbackTest) ... ok
test_decodehelper (__main__.CodecCallbackTest) ... ok
test_encodehelper (__main__.CodecCallbackTest) ... ok
test_longstrings (__main__.CodecCallbackTest) ...

and at that point it dies with a segfault.  It's really hosed at that
point -- I can't get into the debugger.  If I start the test under the
debugger, the debugger craps out.  So no easy clues from me either.

Just to cheer things up <wink>, the bsddb3 tests are in much worse shape now
on Win98SE than they are in 2.3.3 (about 50 more errors under current CVS).

I expect that 2.3.3 will be the last Python release <wink>.

BTW, when I cvs up'ed and recompiled to run the above, the only file I
noticed getting recompiled was unicodeobject.c.


From martin at v.loewis.de  Sun Dec 21 20:08:34 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Sun Dec 21 20:09:38 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <20031221230655.GA4417@unpythonic.net>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
Message-ID: <3FE64412.3010500@v.loewis.de>

Jeff Epler wrote:

> When using pthread_atfork, os.system never triggers my code.  However,
> reimplementing os.system in terms of os.fork+os.execv, it does.  I don't
> know if this is right or wrong according to pthread, but since it doesn't
> work on my platform the question is academic for me.

Interesting. I'd be curious to find out why this fails - it may be a bug
in your system, in which case I'd say "tough luck, complain to the
system vendor" (for Redhat 9, I'm tempted to say that anyway ...)

Looking at what likely is the source of your system(3) implementation
(glibc 2.3.2, sysdeps/unix/sysv/linux/i386/system.c), I see that the
fork used inside system(3) is

# define FORK() \
   INLINE_SYSCALL (clone, 3, CLONE_PARENT_SETTID | SIGCHLD, 0, &pid)

Atleast, this is the fork being used if __ASSUME_CLONE_THREAD_FLAGS
is defined, which is the case for Linux >2.5.50.

With this fork() implementation, atfork handlers won't be invoked,
which clearly looks like a bug to me. You might want to upgrade glibc
to glibc-2.3.2-27.9.7.i686.rpm and nptl-devel-2.3.2-27.9.7.i686.rpm.
In this version, the definition of FORK is changed to

#if defined __ASSUME_CLONE_THREAD_FLAGS && !defined FORK
# define FORK() \
   INLINE_SYSCALL (clone, 3, CLONE_PARENT_SETTID | SIGCHLD, 0, &pid)
#endif

which might actually do the right thing, assuming FORK is already
defined to one that calls the atfork handlers.

> Wouldn't the PyOS_AfterFork approach also require python to provide its
> own versions of any POSIX APIs that would typically be implemented in
> terms of fork (system(), popen(), and spawn*() come to mind)?

You are right. system(3) won't call our version of fork, so
PyOS_AfterFork won't be invoked. So forget about this approach.

Regards,
Martin


From perky at i18n.org  Sun Dec 21 20:39:37 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Sun Dec 21 20:39:43 2003
Subject: [Python-Dev] make test failing in a spectacular way
	intest_codeccallbacks
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAECJHPAB.tim.one@comcast.net>
References: <16357.64682.286170.38911@montanaro.dyndns.org>
	<LNBBLJKPBEHFEDALKOLCAECJHPAB.tim.one@comcast.net>
Message-ID: <20031222013937.GA92298@i18n.org>

On Sun, Dec 21, 2003 at 08:07:59PM -0500, Tim Peters wrote:
> [Skip]
> > Fresh cvs up (no local changes) on Mac OS X 10.2.8 yields this while
> > running test_codeccallbacks (non-framework build, configured only
> > using --with-pydebug):
> > ...
> > Anyone else seen this?
> 
> Just tried on Windows.  test_codeccallbacks crashes here in release or debug
> builds.  Here from a debug build:
> 
[snip]
> and at that point it dies with a segfault.  It's really hosed at that
> point -- I can't get into the debugger.  If I start the test under the
> debugger, the debugger craps out.  So no easy clues from me either.

Just fixed in unicodeobject.c 2.206. I'm sorry for breaking the
regrtest.  I guess that I ran regrtest with built previous for
checking sanity before the commit. (./python versus python)


Hye-Shik

From jepler at unpythonic.net  Sun Dec 21 21:07:37 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Sun Dec 21 21:07:36 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <3FE64412.3010500@v.loewis.de>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
Message-ID: <20031222020728.GA15895@unpythonic.net>

On Mon, Dec 22, 2003 at 02:08:34AM +0100, Martin v. Loewis wrote:
> With this fork() implementation, atfork handlers won't be invoked,
> which clearly looks like a bug to me. You might want to upgrade glibc
> to glibc-2.3.2-27.9.7.i686.rpm and nptl-devel-2.3.2-27.9.7.i686.rpm.
> In this version, the definition of FORK is changed to

I also tested a fedora machine with 
	glibc-2.3.2-101.1
	nptl-devel-2.3.2-101.1
and the pthread_atfork handler is still not called for system().

Opengroup's documentation for system() says
	[...] The environment of the executed command shall be as if a
	child process were created using fork(), and the child process
	invoked the sh utility using execl() [...]

	http://www.opengroup.org/onlinepubs/007904975/functions/system.html

so this looks like a glibc/nptl bug.  I filed it in redhat's bugzilla:
https://bugzilla.redhat.com/bugzilla/show_bug.cgi?id=112517

Given all this, does it make sense to adopt a patch similar to the one
I posted earlier, and ignore the bug in system() on any particular OS?
system() and popen() are easy to replace in Python if anybody is really
bothered by the problem.

Jeff

From tim.one at comcast.net  Sun Dec 21 21:29:22 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 21 21:29:27 2003
Subject: [Python-Dev] make test failing in a spectacular
	wayintest_codeccallbacks
In-Reply-To: <20031222013937.GA92298@i18n.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCKECPHPAB.tim.one@comcast.net>

[Hye-Shik Chang]
> Just fixed in unicodeobject.c 2.206.

Worked for me -- thanks!

> I'm sorry for breaking the regrtest.  I guess that I ran regrtest
> with built previous for checking sanity before the commit. (./python
> versus python)

That's OK.  Don't make the same mistake again, and even Guido will forgive
you <wink>.


From tjreedy at udel.edu  Sun Dec 21 21:36:44 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Sun Dec 21 21:36:46 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
Message-ID: <bs5lbp$ut1$1@sea.gmane.org>


"Luke Kenneth Casson Leighton" <lkcl@lkcl.net>
wrote in message
news:20031220121942.GF1933@lkcl.net...
> if you "feed" access to __class__ via a single
function, you have
> a bottleneck again which can be replaced.
>
> so, the proposal is simple: create an
__builtin__ function called
> __get_class__ through which access to __class__
is made.
>

I wonder if it would not be better to turn
__class__ into a property with a get function that
conditionally gives out the real thing.

TJR




From tim.one at comcast.net  Sun Dec 21 21:52:48 2003
From: tim.one at comcast.net (Tim Peters)
Date: Sun Dec 21 21:52:53 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <3FE63997.4000600@tismer.com>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEDCHPAB.tim.one@comcast.net>

[Tim]
>> Still, the possibility to switch threads across generator
>> resumptions seems darned hard to view as "a feature".  I'd advise
>> people not to rely on it <wink>.

[Christian Tismer]
> Well, this might only work for generators if it is guaranteed
> that the thread switch doesn't happen while the generator
> is executing. I think this will give an exception. But most
> probably the result will be hard to predict.

I'm not sure I'm following, but nothing in Python now *stops* you from
resuming a suspended generator from another thread.  Here's an extreme
example of that:

"""
import threading

def myid():
    while True:
        yield threading.currentThread()
next = myid().next

mut = threading.Lock()

class Worker(threading.Thread):
    def run(self):
        while True:
            mut.acquire()
            print next()
            mut.release()

for i in range(3):
    Worker().start()
"""

Then the body of the single generator is executed by 3 different threads,
more or less randomly, although never by the thread that created the
generator.

If the body of the generator were relying on thread-local state (like, e.g.,
the mass of settings in the decimal arithmetic package's per-thread default
"numeric context" object), the result would sure be unpredictable.  It
wouldn't crash Python, it would just be a bitch to debug.  Then again, lots
of bad threading practices lead to bitch-to-debug code, so I'm not really
inclined to offer extra protection here.

> Other for my "tasklet" tiny threads. Their first purpose is
> to have concurrent small tasks without interaction (or if
> with interaction, then via channels and blocking). A tasklet
> being interrupted by a thread switch will not be auto-schedulable
> until the thread is resumed, but other tasklets might be, and
> they don't need to care about which real thread is actually in
> charge. This is one of my reasons why I want to get rid of
> f_tstate -- it does not really apply to most of my frames.
> Being tied to a thread is a property of the tasklet, and it
> does not apply all the time, only when I have to keep C state.

If a tasklet can be affected by thread-local state (like numeric context --
Python has little of this nature now, but it's bound to increase), then a
tasklet will also start to care "who's in charge".  So I expect a patch to
reintroduce f_tstate in about a year <wink>.


From martin at v.loewis.de  Sun Dec 21 21:46:14 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Sun Dec 21 22:52:44 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <20031222020728.GA15895@unpythonic.net>
References: <1071669660.18260.28.camel@gandalf.tm.informatik.uni-frankfurt.de>
	<20031217151450.GB28436@unpythonic.net>
	<20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
Message-ID: <3FE65AF6.6010403@v.loewis.de>

Jeff Epler wrote:

> Given all this, does it make sense to adopt a patch similar to the one
> I posted earlier, and ignore the bug in system() on any particular OS?
> system() and popen() are easy to replace in Python if anybody is really
> bothered by the problem.

I'd think so, yes. The patch needs to be elaborated, and I'd delay
integration to wait for a Redhat response. We should also restrict
this for 2.4 at the moment, to find out how other pthread
implementations react to pthread_atfork.

Regards,
Martin


From guido at python.org  Sun Dec 21 23:48:01 2003
From: guido at python.org (Guido van Rossum)
Date: Sun Dec 21 23:48:09 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Fri, 19 Dec 2003 16:30:31 EST."
	<20031219213031.GD1246@wopr> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz> 
	<20031219213031.GD1246@wopr> 
Message-ID: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>

> I like syntax that reads most important left-to-right, so what about
> from MODULE import NAMES as RENAME searching HOW
> or
> import NAMES as RENAME from MODULE searching HOW
> 
> searching could be 'absolute' 'relative' or any of the other
> suggested words.  If you wanted to get fancy it could be a list, if
> it isn't a list people who truly care could cascade try/except on
> ImportError.
> 
> -jackdied
> 
> ps, I like the second version better but it is less like the current version,
> which is a drawback.

This got several encouraging responses, so I have to consider it.

Unfortunately it looks like 'searching' would have to become a
reserved word; if we use the same trick we used to avoid making 'as' a
reserved word, that would be too confusing to decode in the code
generator.  The HOW could be just an identifier then.

I'm not sure how well it'll read in actual examples.  Let's try.
Adapting an example from Opal that was used here before...

  from COGS import generate searching relative

To me, that looks a lot like someone left the commas out of

  from COGS import generate, searching, relative

In general I'm not a big fan of "wordy" clauses like this -- they work
better in languages like SQL or COBOL because those are case-sensitive
so you can use a difference in capitalization there; there the former
would be

  FROM COGS IMPORT generate SEARCHING RELATIVE

(using imaginary syntax) while the latter would be

  FROM COGS IMPORT generate, searching, relative

while in Python the long sequence of lowercase words becomes a blur.

So if the two alternatives are simply 'searching absolute' and
'searching relative', with the default being 'searching absolute' in
Python 3.0 (but the current ambiguous search in Python 2.x) I'd still
prefer making the distinction with "from ...MODULE" vs. "from MODULE"
rather than with a searching clause.  And I still prefer the
single-dot syntax over three dots, because it can be used to spell
importing from the parent or grandparent package explicitly.

A separate suggestion is to switch from "from X import Y" to "import Y
from X".  This seems a gratuitous change even if it reads or edits
marginally better.  Now's not the time to tinker with marginal stuff
-- that should've been done 10 years ago.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jack at performancedrivers.com  Mon Dec 22 02:52:19 2003
From: jack at performancedrivers.com (Jack Diederich)
Date: Mon Dec 22 02:52:24 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem>
	<20031218031645.GH473@frobozz> <20031219213031.GD1246@wopr>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
Message-ID: <20031222075219.GA3426@performancedrivers.com>

On Sun, Dec 21, 2003 at 08:48:01PM -0800, Guido van Rossum wrote:
> > I like syntax that reads most important left-to-right, so what about
> > from MODULE import NAMES as RENAME searching HOW
> > or
> > import NAMES as RENAME from MODULE searching HOW
> > 
> > searching could be 'absolute' 'relative' or any of the other
> > suggested words.  If you wanted to get fancy it could be a list, if
> > it isn't a list people who truly care could cascade try/except on
> > ImportError.
> 
> Unfortunately it looks like 'searching' would have to become a
> reserved word; if we use the same trick we used to avoid making 'as' a
> reserved word, that would be too confusing to decode in the code
> generator.  The HOW could be just an identifier then.

Too bad the decorator PEP (PEP 318) seems to favor 'as' in the syntax,

import foo [relative] 

is more explicit than the dot syntax and has a nice visual seperator
while avoiding the wordiness problem of 'searching'

> I'm not sure how well it'll read in actual examples.  Let's try.
> Adapting an example from Opal that was used here before...
> 
>   from COGS import generate searching relative
> 
> To me, that looks a lot like someone left the commas out of
> 
>   from COGS import generate, searching, relative

moving the 'import' to always be first would alleviate this a bit
because 'searching' would always follow a non-list.

import generate from COGS searching relative

or if we decorate

import generate from COGS [relative]
from COGS import generate [relative]

> In general I'm not a big fan of "wordy" clauses like this -- they work
> better in languages like SQL or COBOL because those are case-sensitive
> so you can use a difference in capitalization there
[snip]
> while in Python the long sequence of lowercase words becomes a blur.

as above, moving import to be first helps but doesn't eliminate the
wordiness problem[*].  'as' and 'from' are short enough to be
easilly seen as keywords, 'find' or 'look' reads less like english but 
would be easier to pick out visually than 'searching'

> So if the two alternatives are simply 'searching absolute' and
> 'searching relative'

Did someone mention 'searching scan' in dusty corner somewhere?
One drawback to the dot syntax is if there can be more than two ways
to do the search.  Adding a third way is easier w/ the searching
clause instead of adding a 'bang' syntax to the 'dot' syntax .  The 
decorator would do one better by being a placeholder for anything that 
modifies the import.

> A separate suggestion is to switch from "from X import Y" to "import Y
> from X".  This seems a gratuitous change even if it reads or edits
> marginally better.  Now's not the time to tinker with marginal stuff
> -- that should've been done 10 years ago.

Damn, I really liked that part.  I don't get tripped up on 
from X import Y
anymore, but I did when I started w/ python.

benign-user-for-life-ly,

-jackdied

[*] I use an emacs hack to display 'lambda' as 'L' to avoid the same
    keyword-reads-like-a-variable effect.

From jcarlson at uci.edu  Mon Dec 22 03:13:14 2003
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Dec 22 03:15:46 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: <20031222075219.GA3426@performancedrivers.com>
References: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
Message-ID: <20031222000819.10EC.JCARLSON@uci.edu>

I just realized I've been forgetting to CC the list.

It is a bit long, but I'll cut out the non-relevant portions that
everyone has seen before.  In chronological order below.

I ask the rest of you, does Some sound so crazy?
 - Josiah


*** message from Josiah Carlson on Sun, 21 Dec 2003 19:05:45 -0800 ***

> -1.  Depending on your algorithm you can already use sys.maxint, or a
> large float (e.g. 1e300), or you can write 1e1000 (or 1e300**2) which
> on most systems evaluates to a floating point Infinity.  And "Some"
> sounds like a really strange spelling of "Infinity".  There's also a
> PEP (PEP 754) to add a proper way to spell floating point Infinity.
> Or you could reverse signs and use min(None, -a).
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

I've read PEP 754, and yes, the author talks about having a proper
Infinity, -Infinity and Not a Number (PosInf, NegInf and NaN
respectively), but considering that it has been in the 'draft' state
since March 28, has not been updated, and it doesn't seem like Python is
any closer to having proper definitions now than it was ~9 months ago,
I'm not going to put my eggs in the IEEE 754 Infinity basket.

In terms of having something useful on a platform, there is only so
useful that infinity can be.  The below is from the windows verstion of
Python 2.3.2
>>> 1e309
1.#INF
>>> -1e309
-1.#INF
>>> 1.#INF
1.0
>>> max(1e309, 0)
1.#INF
>>> min(1e309, 10**309)
Traceback (most recent call last):
  File "<stdin>", line 1, in ?
OverflowError: long int too large to convert to float

The real trick is that it is not so much about being able to have
max(Some, a) -> Some, it is about being able to have min(Some, a) -> a. 
Then you don't need to worry about sys.maxint not being big enough
(because there are larger numbers), nor do you need to worry about
float casts of large integers producing OverflowErrors.

As for an application, many path searches in graphs start out with
initializing distances to infinity, Some would be suitable for this.
Other algorithms require a negative infinity for this, None has been
working fine.  As an initialization value, I believe that Some would be
quite a useful object.

Certainly Some seems like a strange way to spell infinity, but it is a
logical infinity, one that you can compare things against, anything, not
just Python floats or things that can be cast into Python floats.

In terms if PEP 754, if it were actually implemented, certainly there
would be some question as to whether Some would be equal to PosInf.  If
so, then would -Some be equal to NegInf?  And if so, would None be equal
to NegInf?  Maybe such things would require a float cast.

I talk about all this in my PEP submission, which isn't up yet.  I
suppose I probably should have given it a few days before discussing it,
but yeah.

 - Josiah
*** end message from Josiah ***

*** message from Guido van Rossum on Sun, 21 Dec 2003 20:45:34 -0800 ***

> I've read PEP 754, and yes, the author talks about having a proper
> Infinity, -Infinity and Not a Number (PosInf, NegInf and NaN
> respectively), but considering that it has been in the 'draft' state
> since March 28, has not been updated, and it doesn't seem like Python is
> any closer to having proper definitions now than it was ~9 months ago,
> I'm not going to put my eggs in the IEEE 754 Infinity basket.

I expect your PEP will fare no better, and given its weirdness,
probably much worse.  Do you know of any language that has a similar
concept?

--Guido van Rossum (home page: http://www.python.org/~guido/)
*** end message from Guido ***

*** message from Josiah Carlson on Sun, 21 Dec 2003 21:47:34 -0800 ***

> I expect your PEP will fare no better, and given its weirdness,
> probably much worse.  Do you know of any language that has a similar
> concept?

Is None so weird?  All I am proposing is a symmetric value to None. 
Just like there is a -1 and 1, -Infinity and Infinity, None logically
should have Some.  Maybe not so much -Some == None, but as I propose in
the PEP, possibly Some == not None.

 - Josiah
*** end message from Josiah ***

From lists at hlabs.spb.ru  Mon Dec 22 06:45:14 2003
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Mon Dec 22 03:45:11 2003
Subject: [Python-Dev] Importing packages from command line
In-Reply-To: <1071839460.3fe2f8e4dbbad@mcherm.com>
References: <1071839460.3fe2f8e4dbbad@mcherm.com>
Message-ID: <3FE6D94A.2060904@hlabs.spb.ru>

Michael Chermside wrote:
>>Just an idea...
>>
>>1. python -p package_name
>>
>>     Equivalent to:
>>
>>     import package_name
> 
> (1) Sourceforge is a great place to request feature enhancements. 
>     Suggestions made on this mailing list is likely to be forgotten
>     sooner or later.

Yes, I know, but the request is not formed yet. :-)

> 
> (2) Can you explain WHY you would want this feature? Is there some
>     use case? I would prefer NOT to have this, because right now
>     if I'm reading code and it uses "package_name.someFunction()"
>     I can scan upward for "package_name" to find out what it's
>     using. With command line imports, I couldn't do that. So unless
>     you've got a good reason for it, I personally wouldn't want
>     this feature.

Sorry for description absentees.

The main idea is to treating package as a program and run package 
initialization code from command line. The advantage is zipping all 
program modules in one zip archive and running the program from command 
line without unzipping it, like Java's jar's. But this idea need more 
thoughts however...

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)


From troy at gci.net  Mon Dec 22 04:03:08 2003
From: troy at gci.net (Troy Melhase)
Date: Mon Dec 22 04:03:19 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <20031222075219.GA3426@performancedrivers.com>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
Message-ID: <200312220003.08810.troy@gci.net>

On Sunday 21 December 2003 10:52 pm, Jack Diederich wrote:
> On Sun, Dec 21, 2003 at 08:48:01PM -0800, Guido van Rossum wrote:
> > > I like syntax that reads most important left-to-right, so what about
> > > from MODULE import NAMES as RENAME searching HOW
> > > import NAMES as RENAME from MODULE searching HOW

I like Jack's original idea and I've been chewing on it for a few days.  His 
was:

import NAMES as RENAME from MODULE searching HOW

My humble suggestion:

import NAMES
import NAMES in MODULE [in HOW]
import NAME, OTHER in FOO in RELATIVE
import NAME as RENAME in MODULE in ABSOLUTE

Playing with it:

import sys
import path, stat in os
import AClass, AValue in AModule in __search__
import AType in Package.Library in __absolute__

It emphasizes the name that's imported and it reduces+reuses a keyword.  
Reusing "in" is a stretch, of course, and it reading it twice in the same 
statement might be confusing for some.

I don't really like the dot/triple dot notation... the leading punctuation 
smells perl-ish.  :)


Only-a-random-two-cents-ly yours,

-- 
Troy Melhase, troy@gci.net
--
Merry Christmas!


From tismer at tismer.com  Mon Dec 22 05:36:07 2003
From: tismer at tismer.com (Christian Tismer)
Date: Mon Dec 22 05:34:34 2003
Subject: [Python-Dev] Last chance!
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEDCHPAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCOEDCHPAB.tim.one@comcast.net>
Message-ID: <3FE6C917.6080107@tismer.com>

Tim Peters wrote:
> [Tim]
> 
>>>Still, the possibility to switch threads across generator
>>>resumptions seems darned hard to view as "a feature".  I'd advise
>>>people not to rely on it <wink>.
> 
> 
> [Christian Tismer]
> 
>>Well, this might only work for generators if it is guaranteed
>>that the thread switch doesn't happen while the generator
>>is executing. I think this will give an exception. But most
>>probably the result will be hard to predict.

Explaining that: I was talking about freely using
the generator in another thread, which would most
likely resume it while it is active, and that's
an exception.

> I'm not sure I'm following, but nothing in Python now *stops* you from
> resuming a suspended generator from another thread.  Here's an extreme
> example of that:

[XTreme example]

> Then the body of the single generator is executed by 3 different threads,
> more or less randomly, although never by the thread that created the
> generator.
> 
> If the body of the generator were relying on thread-local state (like, e.g.,
> the mass of settings in the decimal arithmetic package's per-thread default
> "numeric context" object), the result would sure be unpredictable.  It
> wouldn't crash Python, it would just be a bitch to debug.  Then again, lots
> of bad threading practices lead to bitch-to-debug code, so I'm not really
> inclined to offer extra protection here.

I agree, sure, cross-thread generators need to
be designed for that.

[tasklets]

> If a tasklet can be affected by thread-local state (like numeric context --
> Python has little of this nature now, but it's bound to increase), then a
> tasklet will also start to care "who's in charge".  So I expect a patch to
> reintroduce f_tstate in about a year <wink>.

It is already there, not in the frames, but in the tasklet.
A tasklet can be thread-bound or not. In that case, it
keeps a reference to the thread state. But only once,
since that information is not local for frames.
Being thead bound is either enforced in the case that
the tasklet is carrying non-trivial C state, otherwise it
can be set as a user option (conservative default, of course).
"Nomad threads" might be nice for some background monitoring.

This is all theory, I haven't tried any real threads
with Stackless so far. But people are mixing tasklets
with threads already, so I have to care *somehow*.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@tismer.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/


From ncoghlan at iinet.net.au  Mon Dec 22 05:50:01 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Mon Dec 22 05:50:05 2003
Subject: [Python-Dev] Importing packages from command line
In-Reply-To: <3FE6D94A.2060904@hlabs.spb.ru>
References: <1071839460.3fe2f8e4dbbad@mcherm.com>
	<3FE6D94A.2060904@hlabs.spb.ru>
Message-ID: <3FE6CC59.10100@iinet.net.au>

Dmitry Vasiliev wrote:

> The main idea is to treating package as a program and run package 
> initialization code from command line. The advantage is zipping all 
> program modules in one zip archive and running the program from command 
> line without unzipping it, like Java's jar's. But this idea need more 
> thoughts however...
> 

   python runzip.py mypkg.zip

where runzip.py is a pure Python script that works as you suggest.

I'm fairly sure zipimport would allow you to write the program you 
describe in pure Python. I've obviously never written such a program, 
though.

It's certainly an interesting packaging idea for small Python programs 
with several files, but for which a full-blown installer would be overkill.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From paul-python at svensson.org  Mon Dec 22 06:48:30 2003
From: paul-python at svensson.org (Paul Svensson)
Date: Mon Dec 22 06:48:38 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz> 
	<20031219213031.GD1246@wopr>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
Message-ID: <20031222060114.K47423@familjen.svensson.org>

Summarizing what I've seen so far:

Status quo:

    [from NAME] import NAME [, NAME]
    [from NAME] import NAME as NAME

There also seems to be consensus on adding:

    [from NAME] import (NAME [, NAME])

To this, we want to as a way to spell out explicitly how the module is found;
for example:
    -a- by searching sys.path
    -b- in the current package
    -c- in the current package or its parent package, recursively
    -d- among the python standard library modules (and nowhere else)
    -e- in the current working directory
    -f- in a specific directory
    -g- combinations of the above

Jack Diederich proposed:

>> from MODULE import NAMES as RENAME searching HOW
>> or
>> import NAMES as RENAME from MODULE searching HOW

Guido van Rossum oppined:

>  from COGS import generate searching relative
>
>To me, that looks a lot like someone left the commas out of
>
>  from COGS import generate, searching, relative

(---)

>A separate suggestion is to switch from "from X import Y" to "import Y
>from X".  This seems a gratuitous change even if it reads or edits
>marginally better.  Now's not the time to tinker with marginal stuff
>-- that should've been done 10 years ago.

Not only a gratuitous change; it was always my impression that "from" was
at the beginning of the statement for exactly the reason Guido is hesitant
about adding "searching" at the end of it.  With this in mind, my suggestion
is to add the searching syntax should before the "import" keyword rather
than at the end of the statement.  Reusing an existing keyword, we get:

    [from NAMES] [in WHERE] import ...

WHERE would then be an object, or a list of objects (-g-) specifying the
import mechanism:

    -d- __stdlib__ or maybe stdlib
    -b- __package__
    -c- __search__
    -e- '.'
    -f- '/home/bellman/python'
    -a- sys.path

Without any "from" clause, "in WHERE import ..." reads a little strangely,
but I think it should be possible to get used to it, just as we've gotten
used to the current "from NAME import ..."

Would this work ?

	/Paul

From skip at pobox.com  Mon Dec 22 08:08:04 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 22 08:08:34 2003
Subject: [Python-Dev] make test failing in a spectacular
	wayintest_codeccallbacks
In-Reply-To: <LNBBLJKPBEHFEDALKOLCKECPHPAB.tim.one@comcast.net>
References: <20031222013937.GA92298@i18n.org>
	<LNBBLJKPBEHFEDALKOLCKECPHPAB.tim.one@comcast.net>
Message-ID: <16358.60596.317938.185310@montanaro.dyndns.org>


    Tim> [Hye-Shik Chang]
    >> Just fixed in unicodeobject.c 2.206.

    Tim> Worked for me -- thanks!

Me also... 

Skip

From mcherm at mcherm.com  Mon Dec 22 08:33:45 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Mon Dec 22 08:33:47 2003
Subject: [Python-Dev] Got None. Maybe Some?
Message-ID: <1072100025.3fe6f2b90ea70@mcherm.com>

  [Josiah proposes a Some object to be the counterpart of None]

  [Guido seems reluctant]

Josiah: I have a counterproposal. The good news is, it's REALLY easy -- you
won't need a PEP, and I am sure that you'll get no objections from anyone
on this list. And it'll provide you with a "Some" object that acts like
you describe (except the one mention you made of "Some == not None"...
that one's a bad idea since right now "True == not None").

Here's the idea: Create a file called "Some.py":

-------- Some.py ------------
class _SomeType(object):
    def __lt__(self, other):
        return False # never less than anything
    def __le__(self, other):
        if other is Some:
            return True # equal to itself
        else:
            return False # less than anything else
    def __gt__(self, other):
        if other is Some:
            return False # equal to itself
        else:
            return True # greater than anything else
    def __ge__(self, other):
        return True # greater than or equal to anything
    def __repr__(self):
        return 'Some'

Some = _SomeType() # singleton instance
---------- end --------------

Place this in your python path, then add this line to the top of your
scripts:

"from Some import Some".

If you really want to, you could put code in your site.py file so
that you don't even need this import line, but I think in the long
run you'll prefer using the import line.

I think that this meets your expectations (if I understood them
properly), and if not, it can probably be adjusted so that it does.
The one thing it doesn't do is that it isn't part of the Standard
Library. But that doesn't mean it can't be part of YOUR standard
library... something you use all the time all over the place. I
think that most people on this list (myself included) would say
that the uses of this object are sufficiently esoteric and the
naming of it is sufficiently quirky that they wouldn't want it in
the language itself, but *everyone* here would stand up for your
right to create and use such an object yourself... something which
would never be possible in (for instance) Java.

Once-we've-got-Some-perhaps-we-should-consider-a-builtin-Milk lly yours,

-- Michael Chermside


From jepler at unpythonic.net  Mon Dec 22 09:03:17 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec 22 09:03:21 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <3FE65AF6.6010403@v.loewis.de>
References: <20031217233126.GA4692@unpythonic.net>
	<m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
	<3FE65AF6.6010403@v.loewis.de>
Message-ID: <20031222140255.GH12586@unpythonic.net>

On Mon, Dec 22, 2003 at 03:46:14AM +0100, Martin v. Loewis wrote:
> I'd think so, yes. The patch needs to be elaborated, and I'd delay
> integration to wait for a Redhat response.

Well, unlike the last two bugs I filed in redhat bugzilla, this one has
already seen some activity.  I hope they don't take the line that this
is not a bug :(

------- Additional Comments From jakub@redhat.com  2003-12-22 03:53 -------
Why?
There is no word about system() in
http://www.opengroup.org/onlinepubs/007904975/functions/pthread_atfork.html
and there doesn't seem to be anything in
http://www.opengroup.org/onlinepubs/007904975/functions/system.html
that would mandate system() to be implemented using fork() function.

------- Additional Comments From jepler@unpythonic.net  2003-12-22 08:01 -------
I base my claim that pthread_atfork() should be called by system()
from reading
http://www.opengroup.org/onlinepubs/007904975/functions/system.html

The third paragraph under the heading "description" says this:
"[CX] The environment of the executed command shall be as if a child
process were created using fork(), and the child process invoked the
sh utility using execl() as follows [...]"

The rationale section expands on what the "environment of the executed
command" is:
"IEEE Std 1003.1-2001 places additional restrictions on system(). It
requires that if there is a command language interpreter, the
environment must be as specified by fork() and exec. This ensures, for
example, that close-on- exec works, that file locks are not inherited,
and that the process ID is different."

The pthread_atfork web page says it
"provides multi-threaded application programs with a standard
mechanism for protecting themselves from fork() calls in a library
routine or the application itself."
I believe that system() is one such library routine, since it is
specified to create an environment "as specified by fork()".

This issue arose in the context of Python.  For various reasons,
signals are blocked in all threads besides the main thread.  This
means that in processes generated by fork() from subthreads, all
signals are blocked, which leads to undesired behavior.  Using the
child argument of pthread_atfork() allows signals to be unblocked when
using fork()+exec() to execute an external program, but not when
system() is used.


From barry at python.org  Mon Dec 22 09:55:16 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 22 09:55:42 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312220003.08810.troy@gci.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net>
Message-ID: <1072104915.18828.47.camel@anthem>

On Mon, 2003-12-22 at 04:03, Troy Melhase wrote:

> I don't really like the dot/triple dot notation... the leading punctuation 
> smells perl-ish.  :)

I don't either, although my biggest problem with it is that it isn't
clear just by looking what the punctuation actually means.  I just fear
this will be a permanent head scratcher.

Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative
-syntax game:

import foo by absolute
import foo.bar by relative as baz
import foo by searching as bar
import foo by absolute from bar
import foo as bar by absolute from baz

IOW, 'by' is a reserved word in the import statement context, and it
specifies how to search for modules.  The question then is whether
'absolute', 'relative', and 'searching' are also reserved words or
built-ins and how those names are equated with the module location
algorithms.

-Barry




From guido at python.org  Mon Dec 22 10:24:30 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 10:27:43 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: Your message of "Sun, 21 Dec 2003 21:47:34 PST."
	<20031221212332.10E9.JCARLSON@uci.edu> 
References: <20031221183714.10E6.JCARLSON@uci.edu>
	<200312220445.hBM4jYS17286@c-24-5-183-134.client.comcast.net> 
	<20031221212332.10E9.JCARLSON@uci.edu> 
Message-ID: <200312221527.hBMFRYt17987@c-24-5-183-134.client.comcast.net>

> Is None so weird?  All I am proposing is a symmetric value to None. 
> Just like there is a -1 and 1, -Infinity and Infinity, None logically
> should have Some.  Maybe not so much -Some == None, but as I propose in
> the PEP, possibly Some == not None.

None's purposr is not to be less than all numbers (in fact in
the past it wasn't); it's to be "not any object", like nil or NULL in
other languages.

I don't see a similar broad role for Some.

And I repeat my question: which other language has that concept?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 10:36:09 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 10:36:14 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Mon, 22 Dec 2003 09:55:16 EST."
	<1072104915.18828.47.camel@anthem> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net> 
	<1072104915.18828.47.camel@anthem> 
Message-ID: <200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net>

> I don't either, although my biggest problem with it is that it isn't
> clear just by looking what the punctuation actually means.  I just
> fear this will be a permanent head scratcher.

This argument by itself has very little value.  The question isn't
whether it's self-describing (very few features beyond decimal
integers are); the question is whether it's easy to remember.  There
are two subclasses of that: easy to remember what it means when you
next see it, and easy to remember how to spell it when you next need
it.  IMO the dot syntax does fine here.

> Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative
> -syntax game:

There have been so many alternatives proposed this morning that I have
to spend some time comparing them all...

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Mon Dec 22 10:44:46 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 22 10:44:55 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net> <1072104915.18828.47.camel@anthem>
	<200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net>
Message-ID: <1072107885.18828.76.camel@anthem>

On Mon, 2003-12-22 at 10:36, Guido van Rossum wrote:
> > I don't either, although my biggest problem with it is that it isn't
> > clear just by looking what the punctuation actually means.  I just
> > fear this will be a permanent head scratcher.
> 
> This argument by itself has very little value.  The question isn't
> whether it's self-describing (very few features beyond decimal
> integers are); the question is whether it's easy to remember.  There
> are two subclasses of that: easy to remember what it means when you
> next see it, and easy to remember how to spell it when you next need
> it.  IMO the dot syntax does fine here.

Sorry, it was a pre-coffee argument.  To me, the dot-syntax doesn't
really tell me whether it forces a relative search or an absolute
search.  The dot tells me that something different is happening, so
maybe when absolute is the default, it tells me there's a switch to
relative searching going on.  I'm not sure though, so I do think it may
potentially fail the "easy to remember what it means when you next see
it" test.  OTOH, it would be nice to see a Greg Wilson-style newbie
poll.

> > Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative
> > -syntax game:
> 
> There have been so many alternatives proposed this morning that I have
> to spend some time comparing them all...

Hopefully, this stuff will make it into the PEP so you don't have to
troll through suggestions buried in endless email threads.

-Barry



From guido at python.org  Mon Dec 22 11:29:31 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 11:29:35 2003
Subject: [Python-Dev] OpenBSD anyone?
Message-ID: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>

John Draper <crunch@shopip.com> (the famous Capt'n Crunch) reports
that on his OpenBSD box, he can't build a debug version of Python
2.3(.3c1, but I doubt that matters) unless he tweaks the configure
file to add -O3 to the C flags.  That sounds fishy to me, but he
insists that's how it is, and he's reproduced it from a clean tarball.
Debugging it further doesn't really seem to be within John's power;
the -O3 trick was found by his partner Steve who's temporarily
unavailable.

The symptom is that without -O3, a "python" binary gets built but its
use to run setup.py segfaults.  Is there anyone here with access to an
OpenBSD box who could try to reproduce this?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Mon Dec 22 11:46:01 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 22 11:46:04 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: <200312221527.hBMFRYt17987@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEFNHPAB.tim.one@comcast.net>

[Guido]
> ...
> And I repeat my question: which other language has that concept?

Variations on None were debated pretty exhaustively during Python's first
year (yes, that was a while ago <wink>).

Some obscure functional languages have "top" and "bottom" values.  There's a
partial ordering among values, and "top" and "bottom" are added  to make a
lattice out of that.  "bottom" is usually identified as "the error value",
and f(..., bottom, ...) is defined to return bottom for all functions f
then.

That's not really like None, it's more like 754's NaN (although NaN**0.0 in
C99 is defined to return 1, not NaN, under the theory that it doesn't matter
if an error occurred during the computation of the base, because base**0 is
1 no matter what base should have been).

Aaron Watters also identified several distinct detailed semantics for
"missing value" markers in assorted database languages, which are closer to
what Python's None usually intends ("I'm not really here, but *something*
has to be here so I'm it").

If all that's really wanted is an object that compares larger than anything
else, it's easy to define such a beast (and I see Michael Chermside posted
code for that already).  If more than that is wanted, then it becomes more a
collection of arbitrary behaviors, and then different apps will want
different arbitrary behaviors.  None is handy and I like it, but it's
arbitrary enough in some cases that it takes time to get used to its quirks
(e.g., why is a value that's "not really there" considered to be false?  why
not?  sometimes that's what you want; sometimes you'd rather get an
exception; None picks an arbitrary answer and sticks to it), so I don't want
see more things like it proliferate (one is enough -- but not too many
<wink>).


From skip at pobox.com  Mon Dec 22 11:50:24 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 22 11:50:52 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net>
References: <16356.62036.22726.310077@montanaro.dyndns.org>
	<LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net>
Message-ID: <16359.8400.767266.692477@montanaro.dyndns.org>

    >> Sounds like a plan.  I modify my local source and see if it affects
    >> anything.

    Tim> It should work fine.

Done.  UCHAR_MAX is now required in Python.h, and it's required to be 255.
That may open up a couple (small) optimization opportunities (places where
gcc 3.3 says explicit tests against char values will always be true or
false).  I'll leave it for others to make those tweaks if they so desire.
Note that some of those gcc messages are probably related to larger types
(e.g. unsigned short) or depend on the presence or absence of other macro
definitions (e.g. Py_UNICODE_WIDE).  Should you wish to code any of these
speedups, make sure you get the cpp tests right.  Guido won't forgive you if
you don't. <wink>

Skip

From blunck at gst.com  Mon Dec 22 12:02:57 2003
From: blunck at gst.com (Christopher Blunck)
Date: Mon Dec 22 12:03:01 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
Message-ID: <20031222170257.GA31352@homer.gst.com>

On Mon, Dec 22, 2003 at 08:29:31AM -0800, Guido van Rossum wrote:
> John Draper <crunch@shopip.com> (the famous Capt'n Crunch) reports
> that on his OpenBSD box, he can't build a debug version of Python
> 2.3(.3c1, but I doubt that matters) unless he tweaks the configure
> file to add -O3 to the C flags.  That sounds fishy to me, but he
> insists that's how it is, and he's reproduced it from a clean tarball.
> Debugging it further doesn't really seem to be within John's power;
> the -O3 trick was found by his partner Steve who's temporarily
> unavailable.
> 
> The symptom is that without -O3, a "python" binary gets built but its
> use to run setup.py segfaults.  Is there anyone here with access to an
> OpenBSD box who could try to reproduce this?

OpenBSD 3.3 builds both HEAD and r233c1 fine for me.  This is code out of the
anoncvs repository btw.


-c

-- 
 12:00:00  up 21 days,  1:44, 12 users,  load average: 0.01, 0.07, 0.09

From aahz at pythoncraft.com  Mon Dec 22 12:07:02 2003
From: aahz at pythoncraft.com (Aahz)
Date: Mon Dec 22 12:07:06 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <1072107885.18828.76.camel@anthem>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net>
	<1072104915.18828.47.camel@anthem>
	<200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net>
	<1072107885.18828.76.camel@anthem>
Message-ID: <20031222170702.GA11581@panix.com>

On Mon, Dec 22, 2003, Barry Warsaw wrote:
> On Mon, 2003-12-22 at 10:36, Guido van Rossum wrote:
>>> 
>>> Here's my entry in the sure-to-be-shot-down-by-the-BDFL-alternative
>>> -syntax game:
>> 
>> There have been so many alternatives proposed this morning that I have
>> to spend some time comparing them all...
> 
> Hopefully, this stuff will make it into the PEP so you don't have to
> troll through suggestions buried in endless email threads.

That's the plan; I expect that people will review the PEP to make sure
their pet proposal is listed.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From aleaxit at yahoo.com  Mon Dec 22 12:07:14 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Mon Dec 22 12:07:20 2003
Subject: [Python-Dev] Importing packages from command line
In-Reply-To: <3FE6D94A.2060904@hlabs.spb.ru>
References: <1071839460.3fe2f8e4dbbad@mcherm.com>
	<3FE6D94A.2060904@hlabs.spb.ru>
Message-ID: <200312221807.14352.aleaxit@yahoo.com>

On Monday 22 December 2003 12:45 pm, Dmitry Vasiliev wrote:
   ...
> >>1. python -p package_name
> >>
> >>     Equivalent to:
> >>
> >>     import package_name
   ...
> The main idea is to treating package as a program and run package
> initialization code from command line. The advantage is zipping all
> program modules in one zip archive and running the program from command
> line without unzipping it, like Java's jar's. But this idea need more
> thoughts however...

Couldn't you use:
    python -c "import package_name"
for this purpose even today?


Alex


From tjreedy at udel.edu  Mon Dec 22 12:20:19 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon Dec 22 12:20:25 2003
Subject: [Python-Dev] Re: Got None. Maybe Some?
References: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net><20031222075219.GA3426@performancedrivers.com>
	<20031222000819.10EC.JCARLSON@uci.edu>
Message-ID: <bs794g$j6r$1@sea.gmane.org>


"Josiah Carlson" <jcarlson@uci.edu> wrote in
message
news:20031222000819.10EC.JCARLSON@uci.edu...
> Is None so weird?  All I am proposing is a
symmetric value to None.

To me that would be All of AllType.  Some is in
between.  Can you think of any uses for All other
than being max element?

TJR




From perky at i18n.org  Mon Dec 22 12:26:25 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Mon Dec 22 12:26:33 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <16359.8400.767266.692477@montanaro.dyndns.org>
References: <16356.62036.22726.310077@montanaro.dyndns.org>
	<LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net>
	<16359.8400.767266.692477@montanaro.dyndns.org>
Message-ID: <20031222172625.GA9189@i18n.org>

On Mon, Dec 22, 2003 at 10:50:24AM -0600, Skip Montanaro wrote:
>     >> Sounds like a plan.  I modify my local source and see if it affects
>     >> anything.
> 
>     Tim> It should work fine.
> 
> Done.  UCHAR_MAX is now required in Python.h, and it's required to be 255.

Thanks!

> That may open up a couple (small) optimization opportunities (places where
> gcc 3.3 says explicit tests against char values will always be true or
> false).I'll leave it for others to make those tweaks if they so desire.
> Note that some of those gcc messages are probably related to larger types
> (e.g. unsigned short) or depend on the presence or absence of other macro
> definitions (e.g. Py_UNICODE_WIDE).  Should you wish to code any of these
> speedups, make sure you get the cpp tests right.  Guido won't forgive you if
> you don't. <wink>

I see. Hehe :-)

BTW, I wonder whether we have any good way to get C integer types
with explicitly defined size such as u_int16_t or int64_t of POSIX.
I got a report from an OpenBSD/sparc64 user that he want to change
an unsigned int variable to int type which is used for socket.inet_aton.
(Modules/socketmodule.c:2887). But I think int type isn't appropriate
due to fearing its being 64-bits on some 64bit platforms. Also
in_addr_t, POSIX return type of inet_addr(3), is not so portable.
If we have types like u_int32_t, it will be very helpful to cope
with these sort of issues.


Hye-Shik

From tjreedy at udel.edu  Mon Dec 22 12:29:47 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Mon Dec 22 12:29:48 2003
Subject: [Python-Dev] Re: Relative import
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com><200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net><20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net>
Message-ID: <bs79m9$k8c$1@sea.gmane.org>


"Troy Melhase" <troy@gci.net> wrote in message
news:200312220003.08810.troy@gci.net...
> My humble suggestion:
>
> import NAMES
> import NAMES in MODULE [in HOW]
> import NAME, OTHER in FOO in RELATIVE
> import NAME as RENAME in MODULE in ABSOLUTE

I had same idea of reusing 'in', but with
relative, absolute changed to perhaps RelMode,
AbsMode so as to grammatically flow better.  'in
NAME' could then indicate place in package to
begin search.  'in Package' and 'in Path' are
clearer (more explicit) alternatives to relative
and absolute.

Terry J. Reedy




From guido at python.org  Mon Dec 22 12:33:55 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 12:34:01 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: Your message of "Mon, 22 Dec 2003 11:46:01 EST."
	<LNBBLJKPBEHFEDALKOLCGEFNHPAB.tim.one@comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCGEFNHPAB.tim.one@comcast.net> 
Message-ID: <200312221733.hBMHXtZ18242@c-24-5-183-134.client.comcast.net>

> None is handy and I like it, but it's arbitrary enough in some cases
> that it takes time to get used to its quirks (e.g., why is a value
> that's "not really there" considered to be false?  why not?
> sometimes that's what you want; sometimes you'd rather get an
> exception; None picks an arbitrary answer and sticks to it), so I
> don't want see more things like it proliferate (one is enough -- but
> not too many <wink>).

None inherits most of its semantics from C's NULL -- that's where None
being false comes from.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 12:37:29 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 12:37:37 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Mon, 22 Dec 2003 12:02:57 EST."
	<20031222170257.GA31352@homer.gst.com> 
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>  
	<20031222170257.GA31352@homer.gst.com> 
Message-ID: <200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>

> > The symptom is that without -O3, a "python" binary gets built but its
> > use to run setup.py segfaults.  Is there anyone here with access to an
> > OpenBSD box who could try to reproduce this?
> 
> OpenBSD 3.3 builds both HEAD and r233c1 fine for me.

Thanks!  That's what I suspected, but it's always good to have
confirmation. :)

> This is code out of the anoncvs repository btw.

Shouldn't matter.

If anybody *does* see the bogus behavior and feels like debugging it,
I'd appreciate it!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 12:39:00 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 12:39:04 2003
Subject: [Python-Dev] Relative import
In-Reply-To: Your message of "Mon, 22 Dec 2003 12:07:02 EST."
	<20031222170702.GA11581@panix.com> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net>
	<1072104915.18828.47.camel@anthem>
	<200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net>
	<1072107885.18828.76.camel@anthem> 
	<20031222170702.GA11581@panix.com> 
Message-ID: <200312221739.hBMHd0G18293@c-24-5-183-134.client.comcast.net>

> > Hopefully, this stuff will make it into the PEP so you don't have to
> > troll through suggestions buried in endless email threads.
> 
> That's the plan; I expect that people will review the PEP to make sure
> their pet proposal is listed.

Thanks -- I'll stop tracking this thread closely then until I see the
PEP.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Mon Dec 22 12:50:24 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 22 12:50:29 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: <200312221733.hBMHXtZ18242@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCGEGHHPAB.tim.one@comcast.net>

[Guido]
> None inherits most of its semantics from C's NULL -- that's where None
> being false comes from.

I know that's where it came from, but Python adds its own twists.  If it
really wanted to act like C's NULL, then

    print >> None, "oops"

should segfault <wink>.  There's nothing "wrong" about None evaluating to
false in a Boolean context, it's simply one choice that *could* have made --
and better than most.


From tim.one at comcast.net  Mon Dec 22 13:10:36 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 22 13:10:41 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <20031222172625.GA9189@i18n.org>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEGIHPAB.tim.one@comcast.net>

[Hye-Shik Chang]
> BTW, I wonder whether we have any good way to get C integer types
> with explicitly defined size such as u_int16_t or int64_t of POSIX.

We do not, and C doesn't guarantee that any such types exist.  For example,
while the Crays we recently talked about do support the illusion of 8-bit
char, they have no 16-bit or 32-bit integral types (just 8 and 64); that's
fine by the C standard.

C99 defines a large pile of names, for things like "exactly 16 bits *if*
such a thing exists", "smallest integral type holding at least 16 bits (this
must exist)", and "fastest integral type holding at least 16 bits (which
also must exist)".  Since not all C compilers support these names yet, they
can't be used directly in Python.  If one is needed, it's intended to be
added, in pyport.h, following this comment:

/* typedefs for some C9X-defined synonyms for integral types.
 *
 * The names in Python are exactly the same as the C9X names, except
 * with a Py_ prefix.  Until C9X is universally implemented, this is the
 * only way to ensure that Python gets reliable names that don't
 * conflict with names in non-Python code that are playing their own
 * tricks to define the C9X names.
 *
 * NOTE: don't go nuts here!  Python has no use for *most* of the C9X
 * integral synonyms.  Only define the ones we actually need.
 */

> ...
> If we have types like u_int32_t, it will be very helpful to cope
> with these sort of issues.

Not really -- any use of an "exactly N bits" name will make the code
uncompilable on some platform.


From kbk at shore.net  Mon Dec 22 13:57:18 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon Dec 22 13:57:23 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 22 Dec 2003 09:37:29 -0800")
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
Message-ID: <m3d6ag3ckx.fsf@float.localdomain>

Guido van Rossum <guido@python.org> writes:

> If anybody *does* see the bogus behavior and feels like debugging it,
> I'd appreciate it!

I've been running cvs Python but not previously with debug switched on.

I get a core when using OPT=-g.

(Note that I'm running Patch 850977 "Modify setup.py to detect Tcl/Tk on BSD",
though I wouldn't think it would cause this, and I doubt he's using it.)

Investigating.  Hints appreciated!

-- 
KBK

Script started on Mon Dec 22 13:30:26 2003
hydra /home/kbk/PYTHON/python/dist/src$ uname -a
OpenBSD hydra.localdomain 3.3 GENERIC_RAID#0 i386
hydra /home/kbk/PYTHON/python/dist/src$ 
hydra /home/kbk/PYTHON/python/dist/src$ make clean
find . -name '*.o' -exec rm -f {} ';'
find . -name '*.s[ol]' -exec rm -f {} ';'
find . -name '*.py[co]' -exec rm -f {} ';'
hydra /home/kbk/PYTHON/python/dist/src$ make OPT=-g
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Modules/python.o Modules/python.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/acceler.o Parser/acceler.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/grammar1.o Parser/grammar1.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/listnode.o Parser/listnode.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/node.o Parser/node.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/parser.o Parser/parser.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/parsetok.o Parser/parsetok.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/bitset.o Parser/bitset.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/metagrammar.o Parser/metagrammar.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/firstsets.o Parser/firstsets.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/grammar.o Parser/grammar.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/pgen.o Parser/pgen.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/myreadline.o Parser/myreadline.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/tokenizer.o Parser/tokenizer.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/abstract.o Objects/abstract.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/boolobject.o Objects/boolobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/bufferobject.o Objects/bufferobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/cellobject.o Objects/cellobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/classobject.o Objects/classobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/cobject.o Objects/cobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/complexobject.o Objects/complexobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/descrobject.o Objects/descrobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/enumobject.o Objects/enumobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/fileobject.o Objects/fileobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/floatobject.o Objects/floatobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/frameobject.o Objects/frameobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/funcobject.o Objects/funcobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/intobject.o Objects/intobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/iterobject.o Objects/iterobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/listobject.o Objects/listobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/longobject.o Objects/longobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/dictobject.o Objects/dictobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/methodobject.o Objects/methodobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/moduleobject.o Objects/moduleobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/object.o Objects/object.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/obmalloc.o Objects/obmalloc.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/rangeobject.o Objects/rangeobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/setobject.o Objects/setobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/sliceobject.o Objects/sliceobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/stringobject.o Objects/stringobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/structseq.o Objects/structseq.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/tupleobject.o Objects/tupleobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/typeobject.o Objects/typeobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/weakrefobject.o Objects/weakrefobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/unicodeobject.o Objects/unicodeobject.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Objects/unicodectype.o Objects/unicodectype.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/bltinmodule.o Python/bltinmodule.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/exceptions.o Python/exceptions.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/ceval.o Python/ceval.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/mysnprintf.o Python/mysnprintf.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/tokenizer_pgen.o Parser/tokenizer_pgen.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/printgrammar.o Parser/printgrammar.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Parser/pgenmain.o Parser/pgenmain.c
gcc -pthread -g   Parser/acceler.o  Parser/grammar1.o  Parser/listnode.o  Parser/node.o  Parser/parser.o  Parser/parsetok.o  Parser/bitset.o  Parser/metagrammar.o  Parser/firstsets.o  Parser/grammar.o  Parser/pgen.o Objects/obmalloc.o  Python/mysnprintf.o  Parser/tokenizer_pgen.o  Parser/printgrammar.o  Parser/pgenmain.o -lpthread  -lutil -o Parser/pgen
Parser/pgen ./Grammar/Grammar ./Include/graminit.h ./Python/graminit.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/compile.o Python/compile.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/codecs.o Python/codecs.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/errors.o Python/errors.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/frozen.o Python/frozen.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/frozenmain.o Python/frozenmain.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/future.o Python/future.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/getargs.o Python/getargs.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/getcompiler.o Python/getcompiler.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/getcopyright.o Python/getcopyright.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/getmtime.o Python/getmtime.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -DPLATFORM='"openbsd3"' -o Python/getplatform.o ./Python/getplatform.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/getversion.o Python/getversion.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/graminit.o Python/graminit.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/import.o Python/import.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -I. -o Python/importdl.o ./Python/importdl.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/marshal.o Python/marshal.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/modsupport.o Python/modsupport.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/mystrtoul.o Python/mystrtoul.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/pyfpe.o Python/pyfpe.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/pystate.o Python/pystate.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/pythonrun.o Python/pythonrun.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/structmember.o Python/structmember.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/symtable.o Python/symtable.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/sysmodule.o Python/sysmodule.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/traceback.o Python/traceback.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/getopt.o Python/getopt.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/dynload_shlib.o Python/dynload_shlib.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Python/thread.o Python/thread.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Modules/config.o Modules/config.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -DPYTHONPATH='":plat-openbsd3:lib-tk"'  -DPREFIX='"/usr/local"'  -DEXEC_PREFIX='"/usr/local"'  -DVERSION='"2.4"'  -DVPATH='""'  -o Modules/getpath.o ./Modules/getpath.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Modules/main.o Modules/main.c
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -o Modules/gcmodule.o Modules/gcmodule.c
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/threadmodule.c -o Modules/threadmodule.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/signalmodule.c -o Modules/signalmodule.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/posixmodule.c -o Modules/posixmodule.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/errnomodule.c -o Modules/errnomodule.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/_sre.c -o Modules/_sre.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/_codecsmodule.c -o Modules/_codecsmodule.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/zipimport.c -o Modules/zipimport.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/symtablemodule.c -o Modules/symtablemodule.o
gcc -pthread -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE  -c ./Modules/xxsubtype.c -o Modules/xxsubtype.o
if test -f buildno; then  expr `cat buildno` + 1 >buildno1;  mv -f buildno1 buildno;  else echo 1 >buildno; fi
gcc -pthread -c -fno-strict-aliasing -g -I. -I./Include  -DPy_BUILD_CORE -DBUILD=`cat buildno` -o Modules/getbuildinfo.o ./Modules/getbuildinfo.c
rm -f libpython2.4.a
ar cr libpython2.4.a Modules/getbuildinfo.o
ar cr libpython2.4.a Parser/acceler.o  Parser/grammar1.o  Parser/listnode.o  Parser/node.o  Parser/parser.o  Parser/parsetok.o  Parser/bitset.o  Parser/metagrammar.o  Parser/firstsets.o  Parser/grammar.o  Parser/pgen.o Parser/myreadline.o Parser/tokenizer.o
ar cr libpython2.4.a Objects/abstract.o  Objects/boolobject.o  Objects/bufferobject.o  Objects/cellobject.o  Objects/classobject.o  Objects/cobject.o  Objects/complexobject.o  Objects/descrobject.o  Objects/enumobject.o  Objects/fileobject.o  Objects/floatobject.o  Objects/frameobject.o  Objects/funcobject.o  Objects/intobject.o  Objects/iterobject.o  Objects/listobject.o  Objects/longobject.o  Objects/dictobject.o  Objects/methodobject.o  Objects/moduleobject.o  Objects/object.o  Objects/obmalloc.o  Objects/rangeobject.o  Objects/setobject.o  Objects/sliceobject.o  Objects/stringobject.o  Objects/structseq.o  Objects/tupleobject.o  Objects/typeobject.o  Objects/weakrefobject.o  Objects/unicodeobject.o Objects/unicodectype.o
ar cr libpython2.4.a Python/bltinmodule.o  Python/exceptions.o  Python/ceval.o  Python/compile.o  Python/codecs.o  Python/errors.o  Python/frozen.o  Python/frozenmain.o  Python/future.o  Python/getargs.o  Python/getcompiler.o  Python/getcopyright.o  Python/getmtime.o  Python/getplatform.o  Python/getversion.o  Python/graminit.o  Python/import.o  Python/importdl.o  Python/marshal.o  Python/modsupport.o  Python/mystrtoul.o  Python/mysnprintf.o  Python/pyfpe.o  Python/pystate.o  Python/pythonrun.o  Python/structmember.o  Python/symtable.o  Python/sysmodule.o  Python/traceback.o  Python/getopt.o  Python/dynload_shlib.o    Python/thread.o
ar cr libpython2.4.a Modules/config.o  Modules/getpath.o  Modules/main.o  Modules/gcmodule.o 
ar cr libpython2.4.a Modules/threadmodule.o  Modules/signalmodule.o  Modules/posixmodule.o  Modules/errnomodule.o  Modules/_sre.o  Modules/_codecsmodule.o  Modules/zipimport.o  Modules/symtablemodule.o  Modules/xxsubtype.o
ranlib libpython2.4.a
gcc -pthread   -o python  Modules/python.o  libpython2.4.a -lpthread  -lutil   -lm  
./Modules/posixmodule.c:5825: warning: tempnam() possibly used unsafely; consider using mkstemp()
./Modules/posixmodule.c:5872: warning: tmpnam() possibly used unsafely; consider using mkstemp()
case $MAKEFLAGS in  *-s*)  CC='gcc -pthread' LDSHARED='ld -Bshareable' OPT='-g' ./python -E ./setup.py -q build;;  *)  CC='gcc -pthread' LDSHARED='ld -Bshareable' OPT='-g' ./python -E ./setup.py build;;  esac
Memory fault (core dumped) 
*** Error code 139

Stop in /home/kbk/PYTHON/python/dist/src (line 335 of Makefile).
hydra /home/kbk/PYTHON/python/dist/src$ 
hydra /home/kbk/PYTHON/python/dist/src$ 
hydra /home/kbk/PYTHON/python/dist/src$ cvs st configure.in
===================================================================
File: configure.in      Status: Up-to-date

   Working revision:    1.442
   Repository revision: 1.442   /cvsroot/python/python/dist/src/configure.in,v
   Sticky Tag:          (none)
   Sticky Date:         (none)
   Sticky Options:      (none)

hydra /home/kbk/PYTHON/python/dist/src$ 
Script done on Mon Dec 22 13:31:57 2003
Script started on Mon Dec 22 13:40:31 2003
hydra /home/kbk/PYTHON/python/dist/src$ 
hydra /home/kbk/PYTHON/python/dist/src$ 
hydra /home/kbk/PYTHON/python/dist/src$ # [re-build with make clean && make]
hydra /home/kbk/PYTHON/python/dist/src$ ./python
Python 2.4a0 (#35, Dec 22 2003, 13:38:03) 
[GCC 2.95.3 20010125 (prerelease, propolice)] on openbsd3
Type "help", "copyright", "credits" or "license" for more information.
>>> 
hydra /home/kbk/PYTHON/python/dist/src$ 
Script done on Mon Dec 22 13:41:26 2003

From devin at whitebread.org  Mon Dec 22 13:36:45 2003
From: devin at whitebread.org (Devin)
Date: Mon Dec 22 14:11:48 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312221739.hBMHd0G18293@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312220945540.12124-100000@whitebread.org>

I've been monitoring all of this discussion about relative imports, and 
have my own humble idea about how to spell relative imports.

The first part stems from Guido's idea about using a '.' at the beginning 
of a relative import.  It seemed a bit weird at first, but I eventually 
became very comfortable with the idea after thinking about it in respect 
to absolute imports.  I'll explain what I mean.

If you currently want to import a module 'a.b.c.d', you write:

    import a.b.c.d

Now, if you're in the module 'a.b', you could still write:

    import a.b.c.d

... but the 'a.b' part is redundant because you're already in the module 
'a.b'.  If you take away 'a.b', then you end up with:

    import .c.d

This flowed well in my head, and looks natural.  However, the suggestion
about '.' or '..' referring to the parent looks _ugly and unnatural_.

Let's say '.' was used to spell "parent module", and let's assume that 
the module 'a.b.c.d' wants to import 'a.b.e'.  The absolute import would 
be spelled:

    import a.b.e

... and the relative import would be spelled:

    import .....e # .[parent].[parent].e

Yuck! (no offense :)

What's needed, I believe, is an identifier/keyword/symbol that spells
'parent' that isn't '.'.  Assuming that symbol is represented by
'[parent]', the above relative import would look like:

    import .[parent].[parent].e

... which is a hell lot more clear IMHO.

My initial idea is to spell parent '__parent__', and to have '__parent__'
be a module-level variable in each module that refers to the parent module
(where a top-level module might have '__parent__' refer to itself, or to
None).  Continuing with the above example, the relative import would be
spelled:

    import .__parent__.__parent__.e

... but I'm not completely convinced that there's not a better 
identifier/keyword/symbol that could be substituted for '[parent]'.

If I'm completely off track, then tell me and I'll continue to lurk until
I get a better feel for the happenings of python-dev.

-- 
Devin
devin@whitebread.org
http://www.whitebread.org/


From blunck at gst.com  Mon Dec 22 14:26:53 2003
From: blunck at gst.com (Christopher Blunck)
Date: Mon Dec 22 14:26:56 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <m3d6ag3ckx.fsf@float.localdomain>
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
Message-ID: <20031222192653.GA31766@homer.gst.com>

On Mon, Dec 22, 2003 at 01:57:18PM -0500, Kurt B. Kaiser wrote:
> Guido van Rossum <guido@python.org> writes:
> 
> > If anybody *does* see the bogus behavior and feels like debugging it,
> > I'd appreciate it!
> 
> I've been running cvs Python but not previously with debug switched on.
> 
> I get a core when using OPT=-g.
> 
> (Note that I'm running Patch 850977 "Modify setup.py to detect Tcl/Tk on BSD",
> though I wouldn't think it would cause this, and I doubt he's using it.)
> 
> Investigating.  Hints appreciated!

Kurt-

Interesting...  When I do OPT=-g, I get a Memory fault (core dumped) 
when linking python.  Is that when you receive the core dump?


-c

-- 
 14:10:00  up 21 days,  3:54, 12 users,  load average: 0.00, 0.02, 0.00

From theller at python.net  Mon Dec 22 14:28:24 2003
From: theller at python.net (Thomas Heller)
Date: Mon Dec 22 14:28:27 2003
Subject: [Python-Dev] Improve Python for embedding
Message-ID: <1xqwodnr.fsf@python.net>

I find the current situation when 'embedding' Python rather unsatisfying.

1. Careful preparation (hacking?) of environment variables is needed to
avoid that Py_Initialize() takes the default actions needed for the
standard Python interpreter (for example, to avoid the default sys.path).

2. A suggestion from /F on c.l.p was to change pythonw.exe so that
instead of hiding all output to stderr a console window would be opened
to show the tracebacks.  This requires to construct an object with a
'write' method doing all this magic, and it must be done after the call
to Py_Initialize() and before PyRun_SimpleFile().  Now, looking at the
code of Py_Main(), it seems impossible without rewriting most of the
code.

Possible improvements I currently can think of are:

- Provide a function like Py_InitializeEx(), which accepts parameters
specifying an initial sys.path, optimize flag, verbose flag, and so on.

- Split Py_Main() into 2 functions Py_Main_Prepare() and Py_Main_Run(),
or let Py_Main() accept a callback function which will be called jsut
after Py_Initialize() has been run.

I'm not ready to write a PEP, but if there are people sharing this
feeling I would suggest we create a wiki entry somewhere acting as a
temporary brainstorming area.

Thomas


From jcarlson at uci.edu  Mon Dec 22 14:25:32 2003
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Dec 22 14:29:05 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: <1072100025.3fe6f2b90ea70@mcherm.com>
References: <1072100025.3fe6f2b90ea70@mcherm.com>
Message-ID: <20031222111017.0A6D.JCARLSON@uci.edu>

>   [Josiah proposes a Some object to be the counterpart of None]
> 
>   [Guido seems reluctant]
> 
> Josiah: I have a counterproposal. The good news is, it's REALLY easy -- you
> won't need a PEP, and I am sure that you'll get no objections from anyone
> on this list. And it'll provide you with a "Some" object that acts like
> you describe (except the one mention you made of "Some == not None"...
> that one's a bad idea since right now "True == not None").

 -- snip --

Yes, renaming the class and being more explicit about the comparisons 
(and tossing the hash function, which turns out to be unneeded) does
pretty much what I want it to.

But then again, I've had Some since I thought of it.  I'm all about this
free software idea, and I know that it could be useful to others.  I
suppose I'll just post information about Some and a sample
implementation (combination of yours and mine) on my website or somesuch.

 - Josiah


From guido at python.org  Mon Dec 22 14:29:40 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 14:29:45 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Mon, 22 Dec 2003 14:26:53 EST."
	<20031222192653.GA31766@homer.gst.com> 
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain> 
	<20031222192653.GA31766@homer.gst.com> 
Message-ID: <200312221929.hBMJTef18533@c-24-5-183-134.client.comcast.net>

> Interesting...  When I do OPT=-g, I get a Memory fault (core dumped) 
> when linking python.  Is that when you receive the core dump?

>From looking at his output, he gets it just after the linking, when
using the freshly linked python executable to run the setup.py script.

Are you saying you didn't try the debug build?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 14:33:29 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 14:33:35 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: Your message of "Mon, 22 Dec 2003 20:28:24 +0100."
	<1xqwodnr.fsf@python.net> 
References: <1xqwodnr.fsf@python.net> 
Message-ID: <200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net>

> I find the current situation when 'embedding' Python rather unsatisfying.

I'm interested in improve it.  Can you explain what exactly your
'embedding' requirements are?

> 1. Careful preparation (hacking?) of environment variables is needed to
> avoid that Py_Initialize() takes the default actions needed for the
> standard Python interpreter (for example, to avoid the default sys.path).

I guess it depends on the embedding needs.  Some embedded uses would
be quite happy with the default sys.path.

> 2. A suggestion from /F on c.l.p was to change pythonw.exe so that
> instead of hiding all output to stderr a console window would be opened
> to show the tracebacks.  This requires to construct an object with a
> 'write' method doing all this magic, and it must be done after the call
> to Py_Initialize() and before PyRun_SimpleFile().  Now, looking at the
> code of Py_Main(), it seems impossible without rewriting most of the
> code.

I'm not sure what this has to do with embedding -- can you clarify?

> Possible improvements I currently can think of are:
> 
> - Provide a function like Py_InitializeEx(), which accepts parameters
> specifying an initial sys.path, optimize flag, verbose flag, and so on.

Except for sys.path, you can set most flags directly before calling
Py_Initialize(), so I'm not sure if there's a need to pass these into
Py_Initialize().

> - Split Py_Main() into 2 functions Py_Main_Prepare() and Py_Main_Run(),
> or let Py_Main() accept a callback function which will be called jsut
> after Py_Initialize() has been run.

OK, now I'm confused.  If you're embedding Python, why would you be
using Py_Main() at all?

> I'm not ready to write a PEP, but if there are people sharing this
> feeling I would suggest we create a wiki entry somewhere acting as a
> temporary brainstorming area.

Go ahead, but count me out of the wiki -- wikis don't work for
discussions for me.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 14:35:42 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 14:35:48 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: Your message of "Mon, 22 Dec 2003 11:25:32 PST."
	<20031222111017.0A6D.JCARLSON@uci.edu> 
References: <1072100025.3fe6f2b90ea70@mcherm.com>  
	<20031222111017.0A6D.JCARLSON@uci.edu> 
Message-ID: <200312221935.hBMJZgD18583@c-24-5-183-134.client.comcast.net>

> But then again, I've had Some since I thought of it.  I'm all about this
> free software idea, and I know that it could be useful to others.  I
> suppose I'll just post information about Some and a sample
> implementation (combination of yours and mine) on my website or somesuch.

Just please, Please, PLEASE come up with a better name.  Some is just
too mysterious.  Remember that it is *not* in most senses the opposite
of None.  And even if it was, Some would be a poor choice of name.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Mon Dec 22 14:41:26 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 14:41:30 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net> (Tim
	Peters's message of "Sat, 20 Dec 2003 21:52:03 -0500")
References: <LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net>
Message-ID: <2mptegej2x.fsf@starship.python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> [Skip Montanaro]
>> Isn't that supposed to always be defined in limits.h or is UCHAR_MAX
>> not a standard macro?
>
> Yes, it's supposed to be there.  OTOH, so is limits.h, i.e. the
> HAVE_LIMITS_H test shouldn't be necessary either.

As Martin regularly points out, we have rather too much of this kind
of thing.  Given that we demand ANSI C, we could surely lose
HAVE_LIMITS_H, and probably much other cruft.

People who have to contend with broken platforms might have a
different view.  I guess there's no real reason to churn, but when
crap like this gets in the way we should strive to kill it.

Cheers,
mwh

-- 
  Any form of evilness that can be detected without *too* much effort
  is worth it...  I have no idea what kind of evil we're looking for
  here or how to detect is, so I can't answer yes or no.
                                       -- Guido Van Rossum, python-dev

From tree at basistech.com  Mon Dec 22 14:40:12 2003
From: tree at basistech.com (Tom Emerson)
Date: Mon Dec 22 14:42:28 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <1xqwodnr.fsf@python.net>
References: <1xqwodnr.fsf@python.net>
Message-ID: <16359.18588.450934.263664@magrathea.basistech.com>

Thomas Heller writes:
> I find the current situation when 'embedding' Python rather unsatisfying.

Indeed. See patch 849278 for some of the work we did to deal with this.

> 1. Careful preparation (hacking?) of environment variables is needed to
> avoid that Py_Initialize() takes the default actions needed for the
> standard Python interpreter (for example, to avoid the default sys.path).

I've done this in the patches submitted (by a coworker) above. The
idea is to separate the embedded interpreter *completely* from the
environment.
 
> 2. A suggestion from /F on c.l.p was to change pythonw.exe so that
> instead of hiding all output to stderr a console window would be opened
> to show the tracebacks.  This requires to construct an object with a
> 'write' method doing all this magic, and it must be done after the call
> to Py_Initialize() and before PyRun_SimpleFile().  Now, looking at the
> code of Py_Main(), it seems impossible without rewriting most of the
> code.

Yes, this is a problem with the embedded interpreter where errors may
be written to stderr without regard to the calling
application. Further, it is possible for the interpreter to call
abort(), which is even worse.

> Possible improvements I currently can think of are:
> 
> - Provide a function like Py_InitializeEx(), which accepts parameters
> specifying an initial sys.path, optimize flag, verbose flag, and so on.

See the patch.

> - Split Py_Main() into 2 functions Py_Main_Prepare() and Py_Main_Run(),
> or let Py_Main() accept a callback function which will be called jsut
> after Py_Initialize() has been run.

Yup.

> I'm not ready to write a PEP, but if there are people sharing this
> feeling I would suggest we create a wiki entry somewhere acting as a
> temporary brainstorming area.

Absolutely: count me in.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Software Architect                                 http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"

From kbk at shore.net  Mon Dec 22 14:45:43 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon Dec 22 14:45:49 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312221907.hBMJ7hg18485@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 22 Dec 2003 11:07:43 -0800")
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<200312221907.hBMJ7hg18485@c-24-5-183-134.client.comcast.net>
Message-ID: <m3vfo81vrs.fsf@float.localdomain>

Guido van Rossum <guido@python.org> writes:

> Very interesting!  Had you done a previous non-debug build in the same
> directory?  (Even if you did a "make clobber" before restarting -- you
> never know what leave-behind could cause this.)

Yes.  I did do a make clobber && ./configure before building the debug
version, but it's not a clean checkout.

OK, just did a clean checkout, no patch, and got the same result.

> The next thing I'd try would be to start the python executable that
>was build under gdb and play around in it, like this:
>
> $ gdb ./python
> [...]
> (gdb) run
> Starting program: /home/guido/projects/python/dist/src/debug/python
> [New Thread 1074895104 (LWP 30404)]
> Python 2.4a0 (#2, Dec 22 2003, 11:02:19)
> [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
> Type "help", "copyright", "credits" or "license" for more information.
>>>> print 2+2
>>>> # try various things that don't need external modules except sys
>>>> # define a function, etc...
>>>> # if that doesn't segfault, try:
>>>> from test import autotest
> [...]
>
> Eventually I expect you'd get a segfault; at that point you can use
> the gdb 'bt' command to get a stack trace.  Hopefully it'll point to
> some innocent C code that gets mistreated by the non-debug compiler...

It doesn't get as far as the banner:

Script started on Mon Dec 22 14:42:19 2003
hydra /home/kbk/proj/sandbox/python_clean$ gdb ./python
GNU gdb 4.16.1
Copyright 1996 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-unknown-openbsd3.3"...
(gdb) run
Starting program: /home/kbk/proj/sandbox/python_clean/./python 

Program received signal SIGSEGV, Segmentation fault.
0x401900a0 in strchr ()
(gdb) quit
The program is running.  Quit anyway (and kill it)? (y or n) y
hydra /home/kbk/proj/sandbox/python_clean$ 
Script done on Mon Dec 22 14:42:38 2003

Investigating.

-- 
KBK

From jcarlson at uci.edu  Mon Dec 22 14:45:19 2003
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Dec 22 14:47:58 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: <200312221524.hBMFOVj17969@c-24-5-183-134.client.comcast.net>
References: <20031221212332.10E9.JCARLSON@uci.edu>
	<200312221524.hBMFOVj17969@c-24-5-183-134.client.comcast.net>
Message-ID: <20031222114456.0A73.JCARLSON@uci.edu>

> > Is None so weird?  All I am proposing is a symmetric value to None. 
> > Just like there is a -1 and 1, -Infinity and Infinity, None logically
> > should have Some.  Maybe not so much -Some == None, but as I propose in
> > the PEP, possibly Some == not None.
> 
> None's purposr is not to be less than all numbers (in fact in
> the past it wasn't); it's to be "not any object", like nil or NULL in
> other languages.
> 
> I don't see a similar broad role for Some.
> 
> And I repeat my question: which other language has that concept?
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

Certainly if one thinks of None as an object, or a possible pointer to
an object, None does fit the role of nil or NULL.  The fact is, NULL and
nil, as pointers, are 0.  None does not equal 0, it is a symbolic
representation of the lack of any value (something I remember you saying
before, but cannot find a quote), who'se original purpose was to be an
equivalent to nil or NULL.  As of right now, None has gone beyone its
original purpose.  As soon as comparisons were able to be done to it and
reveal that None was smaller than anything else (even the floating point
-infinity), it seemingly has become a more consistant minimum value.

Some is just begging to be a more consistant maximum value.  It is
bigger than sys.maxint, float(1e309), and reveals that the current state
is special.

What other language has the concept?  Python already has the concept.
None is currently used as negative infinity in some projects, an actual
negative infinity, one that doesn't crap out on integers that cannot be
cast into floats.  Some would become positive infinity.

In terms of non-Python languages, I don't know of any, but that doesn't
mean that they don't exist - it just means that I have limited knowledge
of programming languages (I only know about 7 or so).

 - Josiah


From mwh at python.net  Mon Dec 22 14:48:29 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 14:48:33 2003
Subject: [Python-Dev] Py_DECREF causes spurious gcc warning
In-Reply-To: <LNBBLJKPBEHFEDALKOLCAENKHOAB.tim.one@comcast.net> (Tim
	Peters's message of "Sat, 20 Dec 2003 17:13:34 -0500")
References: <LNBBLJKPBEHFEDALKOLCAENKHOAB.tim.one@comcast.net>
Message-ID: <2mllp4eir6.fsf@starship.python.net>

"Tim Peters" <tim.one@comcast.net> writes:

['>>' is Zack]

>> I am not disputing that MSVC does this silly thing you describe, but
>> MSVC has a long track record of bugs
>
> That doesn't match our experience.  I don't believe Python code has ever
> tickled an MSVC codegen or optimization bug, neither any in gcc-for-x86.

Some version of gcc for OpenBSD (on intel) miscompiles
(w/optimization) the usual definition of Py_IS_OVERFLOW (or whatever
that macro's called).  Just nitpicking :-)

>> GCC does not generate junk code even at -O0.

I'm not sure I understand the context of this remark, but out of
context it sure sounds implausible.

Cheers,
mwh

-- 
  The use of COBOL cripples the mind; its teaching should, therefore,
  be regarded as a criminal offence.
           -- Edsger W. Dijkstra, SIGPLAN Notices, Volume 17, Number 5

From kbk at shore.net  Mon Dec 22 14:48:43 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon Dec 22 14:48:51 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <20031222192653.GA31766@homer.gst.com> (Christopher Blunck's
	message of "Mon, 22 Dec 2003 14:26:53 -0500")
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<20031222192653.GA31766@homer.gst.com>
Message-ID: <m3n09k1vms.fsf@float.localdomain>

Christopher Blunck <blunck@gst.com> writes:

> Interesting...  When I do OPT=-g, I get a Memory fault (core dumped) 
> when linking python.  Is that when you receive the core dump?

Sounds like we're seeing the same thing.

-- 
KBK

From guido at python.org  Mon Dec 22 14:54:01 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 14:54:07 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: Your message of "Mon, 22 Dec 2003 14:40:12 EST."
	<16359.18588.450934.263664@magrathea.basistech.com> 
References: <1xqwodnr.fsf@python.net>  
	<16359.18588.450934.263664@magrathea.basistech.com> 
Message-ID: <200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net>

> Indeed. See patch 849278 for some of the work we did to deal with this.

This mostly removes the use of the PYTHONXXX environment variables.
Can you explain why you don't want these to work?

Also note that you can already disable *all* use of the environment
(not just in Py_Initialize()) by setting the global
Py_IgnoreEnvironmentFlag to a non-zero value.  I'd reject that part of
the patch for that reason.

Py_SetSysPaths() might be a valuable addition, but the game you play
with the prefiX name is abject. :)

> > 1. Careful preparation (hacking?) of environment variables is needed to
> > avoid that Py_Initialize() takes the default actions needed for the
> > standard Python interpreter (for example, to avoid the default sys.path).
> 
> I've done this in the patches submitted (by a coworker) above. The
> idea is to separate the embedded interpreter *completely* from the
> environment.
>  
> > 2. A suggestion from /F on c.l.p was to change pythonw.exe so that
> > instead of hiding all output to stderr a console window would be opened
> > to show the tracebacks.  This requires to construct an object with a
> > 'write' method doing all this magic, and it must be done after the call
> > to Py_Initialize() and before PyRun_SimpleFile().  Now, looking at the
> > code of Py_Main(), it seems impossible without rewriting most of the
> > code.
> 
> Yes, this is a problem with the embedded interpreter where errors may
> be written to stderr without regard to the calling
> application. Further, it is possible for the interpreter to call
> abort(), which is even worse.

What do you suggest the interpreter should do instead of calling
abort()?  This is only called in response to *fatal* errors --
situations where raising an exception is *not* an acceptable
alternative for various reasons (maybe because it's in a piece of code
that is part of the implementation of exceptions, or maybe because a
corruption is detected in the very interpreter environment that's
needed to raise an exception).  Anout the only alternative I can think
of would be a longjmp() call to a handler you have set up, but that
can't be safe when the fatal error is discovered in a different
thread.

> > Possible improvements I currently can think of are:
> > 
> > - Provide a function like Py_InitializeEx(), which accepts parameters
> > specifying an initial sys.path, optimize flag, verbose flag, and so on.
> 
> See the patch.
> 
> > - Split Py_Main() into 2 functions Py_Main_Prepare() and Py_Main_Run(),
> > or let Py_Main() accept a callback function which will be called just
> > after Py_Initialize() has been run.
> 
> Yup.

What's your embedding environment?  Why do you want to use Py_Main()
at all?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at python.net  Mon Dec 22 14:55:38 2003
From: theller at python.net (Thomas Heller)
Date: Mon Dec 22 14:55:43 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 22 Dec 2003 11:33:29 -0800")
References: <1xqwodnr.fsf@python.net>
	<200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net>
Message-ID: <u13smxtx.fsf@python.net>


[Thomas Heller]
>> I find the current situation when 'embedding' Python rather unsatisfying.
>

[Guido van Rossum]
> I'm interested in improve it.  Can you explain what exactly your
> 'embedding' requirements are?

see below.

>> 1. Careful preparation (hacking?) of environment variables is needed to
>> avoid that Py_Initialize() takes the default actions needed for the
>> standard Python interpreter (for example, to avoid the default sys.path).
>
> I guess it depends on the embedding needs.  Some embedded uses would
> be quite happy with the default sys.path.

Sure, but others not.

>> Possible improvements I currently can think of are:
>> 
>> - Provide a function like Py_InitializeEx(), which accepts parameters
>> specifying an initial sys.path, optimize flag, verbose flag, and so on.
>
> Except for sys.path, you can set most flags directly before calling
> Py_Initialize(), so I'm not sure if there's a need to pass these into
> Py_Initialize().

I want to specify the Python path before calling Py_Initialize(), and I
don't want it to use *any* environment variables which (again) set the
flags.  So I thought of Py_InitializeEx() which would take parameters
specifying these things, and Py_Initialize() could be changed to call
Py_InitializeEx(NULL), maybe.

>> 2. A suggestion from /F on c.l.p was to change pythonw.exe so that
>> instead of hiding all output to stderr a console window would be opened
>> to show the tracebacks.  This requires to construct an object with a
>> 'write' method doing all this magic, and it must be done after the call
>> to Py_Initialize() and before PyRun_SimpleFile().  Now, looking at the
>> code of Py_Main(), it seems impossible without rewriting most of the
>> code.
>
> I'm not sure what this has to do with embedding -- can you clarify?

The Python interpreter (dll) embedded into pythonw.exe ;-)

>
>> - Split Py_Main() into 2 functions Py_Main_Prepare() and Py_Main_Run(),
>> or let Py_Main() accept a callback function which will be called jsut
>> after Py_Initialize() has been run.
>
> OK, now I'm confused.  If you're embedding Python, why would you be
> using Py_Main() at all?

I'm probably not good at explaining these things.

I was thinking of pythonw.exe (WinMain.c).  I want to create a Python
object (an stderr writer), and assign this to sys.stderr, without
changing too much code.

Does it make more sense now?

Thomas


From mwh at python.net  Mon Dec 22 14:56:32 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 14:56:36 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <bs5lbp$ut1$1@sea.gmane.org> (Terry Reedy's message of "Sun, 21
	Dec 2003 21:36:44 -0500")
References: <20031220112120.GE1933@lkcl.net> <20031220121942.GF1933@lkcl.net>
	<bs5lbp$ut1$1@sea.gmane.org>
Message-ID: <2m65g8eidr.fsf@starship.python.net>

"Terry Reedy" <tjreedy@udel.edu> writes:

> "Luke Kenneth Casson Leighton" <lkcl@lkcl.net>
> wrote in message
> news:20031220121942.GF1933@lkcl.net...
>> if you "feed" access to __class__ via a single
> function, you have
>> a bottleneck again which can be replaced.
>>
>> so, the proposal is simple: create an
> __builtin__ function called
>> __get_class__ through which access to __class__
> is made.
>>
>
> I wonder if it would not be better to turn
> __class__ into a property with a get function that
> conditionally gives out the real thing.

As much as it makes sense to say it, __class__ is a property -- really
a getset descriptor, but that's just because it's in C.

It would be a imple matter to add a call to PyEval_GetRestricted in
object_get_class (not sure whether this is wise or helpful -- only
been skimming this thread -- but it certainly ain't hard).

Cheers,
mwh

-- 
  I don't remember any dirty green trousers.
                                             -- Ian Jackson, ucam.chat

From mwh at python.net  Mon Dec 22 14:58:55 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 14:58:59 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <20031222192653.GA31766@homer.gst.com> (Christopher Blunck's
	message of "Mon, 22 Dec 2003 14:26:53 -0500")
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<20031222192653.GA31766@homer.gst.com>
Message-ID: <2m1xqwei9s.fsf@starship.python.net>

Christopher Blunck <blunck@gst.com> writes:

> On Mon, Dec 22, 2003 at 01:57:18PM -0500, Kurt B. Kaiser wrote:
>> Guido van Rossum <guido@python.org> writes:
>> 
>> > If anybody *does* see the bogus behavior and feels like debugging it,
>> > I'd appreciate it!
>> 
>> I've been running cvs Python but not previously with debug switched on.
>> 
>> I get a core when using OPT=-g.
>> 
>> (Note that I'm running Patch 850977 "Modify setup.py to detect Tcl/Tk on BSD",
>> though I wouldn't think it would cause this, and I doubt he's using it.)
>> 
>> Investigating.  Hints appreciated!
>
> Kurt-
>
> Interesting...  When I do OPT=-g, I get a Memory fault (core dumped) 
> when linking python.  Is that when you receive the core dump?

What version of gcc?  Anthony and I hacked around on HP's compile farm
in the 2.3.2 timeframe and saw gcc problems which we hacked around,
but nothing like this.

Cheers,
mwh

-- 
  I've even been known to get Marmite *near* my mouth -- but never
  actually in it yet.  Vegamite is right out.
 UnicodeError: ASCII unpalatable error: vegamite found, ham expected
                                       -- Tim Peters, comp.lang.python

From guido at python.org  Mon Dec 22 14:59:43 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 14:59:48 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Mon, 22 Dec 2003 14:45:43 EST."
	<m3vfo81vrs.fsf@float.localdomain> 
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<200312221907.hBMJ7hg18485@c-24-5-183-134.client.comcast.net> 
	<m3vfo81vrs.fsf@float.localdomain> 
Message-ID: <200312221959.hBMJxhY18697@c-24-5-183-134.client.comcast.net>

> It doesn't get as far as the banner:
> 
> Script started on Mon Dec 22 14:42:19 2003
> hydra /home/kbk/proj/sandbox/python_clean$ gdb ./python
> GNU gdb 4.16.1
> Copyright 1996 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-unknown-openbsd3.3"...
> (gdb) run
> Starting program: /home/kbk/proj/sandbox/python_clean/./python 
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x401900a0 in strchr ()
> (gdb) quit
> The program is running.  Quit anyway (and kill it)? (y or n) y
> hydra /home/kbk/proj/sandbox/python_clean$ 
> Script done on Mon Dec 22 14:42:38 2003
> 
> Investigating.

That suggests it's still in Py_Initialize().  What does the gdb
command 'bt' say???

I'd also try another experiment: instead of "run" try "run -S".  This
passes the -S option to Python when it is started, so that it doesn't
try to load site.py (which executes rather a lot of Python code).  I'd
be interested in seeing how much you can do interactively in that
case, of if it still crashes in Py_Initialize().

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jcarlson at uci.edu  Mon Dec 22 14:59:00 2003
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Dec 22 15:00:56 2003
Subject: [Python-Dev] Re: Got None. Maybe Some?
In-Reply-To: <bs794g$j6r$1@sea.gmane.org>
References: <20031222000819.10EC.JCARLSON@uci.edu> <bs794g$j6r$1@sea.gmane.org>
Message-ID: <20031222114715.0A76.JCARLSON@uci.edu>

> "Josiah Carlson" <jcarlson@uci.edu> wrote in
> message
> news:20031222000819.10EC.JCARLSON@uci.edu...
> > Is None so weird?  All I am proposing is a
> symmetric value to None.
> 
> To me that would be All of AllType.  Some is in
> between.  Can you think of any uses for All other
> than being max element?
> 
> TJR

Not really.  However, I was thinking of Some in the context of graph
and dynamic programming algorithms.  When programming some algorithms in
Python, you need to either pick some large constant (that you hope will
be big enough), or have a test to see if a value is None. That is ugly
and very not Pythonic.

I do like All and AllType though, which gives a better name, as
suggested by Guido.

 - Josiah

From guido at python.org  Mon Dec 22 15:03:40 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 15:03:47 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: Your message of "Mon, 22 Dec 2003 20:55:38 +0100."
	<u13smxtx.fsf@python.net> 
References: <1xqwodnr.fsf@python.net>
	<200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net> 
	<u13smxtx.fsf@python.net> 
Message-ID: <200312222003.hBMK3eH18732@c-24-5-183-134.client.comcast.net>

> [Thomas Heller]
> >> I find the current situation when 'embedding' Python rather unsatisfying.
> >
> 
> [Guido van Rossum]
> > I'm interested in improve it.  Can you explain what exactly your
> > 'embedding' requirements are?
> 
> see below.

The only clue I find below is that you're trying to inmprove the error
reporting of pythonw.exe, which is certainly laudable, but hardly
qualifies as "embedding" IMO.

> >> 1. Careful preparation (hacking?) of environment variables is needed to
> >> avoid that Py_Initialize() takes the default actions needed for the
> >> standard Python interpreter (for example, to avoid the default sys.path).
> >
> > I guess it depends on the embedding needs.  Some embedded uses would
> > be quite happy with the default sys.path.
> 
> Sure, but others not.

But the only use *you* are interested in is pythonw.exe?

> >> Possible improvements I currently can think of are:
> >> 
> >> - Provide a function like Py_InitializeEx(), which accepts parameters
> >> specifying an initial sys.path, optimize flag, verbose flag, and so on.
> >
> > Except for sys.path, you can set most flags directly before calling
> > Py_Initialize(), so I'm not sure if there's a need to pass these into
> > Py_Initialize().
> 
> I want to specify the Python path before calling Py_Initialize(), and I
> don't want it to use *any* environment variables which (again) set the
> flags.  So I thought of Py_InitializeEx() which would take parameters
> specifying these things, and Py_Initialize() could be changed to call
> Py_InitializeEx(NULL), maybe.

If you don't want the environment to be used, set the global
Py_IgnoreEnvironmentFlag before calling Py_Initialize().

> >> 2. A suggestion from /F on c.l.p was to change pythonw.exe so that
> >> instead of hiding all output to stderr a console window would be opened
> >> to show the tracebacks.  This requires to construct an object with a
> >> 'write' method doing all this magic, and it must be done after the call
> >> to Py_Initialize() and before PyRun_SimpleFile().  Now, looking at the
> >> code of Py_Main(), it seems impossible without rewriting most of the
> >> code.
> >
> > I'm not sure what this has to do with embedding -- can you clarify?
> 
> The Python interpreter (dll) embedded into pythonw.exe ;-)
> 
> >
> >> - Split Py_Main() into 2 functions Py_Main_Prepare() and Py_Main_Run(),
> >> or let Py_Main() accept a callback function which will be called jsut
> >> after Py_Initialize() has been run.
> >
> > OK, now I'm confused.  If you're embedding Python, why would you be
> > using Py_Main() at all?
> 
> I'm probably not good at explaining these things.
> 
> I was thinking of pythonw.exe (WinMain.c).  I want to create a Python
> object (an stderr writer), and assign this to sys.stderr, without
> changing too much code.
> 
> Does it make more sense now?

Not quite -- I don't understand why *you* don't want the environment
variables to be used, if all you want is a better pythonw.exe.

If that's your only experience with embedding Python, you can't really
speak for other embedders.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pf_moore at yahoo.co.uk  Mon Dec 22 15:04:24 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Dec 22 15:04:19 2003
Subject: [Python-Dev] Re: Got None. Maybe Some?
References: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<20031222000819.10EC.JCARLSON@uci.edu>
Message-ID: <u13sliuv.fsf@yahoo.co.uk>

Josiah Carlson <jcarlson@uci.edu> writes:

> I ask the rest of you, does Some sound so crazy?

To be honest, yes. To me, None is not the low end of a scale, rather
it is a unique distinguished value, more related to the concept of
"uninitialised" or "not used" than to "smallest". The fact that it
compares less than every other value is to me incidental, and
arbitrary.

I see None as inhabiting the same sort of niche as the SQL NULL value.

>From the above, you'll gather that from my viewpoint, there really is
no "other extreme" from None, any more than there is from zero.

Paul.
-- 
This signature intentionally left blank


From tree at basistech.com  Mon Dec 22 15:06:42 2003
From: tree at basistech.com (Tom Emerson)
Date: Mon Dec 22 15:08:10 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net>
References: <1xqwodnr.fsf@python.net>
	<16359.18588.450934.263664@magrathea.basistech.com>
	<200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net>
Message-ID: <16359.20178.766911.979542@magrathea.basistech.com>

Guido van Rossum writes:
> > Indeed. See patch 849278 for some of the work we did to deal with this.
> 
> This mostly removes the use of the PYTHONXXX environment variables.
> Can you explain why you don't want these to work?

If the user installs our application and has a separate Python
installation on their machine for which they use those environment
variables, bad things can happen. I do not want to be in a position
where I'm told our app doesn't work because of environment variables
we have no control over, and it is the Wrong Answer to tell the user
to change their environment.

> Also note that you can already disable *all* use of the environment
> (not just in Py_Initialize()) by setting the global
> Py_IgnoreEnvironmentFlag to a non-zero value.  I'd reject that part of
> the patch for that reason.

I did not know about or notice this flag when I made the changes ---
in all the places I looked at the time the environment variables were
read without such a check, at least as I remember. It certainly makes
more sense to use the built in support.

> Py_SetSysPaths() might be a valuable addition, but the game you play
> with the prefiX name is abject. :)

Sure, whatever. These were made for our own use and later submitted,
They can be cleaned up as needed.

> What do you suggest the interpreter should do instead of calling
> abort()?  This is only called in response to *fatal* errors --
> situations where raising an exception is *not* an acceptable
> alternative for various reasons (maybe because it's in a piece of code
> that is part of the implementation of exceptions, or maybe because a
> corruption is detected in the very interpreter environment that's
> needed to raise an exception).

During initialization if there is a fatal error I can just disable the
functionality that uses the interpreter --- having my app abort
because Python cannot initialize for whatever reason is a Bad
Thing. Similarly with writing messages directly to stderr: I do not
necessarily want our users to see these since they don't know what to
do with them.

> Anout the only alternative I can think of would be a longjmp() call
> to a handler you have set up, but that can't be safe when the fatal
> error is discovered in a different thread.

In our use of Python we build without threads, so the longjmp() would
work. In the case of threads all bets are off.

> What's your embedding environment?  Why do you want to use Py_Main()
> at all?

I don't use Py_Main() in my embedding (multiple Unix flavors, 32- and
64-bit, and Win32 and Win64)... I wasn't reading closely enough.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Software Architect                                 http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"

From logistix at cathoderaymission.net  Mon Dec 22 15:18:24 2003
From: logistix at cathoderaymission.net (logistix)
Date: Mon Dec 22 15:09:52 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <m3d6ag3ckx.fsf@float.localdomain>
Message-ID: <000001c3c8c8$c0c116c0$20bba8c0@XP>



> -----Original Message-----
> From: 
> python-dev-bounces+logistix=cathoderaymission.net@python.org 
> [mailto:python-dev-bounces+logistix=cathoderaymission.net@pyth
> on.org] On Behalf Of Kurt B. Kaiser
> Sent: Monday, December 22, 2003 1:57 PM
> To: python-dev@python.org
> Subject: Re: [Python-Dev] OpenBSD anyone?
> 
> 
> Guido van Rossum <guido@python.org> writes:
> 
> > If anybody *does* see the bogus behavior and feels like 
> debugging it, 
> > I'd appreciate it!
> 
> I've been running cvs Python but not previously with debug 
> switched on.
> 
> I get a core when using OPT=-g.
> 
> (Note that I'm running Patch 850977 "Modify setup.py to 
> detect Tcl/Tk on BSD", though I wouldn't think it would cause 
> this, and I doubt he's using it.)
> 
> Investigating.  Hints appreciated!
> 
> -- 
> KBK
> 
>

I saw this a while ago and filed bug 712056.  In my case it was problems
with unicode and the newly introduced iconv module.  At the time, this
and a few other bugs caused the developers to yank the iconv module from
the release and things compiled properly.

Could be completely unrelated but I thought I'd mention it.


From pf_moore at yahoo.co.uk  Mon Dec 22 15:08:54 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Mon Dec 22 15:10:30 2003
Subject: [Python-Dev] Re: Relative import
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net> <1072104915.18828.47.camel@anthem>
	<200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net>
Message-ID: <pteglind.fsf@yahoo.co.uk>

Guido van Rossum <guido@python.org> writes:

>> I don't either, although my biggest problem with it is that it isn't
>> clear just by looking what the punctuation actually means.  I just
>> fear this will be a permanent head scratcher.
>
> This argument by itself has very little value.  The question isn't
> whether it's self-describing (very few features beyond decimal
> integers are); the question is whether it's easy to remember.  There
> are two subclasses of that: easy to remember what it means when you
> next see it, and easy to remember how to spell it when you next need
> it.  IMO the dot syntax does fine here.

The biggest "clarity" issue I have with the dot notation is that
there's no obvious keyword to look up when you want to find out what's
going on. The best you could do is look up the "import" section of the
manual. Heaven help you if you want to use Google. (Has anyone tried
to get any meaningful information from Google on Microsoft's two
technologies, COM and .NET? You'd almost think they'd done it
deliberately...

Seriously, the "what do I search for?" factor bugs me every time I hit
it.

Paul.
-- 
This signature intentionally left blank


From guido at python.org  Mon Dec 22 15:13:55 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 15:14:01 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: Your message of "Mon, 22 Dec 2003 15:06:42 EST."
	<16359.20178.766911.979542@magrathea.basistech.com> 
References: <1xqwodnr.fsf@python.net>
	<16359.18588.450934.263664@magrathea.basistech.com>
	<200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net> 
	<16359.20178.766911.979542@magrathea.basistech.com> 
Message-ID: <200312222013.hBMKDt118815@c-24-5-183-134.client.comcast.net>

> > This mostly removes the use of the PYTHONXXX environment variables.
> > Can you explain why you don't want these to work?
> 
> If the user installs our application and has a separate Python
> installation on their machine for which they use those environment
> variables, bad things can happen. I do not want to be in a position
> where I'm told our app doesn't work because of environment variables
> we have no control over, and it is the Wrong Answer to tell the user
> to change their environment.

Fair enough.  (Though you could have built your app to work even if
Python wrote some random stuff to stderr or optimized its bytecode.)

> > Also note that you can already disable *all* use of the environment
> > (not just in Py_Initialize()) by setting the global
> > Py_IgnoreEnvironmentFlag to a non-zero value.  I'd reject that part of
> > the patch for that reason.
> 
> I did not know about or notice this flag when I made the changes ---
> in all the places I looked at the time the environment variables were
> read without such a check, at least as I remember. It certainly makes
> more sense to use the built in support.

Perhaps it's easy to overlook that it's using Py_GETENV() rather than
getenv(); this macro takes tests for Py_IgnoreEnvironmentFlag first.

> > Py_SetSysPaths() might be a valuable addition, but the game you play
> > with the prefiX name is abject. :)
> 
> Sure, whatever. These were made for our own use and later submitted,
> They can be cleaned up as needed.

Please do.  Add some docs too -- undocumented new code isn't very
welcome, since we've already got enough of that. :-)

> > What do you suggest the interpreter should do instead of calling
> > abort()?  This is only called in response to *fatal* errors --
> > situations where raising an exception is *not* an acceptable
> > alternative for various reasons (maybe because it's in a piece of code
> > that is part of the implementation of exceptions, or maybe because a
> > corruption is detected in the very interpreter environment that's
> > needed to raise an exception).
> 
> During initialization if there is a fatal error I can just disable the
> functionality that uses the interpreter --- having my app abort
> because Python cannot initialize for whatever reason is a Bad
> Thing. Similarly with writing messages directly to stderr: I do not
> necessarily want our users to see these since they don't know what to
> do with them.
> 
> > Anout the only alternative I can think of would be a longjmp() call
> > to a handler you have set up, but that can't be safe when the fatal
> > error is discovered in a different thread.
> 
> In our use of Python we build without threads, so the longjmp() would
> work. In the case of threads all bets are off.

So that's an indirect answer to my question: you'd be happy if instead
of calling abort() it would longjmp() to a location you specified?  I
guess a better (more general) solution would be to have a callback
function that's called instead of abort(), and to initialize this with
something that calls abort() by default.  That's a useful feature; if
you upload it to SF with reference to this thread it ought to be
accepted.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From theller at python.net  Mon Dec 22 15:15:08 2003
From: theller at python.net (Thomas Heller)
Date: Mon Dec 22 15:15:27 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <200312222003.hBMK3eH18732@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 22 Dec 2003 12:03:40 -0800")
References: <1xqwodnr.fsf@python.net>
	<200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net>
	<u13smxtx.fsf@python.net>
	<200312222003.hBMK3eH18732@c-24-5-183-134.client.comcast.net>
Message-ID: <llp4mwxf.fsf@python.net>


>> I want to specify the Python path before calling Py_Initialize(), and I
>> don't want it to use *any* environment variables which (again) set the
>> flags.  So I thought of Py_InitializeEx() which would take parameters
>> specifying these things, and Py_Initialize() could be changed to call
>> Py_InitializeEx(NULL), maybe.
>
> If you don't want the environment to be used, set the global
> Py_IgnoreEnvironmentFlag before calling Py_Initialize().

I didn't know that, thanks for the hint.

[...]
>> Does it make more sense now?
>
> Not quite -- I don't understand why *you* don't want the environment
> variables to be used, if all you want is a better pythonw.exe.

I did probably mix two different things here.

> If that's your only experience with embedding Python, you can't really
> speak for other embedders.

That may be - my other experiences are py2exe and the embedding done in
bdist_wininst.

Now knowing Py_IgnoreEnvironmentFlag I would be satisfied with the
addition of a PySys_SetPath function ;-).

Thomas


From guido at python.org  Mon Dec 22 15:15:44 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 15:15:50 2003
Subject: [Python-Dev] Re: Relative import
In-Reply-To: Your message of "Mon, 22 Dec 2003 20:08:54 GMT."
	<pteglind.fsf@yahoo.co.uk> 
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<20031222075219.GA3426@performancedrivers.com>
	<200312220003.08810.troy@gci.net>
	<1072104915.18828.47.camel@anthem>
	<200312221536.hBMFa9S18026@c-24-5-183-134.client.comcast.net> 
	<pteglind.fsf@yahoo.co.uk> 
Message-ID: <200312222015.hBMKFi918839@c-24-5-183-134.client.comcast.net>

> The biggest "clarity" issue I have with the dot notation is that
> there's no obvious keyword to look up when you want to find out what's
> going on. The best you could do is look up the "import" section of the
> manual. Heaven help you if you want to use Google. (Has anyone tried
> to get any meaningful information from Google on Microsoft's two
> technologies, COM and .NET? You'd almost think they'd done it
> deliberately...
> 
> Seriously, the "what do I search for?" factor bugs me every time I hit
> it.

There's no deep reason why the language reference couldn't contain
non-alphanumeric characters, although this won't help for Google.  But
in this case, "from" would be the thing to look up, so I'm not sure I
see it as such a serious problem.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From mwh at python.net  Mon Dec 22 15:24:12 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 15:24:17 2003
Subject: [Python-Dev] Got None. Maybe Some?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCGEFNHPAB.tim.one@comcast.net> (Tim
	Peters's message of "Mon, 22 Dec 2003 11:46:01 -0500")
References: <LNBBLJKPBEHFEDALKOLCGEFNHPAB.tim.one@comcast.net>
Message-ID: <2mwu8od2j7.fsf@starship.python.net>

"Tim Peters" <tim.one@comcast.net> writes:

> None is handy and I like it, but it's arbitrary enough in some cases
> that it takes time to get used to its quirks

At least we don't pretend it's the empty list as well...

Cheers,
mwh

-- 
  On the other hand, the following areas are subject to boycott
  in reaction to the rampant impurity of design or execution, as
  determined after a period of study, in no particular order:
    ...                              http://www.naggum.no/profile.html

From tim.one at comcast.net  Mon Dec 22 15:34:38 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 22 15:34:42 2003
Subject: [Python-Dev] Re: Got None. Maybe Some?
In-Reply-To: <u13sliuv.fsf@yahoo.co.uk>
Message-ID: <LNBBLJKPBEHFEDALKOLCOEHMHPAB.tim.one@comcast.net>

[Paul Moore]
> ...
> The fact that [None] compares less than every other value is to me
> incidental, and arbitrary.

Arbitrary?!  I clearly recall when the decision was made that None would
compare less than anything else.  I was in the same room with Guido, and
this is a verbatim transcript of the debate:

"As long as we're changing everything here anyway, maybe comparing with None
should yield a consistent result."

"Like what, maybe smaller than anything else?"

"Sure.  Or larger."

"Ya, one of those.  It's not worth it if it just ends up in the middle
somewhere."

"What does Python do now?"

... [head-scratching] ...

"So what did it do before?"

... [more head-scratching] ...

"Well, that's no help, then."

"The Reference Manual doesn't promise anything about how None will compare,
just the 'consistent total ordering' bit".

"So that doesn't constrain us."

"No, it's neutral."

"So maybe it should just be smaller than everything else."

"Sure.  Or larger."

"If it were smaller, then when you sorted a mixed-type list, all the Nones
would float to the start."

"Good point.  If it were larger, they'd float to the end."

"I wouldn't like that much."

"Me neither."

"OK, it's settled."

Two lines of code later, the Only Objectively Correct result was achieved.

history-is-more-impressive-in-textbooks-ly y'rs  - tim


From guido at python.org  Mon Dec 22 16:04:11 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 16:04:17 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
Message-ID: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.net>

New-style classes inherit a default __hash__ from object, but this
is incorrect if the inheriting object defines __eq__ or __cmp__.

This has been reported several times; the roll-up SF entry is 660098,
which has the proposed patch attached as newpatch.txt.

I've long pondered this, and it seems the best solution is to simply
not define __hash__ on the base object class.  After all, you don't
inherit __eq__ or __cmp__ either; the default comparison behavior is
supplied by intrinsic behavior of comparisons, and the default hash()
behavior is also supplied by intrinsic behavior of hash().

Does anybody have any reason why I shouldn't check in this change in
2.4?  There's no unit test that fails if I do this.

It *is* a slight incompatibility though: if someone implemented their
__hash__ in terms of object.__hash__, they would get a somewhat
puzzling error message after the change, because object.__hash__ will
refer to the type.__hash__ function bound to object, and refuse to
take an argument:

  >>> object.__hash__()
  135328576
  >>> object.__hash__(42)
  Traceback (most recent call last):
    File "<stdin>", line 1, in ?
  TypeError:  expected 0 arguments, got 1
  >>>

Since the default hash simply takes id() of the object, it's easy to
fix such code once the failure is understood though.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Mon Dec 22 16:23:26 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon Dec 22 16:20:00 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.ne
 t>
Message-ID: <5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch>

At 13:04 22.12.2003 -0800, Guido van Rossum wrote:


>Since the default hash simply takes id() of the object, it's easy to
>fix such code once the failure is understood though.

Jython 2.2a0 on java1.4.0_02 (JIT: null)
Type "copyright", "credits" or "license" for more information.
 >>> class C: pass
...
 >>> id(C)
1
 >>> hash(C)
9174506
 >>>

I see, but then there should be probably be a different way to spell the 
default hash,
because id is not a sensible option for Jython etc



From pje at telecommunity.com  Mon Dec 22 16:19:10 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec 22 16:26:29 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.ne
 t>
Message-ID: <5.1.1.6.0.20031222161608.02e98d50@telecommunity.com>

At 01:04 PM 12/22/03 -0800, Guido van Rossum wrote:
>New-style classes inherit a default __hash__ from object, but this
>is incorrect if the inheriting object defines __eq__ or __cmp__.
>
>This has been reported several times; the roll-up SF entry is 660098,
>which has the proposed patch attached as newpatch.txt.
>
>I've long pondered this, and it seems the best solution is to simply
>not define __hash__ on the base object class.  After all, you don't
>inherit __eq__ or __cmp__ either; the default comparison behavior is
>supplied by intrinsic behavior of comparisons, and the default hash()
>behavior is also supplied by intrinsic behavior of hash().
>
>Does anybody have any reason why I shouldn't check in this change in
>2.4?  There's no unit test that fails if I do this.

Would this mean that instances of the following class:

class Dummy(object):
     pass

would no longer be usable as dictionary keys?  I guess the parts I'm not 
clear on are 1) whether dictionaries call the equivalent of 'hash(key)' or 
'key.__hash__()', and 2) whether 'hash(Dummy())' will do something meaningful.


From guido at python.org  Mon Dec 22 16:29:23 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 16:29:29 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: Your message of "Mon, 22 Dec 2003 16:19:10 EST."
	<5.1.1.6.0.20031222161608.02e98d50@telecommunity.com> 
References: <5.1.1.6.0.20031222161608.02e98d50@telecommunity.com> 
Message-ID: <200312222129.hBMLTNT19225@c-24-5-183-134.client.comcast.net>

> Would this mean that instances of the following class:
> 
> class Dummy(object):
>      pass
> 
> would no longer be usable as dictionary keys?  I guess the parts I'm
> not clear on are 1) whether dictionaries call the equivalent of
> 'hash(key)' or 'key.__hash__()', and 2) whether 'hash(Dummy())' will
> do something meaningful.

That class would still be usable.  hash(Dummy()) sees that there's no
__eq__ or __cmp__ override and then uses the default hash.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 16:32:15 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 16:32:21 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: Your message of "Mon, 22 Dec 2003 22:23:26 +0100."
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch> 
References: <5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch> 
Message-ID: <200312222132.hBMLWFe19253@c-24-5-183-134.client.comcast.net>

> >Since the default hash simply takes id() of the object, it's easy to
> >fix such code once the failure is understood though.
> 
> Jython 2.2a0 on java1.4.0_02 (JIT: null)
> Type "copyright", "credits" or "license" for more information.
>  >>> class C: pass
> ...
>  >>> id(C)
> 1
>  >>> hash(C)
> 9174506
>  >>>
> 
> I see, but then there should be probably be a different way to spell
> the default hash, because id is not a sensible option for Jython etc

Ow, you're right.  I bet this is why object.__hash__ was introduced in
the first place.

We're either back to square one, or we can add a __default_hash__ to
object which has the default hash implementation -- this isn't very
pretty but at least it works.

(I should add that this topic rose to my attention when I had to debug
the issue in some code of my own -- I had a class whose __eq__ was
overridden but whose __hash__ wasn't, and it was being used as a dict
key...  It is indeed a pain to debug this. :-( )

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Mon Dec 22 16:43:40 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon Dec 22 16:40:25 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <200312222132.hBMLWFe19253@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Mon, 22 Dec 2003 22:23:26 +0100."
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch>
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20031222224015.02f0a640@pop.bluewin.ch>

At 13:32 22.12.2003 -0800, Guido van Rossum wrote:
> > >Since the default hash simply takes id() of the object, it's easy to
> > >fix such code once the failure is understood though.
> >
> > Jython 2.2a0 on java1.4.0_02 (JIT: null)
> > Type "copyright", "credits" or "license" for more information.
> >  >>> class C: pass
> > ...
> >  >>> id(C)
> > 1
> >  >>> hash(C)
> > 9174506
> >  >>>
> >
> > I see, but then there should be probably be a different way to spell
> > the default hash, because id is not a sensible option for Jython etc
>
>Ow, you're right.  I bet this is why object.__hash__ was introduced in
>the first place.
>
>We're either back to square one, or we can add a __default_hash__ to
>object which has the default hash implementation -- this isn't very
>pretty but at least it works.

yes, or hash should grow a keyword argument to get access to the default 
impl, which would be id for CPython but not for all Python impls in 
general. The problem is that in general it cannot be expected from Python 
implementations to implement the default hash in terms of id because the 
latter can be much more costly beacuse of its uniqueness constraints.

regards. 


From kbk at shore.net  Mon Dec 22 16:41:13 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Mon Dec 22 16:41:18 2003
Subject: [Python-Dev] OpenBSD anyone?
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<200312221907.hBMJ7hg18485@c-24-5-183-134.client.comcast.net>
	<m3vfo81vrs.fsf@float.localdomain>
	<200312221959.hBMJxhY18697@c-24-5-183-134.client.comcast.net>
Message-ID: <m3d6agzg1y.fsf@float.localdomain>

Guido van Rossum <guido@python.org> writes:

> That suggests it's still in Py_Initialize().  What does the gdb
> command 'bt' say???
>
> I'd also try another experiment: instead of "run" try "run -S".  This
> passes the -S option to Python when it is started, so that it doesn't
> try to load site.py (which executes rather a lot of Python code).  I'd
> be interested in seeing how much you can do interactively in that
> case, of if it still crashes in Py_Initialize().

Script started on Mon Dec 22 15:20:25 2003
hydra /home/kbk/proj/sandbox/python_clean$ gdb ./python
GNU gdb 4.16.1
Copyright 1996 Free Software Foundation, Inc.
GDB is free software, covered by the GNU General Public License, and you are
welcome to change it and/or distribute copies of it under certain conditions.
Type "show copying" to see the conditions.
There is absolutely no warranty for GDB.  Type "show warranty" for details.
This GDB was configured as "i386-unknown-openbsd3.3"...
(gdb) run -S
Starting program: /home/kbk/proj/sandbox/python_clean/./python -S

Program received signal SIGSEGV, Segmentation fault.
0x401900a0 in strchr ()
(gdb) bt
#0  0x401900a0 in strchr ()
#1  0x1d11d in load_next (mod=0x10905c, altmod=0x10905c, p_name=0xcfbfd640, 
    buf=0xcfbfd230 "__builtin__", p_buflen=0xcfbfd22c) at Python/import.c:2004
#2  0x1cc40 in import_module_ex (name=0x19786 "__builtin__", globals=0x0, locals=0x0, 
    fromlist=0x0) at Python/import.c:1888
#3  0x1ce29 in PyImport_ImportModuleEx (name=0x19786 "__builtin__", globals=0x0, locals=0x0, 
    fromlist=0x0) at Python/import.c:1922
#4  0x1dfe9 in PyImport_Import (module_name=0x115598) at Python/import.c:2333
#5  0x1caec in PyImport_ImportModule (name=0xb895e "__builtin__") at Python/import.c:1853
#6  0xb8b3d in _PyExc_Init () at Python/exceptions.c:1755
#7  0x25ad4 in Py_Initialize () at Python/pythonrun.c:205
#8  0x282f in Py_Main (argc=2, argv=0xcfbfd82c) at Modules/main.c:376
#9  0x17e3 in main (argc=2, argv=0xcfbfd82c) at Modules/python.c:23
(gdb) q
The program is running.  Quit anyway (and kill it)? (y or n) y
hydra /home/kbk/proj/sandbox/python_clean$ 
Script done on Mon Dec 22 15:21:06 2003


Same result w/o -S

=============================================================
Slightly later:

(gdb) b import.c:2004
Breakpoint 1 at 0x1d10f: file Python/import.c, line 2004.
(gdb) r
Starting program: /home/kbk/proj/sandbox/python_clean/./python 

Breakpoint 1, load_next (mod=0xe98ec, altmod=0xe98ec, p_name=0xcfbfd88c, buf=0xcfbfd47c "", 
    p_buflen=0xcfbfd478) at Python/import.c:2004
2004            char *dot = strchr(name, '.');
(gdb) p *p_name
$1 = 0x19786 "__builtin__"
(gdb) p name
$2 = 0x19786 "__builtin__"
(gdb) p strchr(name, '.')

Program received signal SIGSEGV, Segmentation fault.

OTOH, if I break at 2004 and then step once, I get by the strchr call
OK.  Also if I stepi through it.  If I continue, it segfaults at the
next execution of line 2004.  Weird.

Investigating.

-- 
KBK

From guido at python.org  Mon Dec 22 16:43:09 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 16:43:16 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: Your message of "Mon, 22 Dec 2003 22:43:40 +0100."
	<5.2.1.1.0.20031222224015.02f0a640@pop.bluewin.ch> 
References: <Your message of "Mon, 22 Dec 2003 22:23:26 +0100."
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch>
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch> 
	<5.2.1.1.0.20031222224015.02f0a640@pop.bluewin.ch> 
Message-ID: <200312222143.hBMLh9C19299@c-24-5-183-134.client.comcast.net>

> > > I see, but then there should be probably be a different way to spell
> > > the default hash, because id is not a sensible option for Jython etc
> >
> >Ow, you're right.  I bet this is why object.__hash__ was introduced in
> >the first place.
> >
> >We're either back to square one, or we can add a __default_hash__ to
> >object which has the default hash implementation -- this isn't very
> >pretty but at least it works.
> 
> yes, or hash should grow a keyword argument to get access to the default 
> impl, which would be id for CPython but not for all Python impls in 
> general. The problem is that in general it cannot be expected from Python 
> implementations to implement the default hash in terms of id because the 
> latter can be much more costly beacuse of its uniqueness constraints.

Hm, the keyword arg would be more work to implement in C, and I don't
really like the ad-hoc extension of the __hash__ signature -- it may
make subclasses think they have to support the keyword too.

(Overriding the __default_hash__ would be possible, but silly, and I
recommend not to support this; if this were Java I'd make it a final
method.  Can you make it final in Jython?)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Mon Dec 22 16:55:48 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Mon Dec 22 16:52:22 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <200312222143.hBMLh9C19299@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Mon, 22 Dec 2003 22:43:40 +0100."
	<5.2.1.1.0.20031222224015.02f0a640@pop.bluewin.ch>
	<Your message of "Mon, 22 Dec 2003 22:23:26 +0100."
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch>
	<5.2.1.1.0.20031222221506.02f2ccf8@pop.bluewin.ch>
	<5.2.1.1.0.20031222224015.02f0a640@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20031222224820.02f31840@pop.bluewin.ch>

At 13:43 22.12.2003 -0800, Guido van Rossum wrote:
> > > > I see, but then there should be probably be a different way to spell
> > > > the default hash, because id is not a sensible option for Jython etc
> > >
> > >Ow, you're right.  I bet this is why object.__hash__ was introduced in
> > >the first place.
> > >
> > >We're either back to square one, or we can add a __default_hash__ to
> > >object which has the default hash implementation -- this isn't very
> > >pretty but at least it works.
> >
> > yes, or hash should grow a keyword argument to get access to the default
> > impl, which would be id for CPython but not for all Python impls in
> > general. The problem is that in general it cannot be expected from Python
> > implementations to implement the default hash in terms of id because the
> > latter can be much more costly beacuse of its uniqueness constraints.
>
>Hm, the keyword arg would be more work to implement in C, and I don't
>really like the ad-hoc extension of the __hash__ signature -- it may
>make subclasses think they have to support the keyword too.
>
>(Overriding the __default_hash__ would be possible, but silly, and I
>recommend not to support this; if this were Java I'd make it a final
>method.  Can you make it final in Jython?)

internally yes. But I would probably hardcode is usage in hash, so that is 
just a way to access the default hash impl and nothing more.



From guido at python.org  Mon Dec 22 16:56:40 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 16:56:52 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Mon, 22 Dec 2003 16:41:13 EST."
	<m3d6agzg1y.fsf@float.localdomain> 
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<200312221907.hBMJ7hg18485@c-24-5-183-134.client.comcast.net>
	<m3vfo81vrs.fsf@float.localdomain>
	<200312221959.hBMJxhY18697@c-24-5-183-134.client.comcast.net> 
	<m3d6agzg1y.fsf@float.localdomain> 
Message-ID: <200312222156.hBMLueA19333@c-24-5-183-134.client.comcast.net>

(JD: yes, it's real!)

> Script started on Mon Dec 22 15:20:25 2003
> hydra /home/kbk/proj/sandbox/python_clean$ gdb ./python
> GNU gdb 4.16.1
> Copyright 1996 Free Software Foundation, Inc.
> GDB is free software, covered by the GNU General Public License, and you are
> welcome to change it and/or distribute copies of it under certain conditions.
> Type "show copying" to see the conditions.
> There is absolutely no warranty for GDB.  Type "show warranty" for details.
> This GDB was configured as "i386-unknown-openbsd3.3"...
> (gdb) run -S
> Starting program: /home/kbk/proj/sandbox/python_clean/./python -S
> 
> Program received signal SIGSEGV, Segmentation fault.
> 0x401900a0 in strchr ()
> (gdb) bt
> #0  0x401900a0 in strchr ()
> #1  0x1d11d in load_next (mod=0x10905c, altmod=0x10905c, p_name=0xcfbfd640, 
>     buf=0xcfbfd230 "__builtin__", p_buflen=0xcfbfd22c) at Python/import.c:2004
> #2  0x1cc40 in import_module_ex (name=0x19786 "__builtin__", globals=0x0, locals=0x0, 
>     fromlist=0x0) at Python/import.c:1888
> #3  0x1ce29 in PyImport_ImportModuleEx (name=0x19786 "__builtin__", globals=0x0, locals=0x0, 
>     fromlist=0x0) at Python/import.c:1922
> #4  0x1dfe9 in PyImport_Import (module_name=0x115598) at Python/import.c:2333
> #5  0x1caec in PyImport_ImportModule (name=0xb895e "__builtin__") at Python/import.c:1853
> #6  0xb8b3d in _PyExc_Init () at Python/exceptions.c:1755
> #7  0x25ad4 in Py_Initialize () at Python/pythonrun.c:205
> #8  0x282f in Py_Main (argc=2, argv=0xcfbfd82c) at Modules/main.c:376
> #9  0x17e3 in main (argc=2, argv=0xcfbfd82c) at Modules/python.c:23
> (gdb) q
> The program is running.  Quit anyway (and kill it)? (y or n) y
> hydra /home/kbk/proj/sandbox/python_clean$ 
> Script done on Mon Dec 22 15:21:06 2003
> 
> 
> Same result w/o -S
> 
> =============================================================
> Slightly later:
> 
> (gdb) b import.c:2004
> Breakpoint 1 at 0x1d10f: file Python/import.c, line 2004.
> (gdb) r
> Starting program: /home/kbk/proj/sandbox/python_clean/./python 
> 
> Breakpoint 1, load_next (mod=0xe98ec, altmod=0xe98ec, p_name=0xcfbfd88c, buf=0xcfbfd47c "", 
>     p_buflen=0xcfbfd478) at Python/import.c:2004
> 2004            char *dot = strchr(name, '.');
> (gdb) p *p_name
> $1 = 0x19786 "__builtin__"
> (gdb) p name
> $2 = 0x19786 "__builtin__"
> (gdb) p strchr(name, '.')
> 
> Program received signal SIGSEGV, Segmentation fault.
> 
> OTOH, if I break at 2004 and then step once, I get by the strchr call
> OK.  Also if I stepi through it.  If I continue, it segfaults at the
> next execution of line 2004.  Weird.
> 
> Investigating.

The most likely cause then is some kind of bug in the platform's
strchr().  This could explain why -O3 fixes the issue: I think I've
heard of GCC replacing calls to strchr(), strcpy() etc. with inline
code, thereby avoiding the buggy library version (and explaining why
the buggy code could persist undetected in the library -- most system
code is of course compiled fully optimized).

As to why stepi doesn't trigger the segfault: possibly it's a timing
bug that doesn't occur when run one instruction at a time.  This would
even make it CPU dependent, which would explain that some folks didn't
see this.  I don't have the OpenBSD strchr.c source code online here
so I'll stop speculating here...

--Guido van Rossum (home page: http://www.python.org/~guido/)


From blunck at gst.com  Mon Dec 22 16:58:55 2003
From: blunck at gst.com (Christopher Blunck)
Date: Mon Dec 22 16:58:59 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312221929.hBMJTef18533@c-24-5-183-134.client.comcast.net>
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<20031222192653.GA31766@homer.gst.com>
	<200312221929.hBMJTef18533@c-24-5-183-134.client.comcast.net>
Message-ID: <20031222215855.GA32420@homer.gst.com>

On Mon, Dec 22, 2003 at 11:29:40AM -0800, Guido van Rossum wrote:
> > Interesting...  When I do OPT=-g, I get a Memory fault (core dumped) 
> > when linking python.  Is that when you receive the core dump?
> 
> >From looking at his output, he gets it just after the linking, when
> using the freshly linked python executable to run the setup.py script.
> 
> Are you saying you didn't try the debug build?


I received the dump when trying to run setup.py build.


-c

-- 
 16:55:00  up 21 days,  6:39, 12 users,  load average: 1.30, 0.60, 0.24

From whisper at oz.net  Mon Dec 22 15:43:09 2003
From: whisper at oz.net (David LeBlanc)
Date: Mon Dec 22 16:59:31 2003
Subject: [Python-Dev] Re: Got None. Maybe Some?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCOEHMHPAB.tim.one@comcast.net>
Message-ID: <LAEHLEAOBBLKAALGAPAFOEABCMAA.whisper@oz.net>


> [Paul Moore]
> > ...
> > The fact that [None] compares less than every other value is to me
> > incidental, and arbitrary.
>
> Arbitrary?!  I clearly recall when the decision was made that None would
> compare less than anything else.  I was in the same room with Guido, and
> this is a verbatim transcript of the debate:
<snip>
> Two lines of code later, the Only Objectively Correct result was achieved.
>
> history-is-more-impressive-in-textbooks-ly y'rs  - tim
>
How about "everything" or "all" as the obverse of "none", ordered higher
than all other values? "Some" is far too indeterminate.

Dave LeBlanc
Seattle, WA USA


From blunck at gst.com  Mon Dec 22 16:59:54 2003
From: blunck at gst.com (Christopher Blunck)
Date: Mon Dec 22 17:00:29 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <2m1xqwei9s.fsf@starship.python.net>
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>
	<20031222170257.GA31352@homer.gst.com>
	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>
	<m3d6ag3ckx.fsf@float.localdomain>
	<20031222192653.GA31766@homer.gst.com>
	<2m1xqwei9s.fsf@starship.python.net>
Message-ID: <20031222215954.GB32420@homer.gst.com>

On Mon, Dec 22, 2003 at 07:58:55PM +0000, Michael Hudson wrote:
> What version of gcc?  Anthony and I hacked around on HP's compile farm
> in the 2.3.2 timeframe and saw gcc problems which we hacked around,
> but nothing like this.

2.95.3.  But I think I was mistaken in my last email.  I get the dump when I
run setup.py build, not when linking python.


-c

-- 
 16:55:00  up 21 days,  6:39, 12 users,  load average: 1.30, 0.60, 0.24

From barry at python.org  Mon Dec 22 17:10:07 2003
From: barry at python.org (Barry Warsaw)
Date: Mon Dec 22 17:10:20 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.net>
References: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.net>
Message-ID: <1072131005.13676.9.camel@anthem>

On Mon, 2003-12-22 at 16:04, Guido van Rossum wrote:

>   >>> object.__hash__()
>   135328576
>   >>> object.__hash__(42)
>   Traceback (most recent call last):
>     File "<stdin>", line 1, in ?
>   TypeError:  expected 0 arguments, got 1
>   >>>

Would it be better if object.__hash__() raised a NotImplementedError?

-Barry



From jepler at unpythonic.net  Mon Dec 22 17:10:58 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec 22 17:11:03 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <20031222140255.GH12586@unpythonic.net>
References: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
	<3FE65AF6.6010403@v.loewis.de>
	<20031222140255.GH12586@unpythonic.net>
Message-ID: <20031222221058.GD24252@unpythonic.net>

On Mon, Dec 22, 2003 at 08:03:17AM -0600, Jeff Epler wrote:
> Well, unlike the last two bugs I filed in redhat bugzilla, this one has
> already seen some activity.  I hope they don't take the line that this
> is not a bug :(

Unfortunately, this is redhat's position.

------- Additional Comments From roland@redhat.com  2003-12-22 16:37 -------
I think it is clear that the specification refers to the elements of
the child process state that survive exec, so that the executed
command can perceive them as part of its "environment".  You could
submit an interpretation request, but I think the committee would
concur with my reading.  The specification of pthread_atfork refers to
calls to fork, not to other parts of the POSIX.1 implementation.  If
your application calls system, and not fork, those clauses do not
apply to it.
 
roland@redhat.com changed:
 
           What    |Removed                     |Added
----------------------------------------------------------------------------
         Resolution|                            |NOTABUG
             Status|ASSIGNED                    |CLOSED

From guido at python.org  Mon Dec 22 17:13:02 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 17:13:15 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: Your message of "Mon, 22 Dec 2003 17:10:07 EST."
	<1072131005.13676.9.camel@anthem> 
References: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.net>  
	<1072131005.13676.9.camel@anthem> 
Message-ID: <200312222213.hBMMD2m19449@c-24-5-183-134.client.comcast.net>

> >   >>> object.__hash__()
> >   135328576
> >   >>> object.__hash__(42)
> >   Traceback (most recent call last):
> >     File "<stdin>", line 1, in ?
> >   TypeError:  expected 0 arguments, got 1
> >   >>>
> 
> Would it be better if object.__hash__() raised a NotImplementedError?

It can't -- it's type.__hash__(object).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Mon Dec 22 17:23:11 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 17:23:17 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: Your message of "Mon, 22 Dec 2003 16:10:58 CST."
	<20031222221058.GD24252@unpythonic.net> 
References: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
	<3FE65AF6.6010403@v.loewis.de>
	<20031222140255.GH12586@unpythonic.net> 
	<20031222221058.GD24252@unpythonic.net> 
Message-ID: <200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>

> Unfortunately, this is redhat's position.
> 
> ------- Additional Comments From roland@redhat.com  2003-12-22 16:37 -------
> I think it is clear that the specification refers to the elements of
> the child process state that survive exec, so that the executed
> command can perceive them as part of its "environment".  You could
> submit an interpretation request, but I think the committee would
> concur with my reading.  The specification of pthread_atfork refers to
> calls to fork, not to other parts of the POSIX.1 implementation.  If
> your application calls system, and not fork, those clauses do not
> apply to it.

How hard would it be to reimplement our own system() and popen() using
only POSIX calls, for POSIX systems?  I've always thought of these to
be pretty simple combinations of fork() and exec(), with an assumption
of a working /bin/sh.  Without error checking:

int
system(char *cmd) {
    pid = fork();
    if (!pid) { /* child */
        execl("/bin/sh", "-c", cmd, NULL);
    }
    else { /* parent */
        int sts;
        waitpid(pid, *sts, 0);
        return sts;
}

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jepler at unpythonic.net  Mon Dec 22 17:38:20 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec 22 17:38:26 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
References: <m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
	<3FE65AF6.6010403@v.loewis.de>
	<20031222140255.GH12586@unpythonic.net>
	<20031222221058.GD24252@unpythonic.net>
	<200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
Message-ID: <20031222223811.GA1685@unpythonic.net>

On Mon, Dec 22, 2003 at 02:23:11PM -0800, Guido van Rossum wrote:
> How hard would it be to reimplement our own system() and popen() using
> only POSIX calls, for POSIX systems?  I've always thought of these to
> be pretty simple combinations of fork() and exec(), with an assumption
> of a working /bin/sh.  Without error checking:

Yeah -- this is something we could do if necessary.  the opengroup
standard lists a few more things to "get right" in terms of signal
handling for system(), but we can do all those in C or in Python..

Jeff

From mwh at python.net  Mon Dec 22 17:39:53 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 17:39:55 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <200312222013.hBMKDt118815@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 22 Dec 2003 12:13:55 -0800")
References: <1xqwodnr.fsf@python.net>
	<16359.18588.450934.263664@magrathea.basistech.com>
	<200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net>
	<16359.20178.766911.979542@magrathea.basistech.com>
	<200312222013.hBMKDt118815@c-24-5-183-134.client.comcast.net>
Message-ID: <2msmjccw92.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> > (not just in Py_Initialize()) by setting the global
>> > Py_IgnoreEnvironmentFlag to a non-zero value.  I'd reject that part of
>> > the patch for that reason.
>> 
>> I did not know about or notice this flag when I made the changes ---
>> in all the places I looked at the time the environment variables were
>> read without such a check, at least as I remember. It certainly makes
>> more sense to use the built in support.
>
> Perhaps it's easy to overlook that it's using Py_GETENV() rather than
> getenv(); this macro takes tests for Py_IgnoreEnvironmentFlag first.

Py_IgnoreEnvironmentFlag and the -E command line option are both
/fairly/ new... 2.2 era?

Cheers,
mwh

-- 
  Worryingly, DEFUN appears to be a function that removes all the
  fun from something: after using it all your code is converted 
  to C++.                              -- Tim Bradshaw, comp.lang.lisp

From mwh at python.net  Mon Dec 22 17:41:37 2003
From: mwh at python.net (Michael Hudson)
Date: Mon Dec 22 17:41:40 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: <200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Mon, 22 Dec 2003 14:23:11 -0800")
References: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net> <3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net> <3FE65AF6.6010403@v.loewis.de>
	<20031222140255.GH12586@unpythonic.net>
	<20031222221058.GD24252@unpythonic.net>
	<200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
Message-ID: <2moeu0cw66.fsf@starship.python.net>

Guido van Rossum <guido@python.org> writes:

>> Unfortunately, this is redhat's position.
>> 
>> ------- Additional Comments From roland@redhat.com  2003-12-22 16:37 -------
>> I think it is clear that the specification refers to the elements of
>> the child process state that survive exec, so that the executed
>> command can perceive them as part of its "environment".  You could
>> submit an interpretation request, but I think the committee would
>> concur with my reading.  The specification of pthread_atfork refers to
>> calls to fork, not to other parts of the POSIX.1 implementation.  If
>> your application calls system, and not fork, those clauses do not
>> apply to it.
>
> How hard would it be to reimplement our own system() and popen() using
> only POSIX calls, for POSIX systems?  I've always thought of these to
> be pretty simple combinations of fork() and exec(), with an assumption
> of a working /bin/sh.  Without error checking:

I think it's a bit harder than what you post, but there's code in APUE
for it...

Cheers,
mwh

-- 
  I'd certainly be shocked to discover a consensus.  ;-)
                                             -- Aahz, comp.lang.python

From jcarlson at uci.edu  Mon Dec 22 17:41:29 2003
From: jcarlson at uci.edu (Josiah Carlson)
Date: Mon Dec 22 17:44:59 2003
Subject: [Python-Dev] Re: Got None. Maybe Some?
In-Reply-To: <LAEHLEAOBBLKAALGAPAFOEABCMAA.whisper@oz.net>
References: <LNBBLJKPBEHFEDALKOLCOEHMHPAB.tim.one@comcast.net>
	<LAEHLEAOBBLKAALGAPAFOEABCMAA.whisper@oz.net>
Message-ID: <20031222144122.0A7F.JCARLSON@uci.edu>

> > [Paul Moore]
> > > ...
> > > The fact that [None] compares less than every other value is to me
> > > incidental, and arbitrary.
> >
> > Arbitrary?!  I clearly recall when the decision was made that None would
> > compare less than anything else.  I was in the same room with Guido, and
> > this is a verbatim transcript of the debate:
> <snip>
> > Two lines of code later, the Only Objectively Correct result was achieved.
> >
> > history-is-more-impressive-in-textbooks-ly y'rs  - tim
> >
> How about "everything" or "all" as the obverse of "none", ordered higher
> than all other values? "Some" is far too indeterminate.
> 
> Dave LeBlanc
> Seattle, WA USA

All seems like a reasonable spelling for an object with the given
behavior.  I started with the name Some because the name made me smile.

 - Josiah

From guido at python.org  Mon Dec 22 17:45:04 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 22 17:45:13 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: Your message of "Mon, 22 Dec 2003 22:39:53 GMT."
	<2msmjccw92.fsf@starship.python.net> 
References: <1xqwodnr.fsf@python.net>
	<16359.18588.450934.263664@magrathea.basistech.com>
	<200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net>
	<16359.20178.766911.979542@magrathea.basistech.com>
	<200312222013.hBMKDt118815@c-24-5-183-134.client.comcast.net> 
	<2msmjccw92.fsf@starship.python.net> 
Message-ID: <200312222245.hBMMj4W19537@c-24-5-183-134.client.comcast.net>

> > Perhaps it's easy to overlook that it's using Py_GETENV() rather than
> > getenv(); this macro takes tests for Py_IgnoreEnvironmentFlag first.
> 
> Py_IgnoreEnvironmentFlag and the -E command line option are both
> /fairly/ new... 2.2 era?

Yeah, but the patch he suggested was using Py_GETENV(), so it's not
like it didn't exist when they did the patch; they must have missed
its significance. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tree at basistech.com  Mon Dec 22 18:49:37 2003
From: tree at basistech.com (Tom Emerson)
Date: Mon Dec 22 18:51:32 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <200312222245.hBMMj4W19537@c-24-5-183-134.client.comcast.net>
References: <1xqwodnr.fsf@python.net>
	<16359.18588.450934.263664@magrathea.basistech.com>
	<200312221954.hBMJs1M18648@c-24-5-183-134.client.comcast.net>
	<16359.20178.766911.979542@magrathea.basistech.com>
	<200312222013.hBMKDt118815@c-24-5-183-134.client.comcast.net>
	<2msmjccw92.fsf@starship.python.net>
	<200312222245.hBMMj4W19537@c-24-5-183-134.client.comcast.net>
Message-ID: <16359.33553.818193.723995@magrathea.basistech.com>

Guido van Rossum writes:
> > > Perhaps it's easy to overlook that it's using Py_GETENV() rather than
> > > getenv(); this macro takes tests for Py_IgnoreEnvironmentFlag first.
> > 
> > Py_IgnoreEnvironmentFlag and the -E command line option are both
> > /fairly/ new... 2.2 era?
> 
> Yeah, but the patch he suggested was using Py_GETENV(), so it's not
> like it didn't exist when they did the patch; they must have missed
> its significance. :-)

Yes, *they* did miss the significance of the use of the macro. 

-- 
Tom Emerson                                          Basis Technology Corp.
Software Architect                                 http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"

From martin at v.loewis.de  Mon Dec 22 15:01:28 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 22 20:46:49 2003
Subject: [Python-Dev] Re: python/dist/src/Objectsunicodeobject.c, 2.204,
	2.205
In-Reply-To: <2mptegej2x.fsf@starship.python.net>
References: <LNBBLJKPBEHFEDALKOLCIEOGHOAB.tim.one@comcast.net>
	<2mptegej2x.fsf@starship.python.net>
Message-ID: <3FE74D98.1020307@v.loewis.de>

Michael Hudson wrote:
 > As Martin regularly points out, we have rather too much of this kind
 > of thing.  Given that we demand ANSI C, we could surely lose
 > HAVE_LIMITS_H, and probably much other cruft.

Indeed, and I would encourage contributions in that direction. Make
sure you put, in the commit message, an elaboration why it is safe
to delete the code in question.

Regards,
Martin


From martin at v.loewis.de  Mon Dec 22 15:39:20 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 22 21:19:40 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <200312222003.hBMK3eH18732@c-24-5-183-134.client.comcast.net>
References: <1xqwodnr.fsf@python.net>	<200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net>
	<u13smxtx.fsf@python.net>
	<200312222003.hBMK3eH18732@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE75678.6090508@v.loewis.de>

Guido van Rossum wrote:

> Not quite -- I don't understand why *you* don't want the environment
> variables to be used, if all you want is a better pythonw.exe.

I haven't embedded Python for a while, but I can sympathize with the
desire to ignore environment variables.

One approach to embedding is to provide a complete (stripped) copy
of Python, inside the target directory of the application. You might
even directly link with a static python23.lib, and freeze all
modules, instead of linking with python23.dll, and providing a copy
of all .py files.

In such an environment, you don't want a regular Python installation
to interfere with your embedded environment. For example, you may
have incorporated the library Foo, but the target system has another
(older) copy of Foo in PYTHONPATH. If your embedded application now
picks up the older copy of Foo, it fails, and all your testing goes
to hell (*) since arbitrary pieces of the library might get arbitrarily
replaced with something else.

So to allow reproducable results, you need a guarantee that nothing
on the target system is incorporated in your application.

Regards,
Martin

(*) DLL hell, to be precise.


From pinard at iro.umontreal.ca  Mon Dec 22 22:20:05 2003
From: pinard at iro.umontreal.ca (=?iso-8859-1?Q?Fran=E7ois?= Pinard)
Date: Mon Dec 22 22:22:34 2003
Subject: [Python-Dev] Improve Python for embedding
In-Reply-To: <3FE75678.6090508@v.loewis.de>
References: <1xqwodnr.fsf@python.net>
	<200312221933.hBMJXTI18559@c-24-5-183-134.client.comcast.net>
	<u13smxtx.fsf@python.net>
	<200312222003.hBMK3eH18732@c-24-5-183-134.client.comcast.net>
	<3FE75678.6090508@v.loewis.de>
Message-ID: <20031223032005.GA16709@titan.progiciels-bpi.ca>

[Martin von L?wis]

> So to allow reproducable results, you need a guarantee that nothing
> on the target system is incorporated in your application.

I did not closely follow this thread, and apologise if I repeat
arguments which have already been said.

While I quite understand the argument and needs which Martin expresses,
there are situations where one is happy to tune the behaviour of the
embedded Python through the usual environment variables.

It might depend if the embedding application intends to either expose or
hide the existence of Python.  If the application offers Python as an
extension language for the system implemented by the application, Python
is then exposed, and the user should be able to use environment variable
to his/her advantage.  If the application uses Python internally, all to
itself, then the application likely wants to hide even the fact that it
uses Python, exerts full control, or at least decide exactly how much or
little the user is allowed to interfere, as Martin explained.

If it is given that only one of these avenues is favoured, I would for
one prefer that the "exposed" direction be the easier to implement,
which may be the statu quo over the current situation.  The "hiding"
direction might be evaluated so it requires a bit more work for
the implementor, about deleting or forcing the values of a few
sensitive environment variables.  Especially given that a Python hiding
application is likely to control other environment variables as well
(like maybe those related to loader path search, or localisation).

-- 
Fran?ois Pinard   http://www.iro.umontreal.ca/~pinard

From lists at hlabs.spb.ru  Tue Dec 23 06:05:37 2003
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Tue Dec 23 03:05:34 2003
Subject: [Python-Dev] Importing packages from command line
In-Reply-To: <3FE6CC59.10100@iinet.net.au>
References: <1071839460.3fe2f8e4dbbad@mcherm.com>	<3FE6D94A.2060904@hlabs.spb.ru>
	<3FE6CC59.10100@iinet.net.au>
Message-ID: <3FE82181.50309@hlabs.spb.ru>

Nick Coghlan wrote:
> Dmitry Vasiliev wrote:
> 
>> The main idea is to treating package as a program and run package 
>> initialization code from command line. The advantage is zipping all 
>> program modules in one zip archive and running the program from 
>> command line without unzipping it, like Java's jar's. But this idea 
>> need more thoughts however...
> 
>   python runzip.py mypkg.zip
> 
> where runzip.py is a pure Python script that works as you suggest.
> 
> I'm fairly sure zipimport would allow you to write the program you 
> describe in pure Python. I've obviously never written such a program, 
> though.

Code is fairly simple.

import sys
sys.path.insert(0, "module.zip")
import module

> It's certainly an interesting packaging idea for small Python programs 
> with several files, but for which a full-blown installer would be overkill.

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)


From lists at hlabs.spb.ru  Tue Dec 23 06:09:27 2003
From: lists at hlabs.spb.ru (Dmitry Vasiliev)
Date: Tue Dec 23 03:09:19 2003
Subject: [Python-Dev] Importing packages from command line
In-Reply-To: <200312221807.14352.aleaxit@yahoo.com>
References: <1071839460.3fe2f8e4dbbad@mcherm.com>	<3FE6D94A.2060904@hlabs.spb.ru>
	<200312221807.14352.aleaxit@yahoo.com>
Message-ID: <3FE82267.3030607@hlabs.spb.ru>

Alex Martelli wrote:
>>The main idea is to treating package as a program and run package
>>initialization code from command line. The advantage is zipping all
>>program modules in one zip archive and running the program from command
>>line without unzipping it, like Java's jar's. But this idea need more
>>thoughts however...
> 
> Couldn't you use:
>     python -c "import package_name"
> for this purpose even today?

python -c "import sys; sys.path.insert(0, 'module.zip'); import module"

Seems ugly... :)

-- 
Dmitry Vasiliev (dima at hlabs.spb.ru)


From aleaxit at yahoo.com  Tue Dec 23 04:05:41 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Tue Dec 23 04:05:51 2003
Subject: [Python-Dev] Importing packages from command line
In-Reply-To: <3FE82267.3030607@hlabs.spb.ru>
References: <1071839460.3fe2f8e4dbbad@mcherm.com>
	<200312221807.14352.aleaxit@yahoo.com>
	<3FE82267.3030607@hlabs.spb.ru>
Message-ID: <200312231005.41130.aleaxit@yahoo.com>

On Tuesday 23 December 2003 12:09 pm, Dmitry Vasiliev wrote:
> Alex Martelli wrote:
> >>The main idea is to treating package as a program and run package
> >>initialization code from command line. The advantage is zipping all
> >>program modules in one zip archive and running the program from command
> >>line without unzipping it, like Java's jar's. But this idea need more
> >>thoughts however...
> >
> > Couldn't you use:
> >     python -c "import package_name"
> > for this purpose even today?
>
> python -c "import sys; sys.path.insert(0, 'module.zip'); import module"
>
> Seems ugly... :)

Well, if the zipfile isn't on your sys.path already, then you'd have to insert
it explicitly anyway -- surely, even if a switch "-p" to mean import existed,
you wouldn't want it to force python to snoop into EVERY zipfile around?!

PYTHONPATH=module.zip python -c "import module"

is one way you might express "insert into path and import" using a decent
shell (cygwin's bash on Windows, for example).  The proposed:

PYTHONPATH=module.zip python -p module

doesn't appear to offer a _major_ enhancement, just a very minor one,
in my personal opinion.


One potential advantage of -p might be to let it be present _together_
with one -c (even better might be to allow multiple -c, actually).  If you
want to run a "while foo.bep(): foo.zlup()" you can do it today only with
a shell that allows easy input of one multiline argument.  But again it
seems a rather marginal issue to me, personally.


Alex



From ncoghlan at iinet.net.au  Tue Dec 23 05:03:55 2003
From: ncoghlan at iinet.net.au (Nick Coghlan)
Date: Tue Dec 23 05:04:00 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <200312222213.hBMMD2m19449@c-24-5-183-134.client.comcast.net>
References: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.net>
	<1072131005.13676.9.camel@anthem>
	<200312222213.hBMMD2m19449@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE8130B.1080601@iinet.net.au>

[Barry]
>>Would it be better if object.__hash__() raised a NotImplementedError?

[Guido]
> It can't -- it's type.__hash__(object).

If I understand Barry's suggestion correctly, he means to keep 
object.__hash__, but have it raise a specific, meaningful error instead 
of making a potentially incorrect assumption as it does now.

E.g. (not real code, since this would behave differently in Py 2.3.3)

 >>> class Works(object): pass
 >>> class Breaks(object):
...   def __cmp__(): return 0 # All instances are created equal!
 >>> work = Works()
 >>> break = Breaks()
 >>> hash(work)
45632543
 >>> hash(break)
Traceback (most recent call last):
      <Traceback info>
   NotImplementedError:  Must explicitly define __hash__ for non-default 
comparisons

hash(work) is fine, since there is no __cmp__ or __eq__ override in 
Works, and hence object.__hash__ never gets called.

hash(break) raises the exception because of the existence of the (rather 
useless) __cmp__ function in the Breaks class.

Cheers,
Nick.

-- 
Nick Coghlan               |     Brisbane, Australia
Email: ncoghlan@email.com  | Mobile: +61 409 573 268


From martin at v.loewis.de  Tue Dec 23 04:33:53 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 23 05:13:45 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312222156.hBMLueA19333@c-24-5-183-134.client.comcast.net>
References: <200312221629.hBMGTV918147@c-24-5-183-134.client.comcast.net>	<20031222170257.GA31352@homer.gst.com>	<200312221737.hBMHbTh18276@c-24-5-183-134.client.comcast.net>	<m3d6ag3ckx.fsf@float.localdomain>	<200312221907.hBMJ7hg18485@c-24-5-183-134.client.comcast.net>	<m3vfo81vrs.fsf@float.localdomain>	<200312221959.hBMJxhY18697@c-24-5-183-134.client.comcast.net>
	<m3d6agzg1y.fsf@float.localdomain>
	<200312222156.hBMLueA19333@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE80C01.9080203@v.loewis.de>

Guido van Rossum wrote:
> I don't have the OpenBSD strchr.c source code online here
> so I'll stop speculating here...

It's this:

http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/string/index.c

I can't see anything wrong in it, and it hasn't significantly
changed in ages, either.

I'd be curious what p_name points to when the process crashes.

(gdb) p *p_name

Regards,
Martin


From martin at v.loewis.de  Tue Dec 23 03:51:01 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 23 05:48:17 2003
Subject: [Python-Dev] Python threads end up blocking signals
	in	subprocesses
In-Reply-To: <20031222221058.GD24252@unpythonic.net>
References: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>	<20031219012347.GB4692@unpythonic.net>	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>	<2mwu8rebcp.fsf@starship.python.net>	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>	<20031221230655.GA4417@unpythonic.net>	<3FE64412.3010500@v.loewis.de>	<20031222020728.GA15895@unpythonic.net>	<3FE65AF6.6010403@v.loewis.de>	<20031222140255.GH12586@unpythonic.net>
	<20031222221058.GD24252@unpythonic.net>
Message-ID: <3FE801F5.9010304@v.loewis.de>

Jeff Epler wrote:
> You could
> submit an interpretation request, but I think the committee would
> concur with my reading.

Hmm. This is what I now did: I submitted an interpretation request
to PASC:

Edition of Specification (Year): 2003

Defect code :  3. Clarification required

It is unclear whether calling system(3) invokes atfork handlers in a 
conforming implementation. system(3) specifies

"The environment of the executed command shall be as if a child process 
were created using fork(), and the child process invoked the sh utility 
using execl() as follows:"

In particular, usage of the word "environment" is confusing here. It may 
refer just to environment variables, however, the "Application usage" 
sections indicates that also signal handlers should be arranged as if 
the  process was created through fork() and execl(). This still makes 
not clear whether handlers installed through pthread_atfork() are invoked.

Action:

The description of system() should change to

  "system() behaves as if a new process was created using fork(),
   and the child process invoked the sh utility using execl() ..."

In addition, the application usage section should make it clear that 
atfork handlers are invoked.

Regards,
Martin


From martin at v.loewis.de  Tue Dec 23 04:01:58 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 23 08:43:12 2003
Subject: [Python-Dev] Python threads end up blocking signals
	in	subprocesses
In-Reply-To: <200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
References: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>	<20031219012347.GB4692@unpythonic.net>	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>	<2mwu8rebcp.fsf@starship.python.net>	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>	<20031221230655.GA4417@unpythonic.net>	<3FE64412.3010500@v.loewis.de>	<20031222020728.GA15895@unpythonic.net>	<3FE65AF6.6010403@v.loewis.de>	<20031222140255.GH12586@unpythonic.net>
	<20031222221058.GD24252@unpythonic.net>
	<200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE80486.9060807@v.loewis.de>

Guido van Rossum wrote:

> How hard would it be to reimplement our own system() and popen() using
> only POSIX calls, for POSIX systems?  I've always thought of these to
> be pretty simple combinations of fork() and exec(), with an assumption
> of a working /bin/sh.  

I would be concerned that we bypass magic that the system vendor put
into system(), which is essential for proper operation. For example,
on Linux, system() blocks SIGINT in the parent process while the
child is running. Also, the shell executable that system() uses
may not be /bin/sh.

OTOH, using the same underlying implementation on all systems makes
Python behave more predictable.

In the specific case, we would not even need pthread_atfork anymore,
as we now could invoke PyOS_AfterFork in the child ourselves.

Regards,
Martin


From jepler at unpythonic.net  Tue Dec 23 09:05:48 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Tue Dec 23 09:06:02 2003
Subject: [Python-Dev] Python threads end up blocking signals
	in	subprocesses
In-Reply-To: <3FE80486.9060807@v.loewis.de>
References: <2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
	<3FE65AF6.6010403@v.loewis.de>
	<20031222140255.GH12586@unpythonic.net>
	<20031222221058.GD24252@unpythonic.net>
	<200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net>
	<3FE80486.9060807@v.loewis.de>
Message-ID: <20031223140547.GA7907@unpythonic.net>

On Tue, Dec 23, 2003 at 10:01:58AM +0100, Martin v. Loewis wrote:
> I would be concerned that we bypass magic that the system vendor put
> into system(), which is essential for proper operation. For example,
> on Linux, system() blocks SIGINT in the parent process while the
> child is running. Also, the shell executable that system() uses
> may not be /bin/sh.

This behavior (blocking SIGINT and SIGQUIT in the parent) is part of the
specification of system():
    The system() function shall ignore the SIGINT and SIGQUIT signals,
    and shall block the SIGCHLD signal, while waiting for the command to
    terminate. If this might cause the application to miss a signal that
    would have killed it, then the application should examine the return
    value from system() and take whatever action is appropriate to the
    application if the command terminated due to receipt of a signal.

Jeff

From tim.one at comcast.net  Tue Dec 23 09:20:49 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Dec 23 09:20:53 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <3FE80C01.9080203@v.loewis.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>

[Guido]
>> I don't have the OpenBSD strchr.c source code online here
>> so I'll stop speculating here...

[Martin v. Loewis]
> It's this:
>
> http://www.openbsd.org/cgi-bin/cvsweb/src/lib/libc/string/index.c
>
> I can't see anything wrong in it, and it hasn't significantly
> changed in ages, either.

Looks fine to me too -- and it's a very simple function.  Google didn't turn
up any suggestion of strchr problems under OpenBSD either.

Hate to say it, but the pointer passed *to* strchr must be insane, and that
makes it more likely a Python, or platform compiler, bug.

> I'd be curious what p_name points to when the process crashes.
>
> (gdb) p *p_name


From kbk at shore.net  Tue Dec 23 10:10:54 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Dec 23 10:11:00 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net> (Tim
	Peters's message of "Tue, 23 Dec 2003 09:20:49 -0500")
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
Message-ID: <m34qvrwow1.fsf@float.localdomain>

"Tim Peters" <tim.one@comcast.net> writes:

> Hate to say it, but the pointer passed *to* strchr must be insane,
> and that makes it more likely a Python, or platform compiler, bug.

There are two calls to load_next() in import_module_ex().  The
segfault is occuring during the second call.

The code is somewhat pathological in that the callee, load_next(), is
modifying the caller's /parameters/ by changing the contents of name.

For some reason, the compiler emits code which makes a copy of
import_module_ex()'s parameters in the stack frame.  When load_next()
is called, the reference &name is the location in the
parameter area of the frame, but when name is tested in the while
loop, the copy in the local area of the frame is used.  Since this has
not been modified by load_next(), the fact that name has been set to
0x00 is missed.  load_next() gets called erroneously and passes a null
pointer to strchr.

I tried a volatile declaration, but no joy.  Adding a proper local,
mod_name, resolved the problem.
 
-- 
KBK

Index: Python/import.c
===================================================================
RCS file: /cvsroot/python/python/dist/src/Python/import.c,v
retrieving revision 2.225
diff -c -r2.225 import.c
*** Python/import.c	20 Nov 2003 01:44:58 -0000	2.225
--- Python/import.c	23 Dec 2003 14:56:40 -0000
***************
*** 1871,1876 ****
--- 1871,1877 ----
  		 PyObject *fromlist)
  {
  	char buf[MAXPATHLEN+1];
+ 	char *mod_name;
  	int buflen = 0;
  	PyObject *parent, *head, *next, *tail;
  
***************
*** 1878,1891 ****
  	if (parent == NULL)
  		return NULL;
  
! 	head = load_next(parent, Py_None, &name, buf, &buflen);
  	if (head == NULL)
  		return NULL;
  
  	tail = head;
  	Py_INCREF(tail);
! 	while (name) {
! 		next = load_next(tail, tail, &name, buf, &buflen);
  		Py_DECREF(tail);
  		if (next == NULL) {
  			Py_DECREF(head);
--- 1879,1893 ----
  	if (parent == NULL)
  		return NULL;
  
! 	mod_name = name;
! 	head = load_next(parent, Py_None, &mod_name, buf, &buflen);
  	if (head == NULL)
  		return NULL;
  
  	tail = head;
  	Py_INCREF(tail);
! 	while (mod_name) {
! 		next = load_next(tail, tail, &mod_name, buf, &buflen);
  		Py_DECREF(tail);
  		if (next == NULL) {
  			Py_DECREF(head);

From charleshixsn at earthlink.net  Tue Dec 23 10:54:46 2003
From: charleshixsn at earthlink.net (Charles Hixson)
Date: Tue Dec 23 11:00:33 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>	<20031216141746.GA3145@burma.localdomain>	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>	<1071714822.27808.88.camel@anthem>
	<20031218030409.GG473@frobozz>	<1071716900.27808.114.camel@anthem>
	<20031218031645.GH473@frobozz> <20031219213031.GD1246@wopr>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
Message-ID: <3FE86546.50003@earthlink.net>

Guido van Rossum wrote:

>> I like syntax that reads most important left-to-right, so what about
>> from MODULE import NAMES as RENAME searching HOW
>> or
>> import NAMES as RENAME from MODULE searching HOW
>>
>> searching could be 'absolute' 'relative' or any of the other
>> suggested words.  If you wanted to get fancy it could be a list, if
>> it isn't a list people who truly care could cascade try/except on
>> ImportError.
>>
>> -jackdied
>>
>> ps, I like the second version better but it is less like the current 
>> version,
>> which is a drawback.
>>   
>
>
> This got several encouraging responses, so I have to consider it.
>
> Unfortunately it looks like 'searching' would have to become a
> reserved word; if we use the same trick we used to avoid making 'as' a
> reserved word, that would be too confusing to decode in the code
> generator.  The HOW could be just an identifier then.
>
> I'm not sure how well it'll read in actual examples.  Let's try.
> Adapting an example from Opal that was used here before...
>
>  from COGS import generate searching relative
>
> To me, that looks a lot like someone left the commas out of
>
>  from COGS import generate, searching, relative
>
> In general I'm not a big fan of "wordy" clauses like this -- they work
> better in languages like SQL or COBOL because those are case-sensitive
> so you can use a difference in capitalization there; there the former
> would be
>
>  FROM COGS IMPORT generate SEARCHING RELATIVE
>
> (using imaginary syntax) while the latter would be
>
>  FROM COGS IMPORT generate, searching, relative
>
> while in Python the long sequence of lowercase words becomes a blur.
>
> So if the two alternatives are simply 'searching absolute' and
> 'searching relative', with the default being 'searching absolute' in
> Python 3.0 (but the current ambiguous search in Python 2.x) I'd still
> prefer making the distinction with "from ...MODULE" vs. "from MODULE"
> rather than with a searching clause.  And I still prefer the
> single-dot syntax over three dots, because it can be used to spell
> importing from the parent or grandparent package explicitly.
>
> A separate suggestion is to switch from "from X import Y" to "import Y
> from X".  This seems a gratuitous change even if it reads or edits
> marginally better.  Now's not the time to tinker with marginal stuff
> -- that should've been done 10 years ago.
>
> --Guido van Rossum (home page: http://www.python.org/~guido/)
>
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: 
> http://mail.python.org/mailman/options/python-dev/charleshixsn%40earthlink.net 
>
>
>  
>
Not necessarily.  import could be a special function that doesn't need 
parentheses around it's arguments or comma separators.  That could be 
syntax sugar for:
import (names, namedAs, fromModule, searchmethod) or
import (names =[theNames],  namedAd=[localNames], fromModule=fromModule, 
searchMethod=upFromCurrent)

OTOH, to me it looks more like Smalltalk than like Python.  An explicit 
import function without special sugar looks more Pythonic.  (But that 
would break backward compatibility.)
Perhaps the current method could be deprecated, and an import function 
be the replacement?



From guido at python.org  Tue Dec 23 11:11:40 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 23 11:11:49 2003
Subject: [Python-Dev] Python threads end up blocking signals in
	subprocesses
In-Reply-To: Your message of "Tue, 23 Dec 2003 10:01:58 +0100."
	<3FE80486.9060807@v.loewis.de> 
References: <m3smjhu8h1.fsf@mira.informatik.hu-berlin.de>
	<20031219012347.GB4692@unpythonic.net>
	<m3smjfztfq.fsf@mira.informatik.hu-berlin.de>
	<2mwu8rebcp.fsf@starship.python.net>
	<m3smjea2ji.fsf@mira.informatik.hu-berlin.de>
	<20031221230655.GA4417@unpythonic.net>
	<3FE64412.3010500@v.loewis.de>
	<20031222020728.GA15895@unpythonic.net>
	<3FE65AF6.6010403@v.loewis.de>
	<20031222140255.GH12586@unpythonic.net>
	<20031222221058.GD24252@unpythonic.net>
	<200312222223.hBMMNBn19477@c-24-5-183-134.client.comcast.net> 
	<3FE80486.9060807@v.loewis.de> 
Message-ID: <200312231611.hBNGBek20630@c-24-5-183-134.client.comcast.net>

> > How hard would it be to reimplement our own system() and popen() using
> > only POSIX calls, for POSIX systems?  I've always thought of these to
> > be pretty simple combinations of fork() and exec(), with an assumption
> > of a working /bin/sh.  
> 
> I would be concerned that we bypass magic that the system vendor put
> into system(), which is essential for proper operation. For example,
> on Linux, system() blocks SIGINT in the parent process while the
> child is running. Also, the shell executable that system() uses
> may not be /bin/sh.

In practice, I think we can do as well as vendors -- there really
isn't that much to it.  Systems where /bin/sh doesn't exist will have
other problems...  And we get to do it right when called from a
thread.

> OTOH, using the same underlying implementation on all systems makes
> Python behave more predictable.
> 
> In the specific case, we would not even need pthread_atfork anymore,
> as we now could invoke PyOS_AfterFork in the child ourselves.

Right!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 23 11:20:08 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 23 11:20:14 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Tue, 23 Dec 2003 10:10:54 EST."
	<m34qvrwow1.fsf@float.localdomain> 
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>  
	<m34qvrwow1.fsf@float.localdomain> 
Message-ID: <200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>

> There are two calls to load_next() in import_module_ex().  The
> segfault is occuring during the second call.
> 
> The code is somewhat pathological in that the callee, load_next(), is
> modifying the caller's /parameters/ by changing the contents of name.
> 
> For some reason, the compiler emits code which makes a copy of
> import_module_ex()'s parameters in the stack frame.  When load_next()
> is called, the reference &name is the location in the
> parameter area of the frame, but when name is tested in the while
> loop, the copy in the local area of the frame is used.  Since this has
> not been modified by load_next(), the fact that name has been set to
> 0x00 is missed.  load_next() gets called erroneously and passes a null
> pointer to strchr.
> 
> I tried a volatile declaration, but no joy.  Adding a proper local,
> mod_name, resolved the problem.

Wow.  Thanks for the analysis.  But this is clearly a compiler bug.
Where do we report that?  And why would it be unique to OpenBSD?

In the mean time, Kurt, please check in your fix -- it can't hurt, and
we might as well avoid the pain for the next person who wants to build
a debugging Python.  The fix could use a comment referring to a
compiler bug, to keep the next maintainer from unfixing it. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From barry at python.org  Tue Dec 23 11:47:02 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Dec 23 11:47:09 2003
Subject: [Python-Dev] getting rid of default object.__hash__ (SF 660098)
In-Reply-To: <3FE8130B.1080601@iinet.net.au>
References: <200312222104.hBML4Bc18950@c-24-5-183-134.client.comcast.net>
	<1072131005.13676.9.camel@anthem>
	<200312222213.hBMMD2m19449@c-24-5-183-134.client.comcast.net>
	<3FE8130B.1080601@iinet.net.au>
Message-ID: <1072198021.31242.8.camel@geddy>

On Tue, 2003-12-23 at 05:03, Nick Coghlan wrote:

> If I understand Barry's suggestion correctly, he means to keep 
> object.__hash__, but have it raise a specific, meaningful error instead 
> of making a potentially incorrect assumption as it does now.

Right!  Thanks Nick.
-Barry



From klm at zope.com  Tue Dec 23 12:03:00 2003
From: klm at zope.com (Ken Manheimer)
Date: Tue Dec 23 12:11:50 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <Pine.LNX.4.44.0312220945540.12124-100000@whitebread.org>
Message-ID: <Pine.LNX.4.44.0312231136470.28510-100000@slab.zope.com>

On Mon, 22 Dec 2003, Devin wrote:

> The first part stems from Guido's idea about using a '.' at the beginning 
> of a relative import.  It seemed a bit weird at first, but I eventually 
> became very comfortable with the idea after thinking about it in respect 
> to absolute imports.  I'll explain what I mean.
> 
> If you currently want to import a module 'a.b.c.d', you write:
> 
>     import a.b.c.d
> 
> Now, if you're in the module 'a.b', you could still write:
> 
>     import a.b.c.d
> 
> ... but the 'a.b' part is redundant because you're already in the module 
> 'a.b'.  If you take away 'a.b', then you end up with:
> 
>     import .c.d

I also look at it this way, and find it both intuitive and mnemonic.
My model is slighly different than yours, though, in a way that
simplifies the expression for going up the relative chain.  I'll
explain...

> This flowed well in my head, and looks natural.  However, the suggestion
> about '.' or '..' referring to the parent looks _ugly and unnatural_.
> 
> Let's say '.' was used to spell "parent module", and let's assume that 
> the module 'a.b.c.d' wants to import 'a.b.e'.  The absolute import would 
> be spelled:
> 
>     import a.b.e
> 
> ... and the relative import would be spelled:
> 
>     import .....e # .[parent].[parent].e
> 
> Yuck! (no offense :)

Truly, yuck.  But in my model, the leading '.' dot, itself, stands for
the containing package, and '..' stands for the containing package's
package, and so forth:

  import ..e   #  [parent-of-parent].e

I don't think it's yucky at all, this way.  (I also don't think it's
un-pythonic - python uses punctuation all over to denote type literals
('{'/'}'), cues which delineate statement suites (':'), and so forth.)
In fact, i think it's a lot more mnemonic and encompassing than the
alternatives that use words (though i haven't looked at all the
alternatives very carefully yet).

Even using the leading '.' syntax, it would be good to have a specific
identifier for the containing package, realized in each module and
respected as a special identifier by the package machinery.  I like
'__pkg__':

  import __pkg__.sibling                # == 'import .sibling'

and

  import __pkg__.__pkg__.uncle          # == 'import ..uncle'
  import __pkg__.__pkg__.uncle.cousin   # == 'import ..uncle.cousin'

I think the leading '.' dot forms are a lot nicer, but the presence of
a module named '__pkg__' would be useful for programmatically mucking
navigating the package structure.

Ken Manheimer
klm@zope.com


From mwh at python.net  Tue Dec 23 12:24:27 2003
From: mwh at python.net (Michael Hudson)
Date: Tue Dec 23 12:24:31 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <3FE86546.50003@earthlink.net> (Charles Hixson's message of
	"Tue, 23 Dec 2003 07:54:46 -0800")
References: <Pine.LNX.4.44.0312151333110.10960-100000@slab.zope.com>
	<200312151954.hBFJskd18127@c-24-5-183-134.client.comcast.net>
	<20031216141746.GA3145@burma.localdomain>
	<200312161826.hBGIQxg20322@c-24-5-183-134.client.comcast.net>
	<1071714822.27808.88.camel@anthem> <20031218030409.GG473@frobozz>
	<1071716900.27808.114.camel@anthem> <20031218031645.GH473@frobozz>
	<20031219213031.GD1246@wopr>
	<200312220448.hBM4m1K17301@c-24-5-183-134.client.comcast.net>
	<3FE86546.50003@earthlink.net>
Message-ID: <2mad5jcur8.fsf@starship.python.net>

Charles Hixson <charleshixsn@earthlink.net> writes:

> Not necessarily.  import could be a special function that doesn't need
> parentheses around it's arguments or comma separators.  That could be
> syntax sugar for:
> import (names, namedAs, fromModule, searchmethod) or
> import (names =[theNames],  namedAd=[localNames],
> fromModule=fromModule, searchMethod=upFromCurrent)
>
> OTOH, to me it looks more like Smalltalk than like Python.  An
> explicit import function without special sugar looks more Pythonic.
> (But that would break backward compatibility.)
> Perhaps the current method could be deprecated, and an import function
> be the replacement?

I really don't think you could make that work (unless you want to
write

math = import("math")

everywhere, which you can almost do today...)

Cheers,
mwh

-- 
  Ya, ya, ya, except ... if I were built out of KSR chips, I'd
  be running at 25 or 50 MHz, and would be wrong about ALMOST
  EVERYTHING almost ALL THE TIME just due to being a computer!
                                              -- Tim Peters, 30 Apr 97

From devin at whitebread.org  Tue Dec 23 13:45:18 2003
From: devin at whitebread.org (Devin)
Date: Tue Dec 23 13:50:12 2003
Subject: [Python-Dev] Relative import
In-Reply-To: <Pine.LNX.4.44.0312231136470.28510-100000@slab.zope.com>
Message-ID: <Pine.LNX.4.44.0312230921500.14760-100000@whitebread.org>

On Tue, 23 Dec 2003, Ken Manheimer wrote:

> > This flowed well in my head, and looks natural.  However, the suggestion
> > about '.' or '..' referring to the parent looks _ugly and unnatural_.
> > 
> > Let's say '.' was used to spell "parent module", and let's assume that 
> > the module 'a.b.c.d' wants to import 'a.b.e'.  The absolute import would 
> > be spelled:
> > 
> >     import a.b.e
> > 
> > ... and the relative import would be spelled:
> > 
> >     import .....e # .[parent].[parent].e
> > 
> > Yuck! (no offense :)
> 
> Truly, yuck.  But in my model, the leading '.' dot, itself, stands for
> the containing package, and '..' stands for the containing package's
> package, and so forth:
> 
>   import ..e   #  [parent-of-parent].e

I can't quite get used to the idea of '.' having two different meanings
depending on how many '.'s are found next to each other in an import
statement.  I believe that the notation above _is_ concise, but _is not_
very intuitive ...

> Even using the leading '.' syntax, it would be good to have a specific
> identifier for the containing package, realized in each module and
> respected as a special identifier by the package machinery.  I like
> '__pkg__':
> 
>   import __pkg__.sibling                # == 'import .sibling'
> 
> and
> 
>   import __pkg__.__pkg__.uncle          # == 'import ..uncle'
>   import __pkg__.__pkg__.uncle.cousin   # == 'import ..uncle.cousin'

... while the notation here _is_ intuitive (to a python programmer), but
_is not_ concise.  I still favor this syntax to the former syntax.

I like '__pkg__' better than my initial suggestion ('__parent__').  It's
more aesthetically pleasing. :)

-- 
Devin
devin@whitebread.org
http://www.whitebread.org/


From chrisandannreedy at comcast.net  Tue Dec 23 15:03:37 2003
From: chrisandannreedy at comcast.net (Chris and Ann Reedy)
Date: Tue Dec 23 15:03:43 2003
Subject: [Python-Dev] Another Strategy for Relative Import
Message-ID: <3FE89F99.8BC0D067@comcast.net>


Ken Manheimer wrote:
> 
> Even using the leading '.' syntax, it would be good to have a specific
> identifier for the containing package, realized in each module and
> respected as a special identifier by the package machinery.  I like
> '__pkg__':
> 
>   import __pkg__.sibling                # == 'import .sibling'
> 
> and
> 
>   import __pkg__.__pkg__.uncle          # == 'import ..uncle'
>   import __pkg__.__pkg__.uncle.cousin   # == 'import ..uncle.cousin'
> 
> I think the leading '.' dot forms are a lot nicer, but the presence of
> a module named '__pkg__' would be useful for programmatically mucking
> navigating the package structure.

Delurking:

Knowing that the statement:

  import foo.bar

has the effect of setting the variable foo to reference package foo,
even 
if the variable foo already references something else, I was wondering
if it would make sense to changes the semantics of import so that if the
variable foo is already assigned and is a package (or maybe a module in
some cases), then importing foo.bar would have the effect of importing
the module or package bar within foo, using foo.__path__ to search for the
module bar and foo.__name__ + '.bar' as the name of the module.

It seems to me that such an approach might be used in a number of cases:

1. If there is a standard name for the parent of a package, e.g. __pkg__,
then

  import__pkg__.bar

just works.

2. This approach allows the following kind of thing to work:

if <something>:
  import xyz as util
else:
  import xyz_old as util

import util.bar

3. This could be used to clean up the problem with os.path. The idea here
is that since initialization of os creates the attribute os.path which
references a module, then:

  import os.path

would have the effect of importing os and then noting that os.path
exists and
is a module, so no further work is required.

I haven't (yet?) attempted to work out whether this could be used to
handle all
the use cases for relative import.

  Chris

P.S. I've been lurking on this list for a about six months. I just moved between
coasts (US) and things are still a mess. I would like to get more
involved with
Python. I'll send a message in the near future introducing myself.

From barry at barrys-emacs.org  Wed Dec 24 12:37:06 2003
From: barry at barrys-emacs.org (Barry Scott)
Date: Wed Dec 24 12:37:15 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.ne
 t>
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
Message-ID: <6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private>

At 23-12-2003 16:20, Guido van Rossum wrote:
>Wow.  Thanks for the analysis.  But this is clearly a compiler bug.
>Where do we report that?  And why would it be unique to OpenBSD?

Because the OpenBSD folks have their own code generator to improve the
security of OpenBSD. They code gen to make it impossible/improbable
to use a stack overflow attack. I found they fix the compiler quickly
once you tell then on the OpenBSD dev list whats wrong with a recipe to
reproduce.

You may find that they will not fix for 3.3 as 3.4 is out and the
tool chain moved from a.out to ELF in 3.4.

Barry



From guido at python.org  Wed Dec 24 12:47:22 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 24 12:47:27 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Wed, 24 Dec 2003 17:37:06 GMT."
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private> 
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net> 
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private> 
Message-ID: <200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>

> >Wow.  Thanks for the analysis.  But this is clearly a compiler bug.
> >Where do we report that?  And why would it be unique to OpenBSD?
> 
> Because the OpenBSD folks have their own code generator to improve the
> security of OpenBSD. They code gen to make it impossible/improbable
> to use a stack overflow attack. I found they fix the compiler quickly
> once you tell then on the OpenBSD dev list whats wrong with a recipe to
> reproduce.

Sigh.  So much for security. :-(

> You may find that they will not fix for 3.3 as 3.4 is out and the
> tool chain moved from a.out to ELF in 3.4.

Double sigh.  Strange approach to security.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec 24 13:12:39 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 24 13:12:45 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: Your message of "Wed, 24 Dec 2003 09:47:22 PST."
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net> 
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private> 
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net> 
Message-ID: <200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.net>

I've tried to look for a way to report OpenBSD bugs, but it appears
the only way to report one is by using their sendbug command, which
assumes you've got OpenBSD installed.  Would someone here be so kind
as to report the bug we've found?  A pointer to Kurt's last message in
this thread should be a good piece of evidence.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From kbk at shore.net  Wed Dec 24 15:22:07 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Dec 24 15:22:13 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Wed, 24 Dec 2003 10:12:39 -0800")
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private>
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>
	<200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.net>
Message-ID: <m3vfo66k5s.fsf@float.localdomain>

Guido van Rossum <guido@python.org> writes:

> I've tried to look for a way to report OpenBSD bugs, but it appears
> the only way to report one is by using their sendbug command, which
> assumes you've got OpenBSD installed.  Would someone here be so kind
> as to report the bug we've found?  A pointer to Kurt's last message in
> this thread should be a good piece of evidence.

I'm not ignoring you, shore.net is delaying inbound messages over 24h
and that's slowing me down.

The problem is related to the ProPolice "stack smash" protector.  It
does exactly what I saw: It copies any pointer arguments to an area in
the frame below the locally declared buffers.  The error appears to be
in referencing the original location in the arguments when making the
call to load_next().

One of the ProPolice ideas is that a pointer (especially a pointer to
a function) can be hacked by a buffer overrun to point to rogue code.

www.trl.ibm.com/projects/security/ssp/

The implementation appears to be copying all argument pointers for
good measure.

I'm looking into this to see if there is a better way to fix the 
import code.  Making a local copy of the pointer fixes the segfault but
it's not clear to me yet whether that's a solid fix for all uses of
this code in Python.

As far as reporting the bug, I can do that once I localize the problem
in gcc and see whether it's been fixed.  A quick search of the OpenBSD
bug database came up dry, but maybe I don't know how to phrase the
question yet.

-- 
KBK

From kbk at shore.net  Wed Dec 24 15:45:16 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Wed Dec 24 15:45:29 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Wed, 24 Dec 2003 09:47:22 -0800")
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private>
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>
Message-ID: <m3r7yu6j37.fsf@float.localdomain>

Guido van Rossum <guido@python.org> writes:

> Double sigh.  Strange approach to security.

Their overall approach is outstanding.  They understand that the right
thing to do is fix insecure code and they audit their base distro
extensively.  I think they have around a dozen people, including some
security professionals, on the audit team.

However, it's not possible to audit all the code in the ports, so they
have implemented several defensive measures which make it more
difficult to mount an attack:

1. ProPolice - anti-stacksmash methods.  Sets a canary after the
   return address, moves local and argument buffers just below that,
   moves other locals below the buffers, and copies argument pointers
   into the latter area.  If a buffer gets overrun, it nukes the canary.

2. W^X - memory which can be written by an app can't be executed, and
   vice-versa, assuming MMU support.

3. Non-executable stack @ 3.2 (note that this can cause problems with
   some functional languages)

4. Greatly reduced use of SUID/GUID binaries.  Use of chroot and
   priviledge separation.

It's not surprising there are some residual bugs in the implementation,
at least at 3.3.  I haven't moved my box to 3.4 yet.

-- 
KBK

From andymac at bullseye.apana.org.au  Thu Dec 25 08:40:35 2003
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Thu Dec 25 08:48:31 2003
Subject: [Python-Dev] status of 2.3 branch for maintenance checkins
Message-ID: <20031226003609.O1176@bullseye.apana.org.au>

I don't recall any advice as to whether the 2.3 branch is again open for
checkins post 2.3 - Is Jack still sorting out the Mac release?

Regards,
Andrew.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia

From anthony at interlink.com.au  Thu Dec 25 12:47:43 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Thu Dec 25 12:48:04 2003
Subject: [Python-Dev] status of 2.3 branch for maintenance checkins 
In-Reply-To: <20031226003609.O1176@bullseye.apana.org.au> 
Message-ID: <200312251747.hBPHlham032180@localhost.localdomain>


>>> Andrew MacIntyre wrote
> I don't recall any advice as to whether the 2.3 branch is again open for
> checkins post 2.3 - Is Jack still sorting out the Mac release?

It's still closed, until Jack says otherwise.

Anthony

-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From jeremy at alum.mit.edu  Fri Dec 26 14:06:31 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Dec 26 14:10:04 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
	listobject.c,2.172,2.173
In-Reply-To: <E1AZxGl-0008So-00@sc8-pr-cvs1.sourceforge.net>
References: <E1AZxGl-0008So-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <1072465591.31901.71.camel@localhost.localdomain>

On Fri, 2003-12-26 at 14:05, jhylton@users.sourceforge.net wrote:
> Update of /cvsroot/python/python/dist/src/Objects
> In directory sc8-pr-cvs1:/tmp/cvs-serv32521
> 
> Modified Files:
> 	listobject.c 
> Log Message:
> Revert previous two checkins to repair test failure.
> 
> The special-case code that was removed could return a value indicating
> success but leave an exception set.  test_fileinput failed in a debug
> build as a result.

If there's some value to the optimization I just removed, let's discuss
it on python-dev.  I could have fixed the new code, but it seems a lot
of hairy new code with magic constants that only covered a few corner
cases.  The checkin message pointed to a python-list discussion that
didn't have much real evidence in it.

Jeremy



From jeremy at alum.mit.edu  Fri Dec 26 15:22:52 2003
From: jeremy at alum.mit.edu (Jeremy Hylton)
Date: Fri Dec 26 15:26:33 2003
Subject: [Python-Dev] test_unicode_file fails on Linux
Message-ID: <1072470171.31901.74.camel@localhost.localdomain>

The utime() call is failing for one of the Unicode file names.  

build> ./python ../Lib/test/test_unicode_file.py
test_directories (__main__.TestUnicodeFiles) ... ok
test_equivalent_files (__main__.TestUnicodeFiles) ... ok
test_single_files (__main__.TestUnicodeFiles) ...
'@test-\xc3\xa0\xc3\xb2'
'@test-\xc3\xa0\xc3\xb2'
u'@test-\xe0\xf2'
ERROR
 
======================================================================
ERROR: test_single_files (__main__.TestUnicodeFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../Lib/test/test_unicode_file.py", line 143, in
test_single_files
    self._test_single(TESTFN_UNICODE)
  File "../Lib/test/test_unicode_file.py", line 117, in _test_single
    self._do_single(filename)
  File "../Lib/test/test_unicode_file.py", line 33, in _do_single
    os.utime(filename, None)
UnicodeEncodeError: 'ascii' codec can't encode characters in position
6-7: ordinal not in range(128)

I'm running on a RH9 system.

I see that the code hasn't changed since the beginning of the month, so
I'm surprised to only discover it now.  I can't recall the last time I
did a full test run, so I'm not sure how long it has been failing.

Jeremy



From skip at pobox.com  Fri Dec 26 16:17:22 2003
From: skip at pobox.com (Skip Montanaro)
Date: Fri Dec 26 16:17:31 2003
Subject: [Python-Dev] test_unicode_file fails on Linux
In-Reply-To: <1072470171.31901.74.camel@localhost.localdomain>
References: <1072470171.31901.74.camel@localhost.localdomain>
Message-ID: <16364.42338.127559.952496@montanaro.dyndns.org>


    Jeremy> The utime() call is failing for one of the Unicode file names.  

Been failing for me for awhile (Mac OS X), but in a bit different fashion:

    % ./python.exe ../Lib/test/regrtest.py -v test_unicode_file
    test_unicode_file
    test_directories (test.test_unicode_file.TestUnicodeFiles) ... FAIL
    test_equivalent_files (test.test_unicode_file.TestUnicodeFiles) ... ok
    test_single_files (test.test_unicode_file.TestUnicodeFiles) ... FAIL

    ======================================================================
    FAIL: test_directories (test.test_unicode_file.TestUnicodeFiles)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/skip/src/python/head/dist/src/Lib/test/test_unicode_file.py", line 155, in test_directories
        self._do_directory(TESTFN_ENCODED+ext, TESTFN_ENCODED+ext, os.getcwd)
      File "/Users/skip/src/python/head/dist/src/Lib/test/test_unicode_file.py", line 103, in _do_directory
        make_name)
    AssertionError: '@test-a\xcc\x80o\xcc\x80.dir' != '@test-\xc3\xa0\xc3\xb2.dir'

    ======================================================================
    FAIL: test_single_files (test.test_unicode_file.TestUnicodeFiles)
    ----------------------------------------------------------------------
    Traceback (most recent call last):
      File "/Users/skip/src/python/head/dist/src/Lib/test/test_unicode_file.py", line 141, in test_single_files
        self._test_single(TESTFN_ENCODED)
      File "/Users/skip/src/python/head/dist/src/Lib/test/test_unicode_file.py", line 116, in _test_single
        self._do_single(filename)
      File "/Users/skip/src/python/head/dist/src/Lib/test/test_unicode_file.py", line 41, in _do_single
        self.failUnless(base in os.listdir(path))
    AssertionError

    ----------------------------------------------------------------------
    Ran 3 tests in 0.031s

    FAILED (failures=2)
    test test_unicode_file failed -- errors occurred; run in verbose mode for details
    1 test failed:
        test_unicode_file

I thought this was a known issue (Unicode normalization differences?), so
didn't bother to report it.

Skip

From Jack.Jansen at cwi.nl  Fri Dec 26 16:58:33 2003
From: Jack.Jansen at cwi.nl (Jack Jansen)
Date: Fri Dec 26 16:58:37 2003
Subject: [Python-Dev] status of 2.3 branch for maintenance checkins
In-Reply-To: <20031226003609.O1176@bullseye.apana.org.au>
References: <20031226003609.O1176@bullseye.apana.org.au>
Message-ID: <A611A218-37EE-11D8-AAC8-000A27B19B96@cwi.nl>


On 25-dec-03, at 14:40, Andrew MacIntyre wrote:

> I don't recall any advice as to whether the 2.3 branch is again open 
> for
> checkins post 2.3 - Is Jack still sorting out the Mac release?

Still busy. I expect to be done this wweekend, but I'll send a note here
when I'm done.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From tim.one at comcast.net  Fri Dec 26 17:01:32 2003
From: tim.one at comcast.net (Tim Peters)
Date: Fri Dec 26 17:01:42 2003
Subject: [Python-Dev] test_unicode_file fails on Linux
In-Reply-To: <1072470171.31901.74.camel@localhost.localdomain>
Message-ID: <LNBBLJKPBEHFEDALKOLCAEGPIAAB.tim.one@comcast.net>

[Jeremy Hylton]
> The utime() call is failing for one of the Unicode file names.
>
> build> ./python ../Lib/test/test_unicode_file.py
> test_directories (__main__.TestUnicodeFiles) ... ok
> test_equivalent_files (__main__.TestUnicodeFiles) ... ok
> test_single_files (__main__.TestUnicodeFiles) ...
> '@test-\xc3\xa0\xc3\xb2'
> '@test-\xc3\xa0\xc3\xb2'
> u'@test-\xe0\xf2'
> ERROR
>
> ======================================================================
> ERROR: test_single_files (__main__.TestUnicodeFiles)
> ----------------------------------------------------------------------
> Traceback (most recent call last):
>   File "../Lib/test/test_unicode_file.py", line 143, in
> test_single_files
>     self._test_single(TESTFN_UNICODE)
>   File "../Lib/test/test_unicode_file.py", line 117, in _test_single
>     self._do_single(filename)
>   File "../Lib/test/test_unicode_file.py", line 33, in _do_single
>     os.utime(filename, None)
> UnicodeEncodeError: 'ascii' codec can't encode characters in position
> 6-7: ordinal not in range(128)
>
> I'm running on a RH9 system.
>
> I see that the code hasn't changed since the beginning of the month,
> so I'm surprised to only discover it now.  I can't recall the last
> time I did a full test run, so I'm not sure how long it has been
> failing.

It's been failing "quite a while" on Win98SE too, in the same way.  I
believe MarkH knows (or should know) about it already, but he's on vacation
now:

C:\Code\python\PCbuild>python ../lib/test/test_unicode_file.py
test_directories (__main__.TestUnicodeFiles) ... ok
test_equivalent_files (__main__.TestUnicodeFiles) ... ok
test_single_files (__main__.TestUnicodeFiles) ... ERROR

======================================================================
ERROR: test_single_files (__main__.TestUnicodeFiles)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 142, in test_single_files
    self._test_single(TESTFN_UNICODE)
  File "../lib/test/test_unicode_file.py", line 116, in _test_single
    self._do_single(filename)
  File "../lib/test/test_unicode_file.py", line 32, in _do_single
    os.utime(filename, None)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-7:
ordinal not in range(128)

----------------------------------------------------------------------
Ran 3 tests in 0.160s

FAILED (errors=1)
Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 171, in ?
    test_main()
  File "../lib/test/test_unicode_file.py", line 168, in test_main
    run_suite(suite)
  File "C:\CODE\PYTHON\lib\test\test_support.py", line 275, in run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "../lib/test/test_unicode_file.py", line 142, in test_single_files
    self._test_single(TESTFN_UNICODE)
  File "../lib/test/test_unicode_file.py", line 116, in _test_single
    self._do_single(filename)
  File "../lib/test/test_unicode_file.py", line 32, in _do_single
    os.utime(filename, None)
UnicodeEncodeError: 'ascii' codec can't encode characters in position 6-7:
ordinal not in range(128)


test_urllib2 is also failing on Win98SE, and that's more recent:

C:\Code\python\PCbuild>python ../lib/test/test_urllib2.py
test_trivial (__main__.TrivialTests) ... ok
test_handled (__main__.OpenerDirectorTests) ... ok
test_handler_order (__main__.OpenerDirectorTests) ... ok
test_http_error (__main__.OpenerDirectorTests) ... ok
test_processors (__main__.OpenerDirectorTests) ... ok
test_raise (__main__.OpenerDirectorTests) ... ok
test_errors (__main__.HandlerTests) ... ok
test_file (__main__.HandlerTests) ... ERROR
test_ftp (__main__.HandlerTests) ... ok
test_http (__main__.HandlerTests) ... ok
test_redirect (__main__.HandlerTests) ... ok
test_build_opener (__main__.MiscTests) ... ok

======================================================================
ERROR: test_file (__main__.HandlerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "../lib/test/test_urllib2.py", line 345, in test_file
    r = h.file_open(Request(url))
  File "C:\CODE\PYTHON\lib\urllib2.py", line 1058, in file_open
    return self.open_local_file(req)
  File "C:\CODE\PYTHON\lib\urllib2.py", line 1073, in open_local_file
    stats = os.stat(localfile)
OSError: [Errno 2] No such file or directory: '\\test.txt'

----------------------------------------------------------------------
Ran 12 tests in 0.170s

FAILED (errors=1)
Traceback (most recent call last):
  File "../lib/test/test_urllib2.py", line 628, in ?
    test_main(verbose=True)
  File "../lib/test/test_urllib2.py", line 624, in test_main
    MiscTests,
  File "C:\CODE\PYTHON\lib\test\test_support.py", line 290, in run_unittest
    run_suite(suite, testclass)
  File "C:\CODE\PYTHON\lib\test\test_support.py", line 275, in run_suite
    raise TestFailed(err)
test.test_support.TestFailed: Traceback (most recent call last):
  File "../lib/test/test_urllib2.py", line 345, in test_file
    r = h.file_open(Request(url))
  File "C:\CODE\PYTHON\lib\urllib2.py", line 1058, in file_open
    return self.open_local_file(req)
  File "C:\CODE\PYTHON\lib\urllib2.py", line 1073, in open_local_file
    stats = os.stat(localfile)
OSError: [Errno 2] No such file or directory: '\\test.txt'


test_bsddb3 is in such bad shape on Win98SE now I don't know where to begin.

The compiler warnings about mixing signed and unsigned in comparisons in
_sre.c also persist.

Other than that, everything's great.


From sjoerd at acm.org  Fri Dec 26 15:39:00 2003
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Fri Dec 26 17:39:55 2003
Subject: [Python-Dev] Fixes for imageop module
Message-ID: <3FEC9C64.5080805@acm.org>

I have some fixes for the imageop module to make it work on systems with 
a different byte-order than IRIX (e.g. Linux).  Shall I check them in or 
is it not worth it for such an old module?  Anything else that needs to 
be changed when I check this in?
The fixes of course will result in different results on Linux than 
before, so that's the main reason for asking.

-- 
Sjoerd Mullender <sjoerd@acm.org>

From guido at python.org  Fri Dec 26 17:49:43 2003
From: guido at python.org (Guido van Rossum)
Date: Fri Dec 26 17:49:49 2003
Subject: [Python-Dev] Fixes for imageop module
In-Reply-To: Your message of "Fri, 26 Dec 2003 21:39:00 +0100."
	<3FEC9C64.5080805@acm.org> 
References: <3FEC9C64.5080805@acm.org> 
Message-ID: <200312262249.hBQMnhf24306@c-24-5-183-134.client.comcast.net>

> I have some fixes for the imageop module to make it work on systems with 
> a different byte-order than IRIX (e.g. Linux).  Shall I check them in or 
> is it not worth it for such an old module?  Anything else that needs to 
> be changed when I check this in?
> The fixes of course will result in different results on Linux than 
> before, so that's the main reason for asking.

I don't see imageop in the list of deprecated modules in PEP 4, and
apparently *you* are still using it, so as long as you don't mind
being potentially the sole maintainer and user, I don't mind these
being in the distribution.

What do you mean by different results on Linux?  Was this module
previously doing something bogus there?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From sjoerd at acm.org  Fri Dec 26 18:09:39 2003
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Fri Dec 26 18:09:44 2003
Subject: [Python-Dev] Fixes for imageop module
In-Reply-To: <200312262249.hBQMnhf24306@c-24-5-183-134.client.comcast.net>
References: <3FEC9C64.5080805@acm.org>
	<200312262249.hBQMnhf24306@c-24-5-183-134.client.comcast.net>
Message-ID: <3FECBFB3.7090906@acm.org>

Guido van Rossum wrote:
>>I have some fixes for the imageop module to make it work on systems with 
>>a different byte-order than IRIX (e.g. Linux).  Shall I check them in or 
>>is it not worth it for such an old module?  Anything else that needs to 
>>be changed when I check this in?
>>The fixes of course will result in different results on Linux than 
>>before, so that's the main reason for asking.
> 
> 
> I don't see imageop in the list of deprecated modules in PEP 4, and
> apparently *you* are still using it, so as long as you don't mind
> being potentially the sole maintainer and user, I don't mind these
> being in the distribution.
> 
> What do you mean by different results on Linux?  Was this module
> previously doing something bogus there?
> 
> --Guido van Rossum (home page: http://www.python.org/~guido/)

The main difference is:
***************
*** 570,577 ****
   		r = (r<<5) | (r<<3) | (r>>1);
   		g = (g<<5) | (g<<3) | (g>>1);
   		b = (b<<6) | (b<<4) | (b<<2) | b;
! 		nvalue = r | (g<<8) | (b<<16);
! 		*ncp++ = nvalue;
   	}
   	return rv;
   }
--- 560,569 ----
   		r = (r<<5) | (r<<3) | (r>>1);
   		g = (g<<5) | (g<<3) | (g>>1);
   		b = (b<<6) | (b<<4) | (b<<2) | b;
! 		*ncp++ = 0;
! 		*ncp++ = b;
! 		*ncp++ = g;
! 		*ncp++ = r;
   	}
   	return rv;
   }

where the old ncp is an "Py_UInt32 *" and the new ncp is an "unsigned 
char *".  This results in different byte ordering on a little endian 
machine such as Intel.

-- 
Sjoerd Mullender <sjoerd@acm.org>

From jackjansen at mac.com  Sat Dec 27 10:37:08 2003
From: jackjansen at mac.com (Jack Jansen)
Date: Sat Dec 27 10:37:15 2003
Subject: [Python-Dev] 2.3.3: test_import wreaks havoc when it runs out of
	memory
Message-ID: <87A87885-3882-11D8-87A5-000A27B19B96@mac.com>

During testing 2.3.3 for MacOS9 I think I came across a bug somewhere 
in import,
but I don't have the time to track it down (and there's a workaround), 
and
it's probably only ever going to happen on OS9.
Should I put it in the bugs database anyway?

If test_import runs out of memory (probably in the 
test_module_with_large_stack
test) it fails with a MemoryError (as it should). However, from that 
point on
there is something fishy in the state of the interpreter: any later 
test that
should simply fail with an ImportError gets a SystemError: NULL return 
without
exception in stead. After a couple of these the interpreter either hangs
completely or crashes (I think around the threading tests, but I'm not 
sure).

Note that it's not a question of simply all memory being tied up: it 
seems
test_import does release most of the memory used, and many tests also 
pass
after test_import.
--
Jack Jansen, <Jack.Jansen@cwi.nl>, http://www.cwi.nl/~jack
If I can't dance I don't want to be part of your revolution -- Emma 
Goldman


From guido at python.org  Sat Dec 27 12:49:46 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Dec 27 12:49:52 2003
Subject: [Python-Dev] Fixes for imageop module
In-Reply-To: Your message of "Sat, 27 Dec 2003 00:09:39 +0100."
	<3FECBFB3.7090906@acm.org> 
References: <3FEC9C64.5080805@acm.org>
	<200312262249.hBQMnhf24306@c-24-5-183-134.client.comcast.net> 
	<3FECBFB3.7090906@acm.org> 
Message-ID: <200312271749.hBRHnkN25210@c-24-5-183-134.client.comcast.net>

[Sjoerd]
> >>I have some fixes for the imageop module to make it work on systems with 
> >>a different byte-order than IRIX (e.g. Linux).  Shall I check them in or 
> >>is it not worth it for such an old module?  Anything else that needs to 
> >>be changed when I check this in?
> >>The fixes of course will result in different results on Linux than 
> >>before, so that's the main reason for asking.

> Guido van Rossum wrote:
> > I don't see imageop in the list of deprecated modules in PEP 4, and
> > apparently *you* are still using it, so as long as you don't mind
> > being potentially the sole maintainer and user, I don't mind these
> > being in the distribution.
> > 
> > What do you mean by different results on Linux?  Was this module
> > previously doing something bogus there?

[Sjoerd]
> The main difference is:
> ***************
> *** 570,577 ****
>    		r = (r<<5) | (r<<3) | (r>>1);
>    		g = (g<<5) | (g<<3) | (g>>1);
>    		b = (b<<6) | (b<<4) | (b<<2) | b;
> ! 		nvalue = r | (g<<8) | (b<<16);
> ! 		*ncp++ = nvalue;
>    	}
>    	return rv;
>    }
> --- 560,569 ----
>    		r = (r<<5) | (r<<3) | (r>>1);
>    		g = (g<<5) | (g<<3) | (g>>1);
>    		b = (b<<6) | (b<<4) | (b<<2) | b;
> ! 		*ncp++ = 0;
> ! 		*ncp++ = b;
> ! 		*ncp++ = g;
> ! 		*ncp++ = r;
>    	}
>    	return rv;
>    }
> 
> where the old ncp is an "Py_UInt32 *" and the new ncp is an "unsigned 
> char *".  This results in different byte ordering on a little endian 
> machine such as Intel.

Hm.  Anybody who uses the imageop module currently on Linux will find
their programs broken.  The strings manipulated by imageop all end up
either being written to a file or handed to some drawing code, and
changing the byte order would definitely break things!

So I don't think this is an acceptable change.  I take it that for
IRIX, the byte order implied by the old code is simply wrong?  Maybe
the module can be given a global (shrudder?) byte order setting that
you can change but that defaults to the old setting?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From sjoerd at acm.org  Sat Dec 27 13:12:50 2003
From: sjoerd at acm.org (Sjoerd Mullender)
Date: Sat Dec 27 13:13:02 2003
Subject: [Python-Dev] Fixes for imageop module
In-Reply-To: <200312271749.hBRHnkN25210@c-24-5-183-134.client.comcast.net>
References: <3FEC9C64.5080805@acm.org>	<200312262249.hBQMnhf24306@c-24-5-183-134.client.comcast.net>
	<3FECBFB3.7090906@acm.org>
	<200312271749.hBRHnkN25210@c-24-5-183-134.client.comcast.net>
Message-ID: <3FEDCBA2.3030607@acm.org>

Guido van Rossum wrote:
> [Sjoerd]
> 
>>>>I have some fixes for the imageop module to make it work on systems with 
>>>>a different byte-order than IRIX (e.g. Linux).  Shall I check them in or 
>>>>is it not worth it for such an old module?  Anything else that needs to 
>>>>be changed when I check this in?
>>>>The fixes of course will result in different results on Linux than 
>>>>before, so that's the main reason for asking.
> 
> 
>>Guido van Rossum wrote:
>>
>>>I don't see imageop in the list of deprecated modules in PEP 4, and
>>>apparently *you* are still using it, so as long as you don't mind
>>>being potentially the sole maintainer and user, I don't mind these
>>>being in the distribution.
>>>
>>>What do you mean by different results on Linux?  Was this module
>>>previously doing something bogus there?
> 
> 
> [Sjoerd]
> 
>>The main difference is:
>>***************
>>*** 570,577 ****
>>   		r = (r<<5) | (r<<3) | (r>>1);
>>   		g = (g<<5) | (g<<3) | (g>>1);
>>   		b = (b<<6) | (b<<4) | (b<<2) | b;
>>! 		nvalue = r | (g<<8) | (b<<16);
>>! 		*ncp++ = nvalue;
>>   	}
>>   	return rv;
>>   }
>>--- 560,569 ----
>>   		r = (r<<5) | (r<<3) | (r>>1);
>>   		g = (g<<5) | (g<<3) | (g>>1);
>>   		b = (b<<6) | (b<<4) | (b<<2) | b;
>>! 		*ncp++ = 0;
>>! 		*ncp++ = b;
>>! 		*ncp++ = g;
>>! 		*ncp++ = r;
>>   	}
>>   	return rv;
>>   }
>>
>>where the old ncp is an "Py_UInt32 *" and the new ncp is an "unsigned 
>>char *".  This results in different byte ordering on a little endian 
>>machine such as Intel.
> 
> 
> Hm.  Anybody who uses the imageop module currently on Linux will find
> their programs broken.  The strings manipulated by imageop all end up
> either being written to a file or handed to some drawing code, and
> changing the byte order would definitely break things!

That's why I asked.

> So I don't think this is an acceptable change.  I take it that for
> IRIX, the byte order implied by the old code is simply wrong?  Maybe
> the module can be given a global (shrudder?) byte order setting that
> you can change but that defaults to the old setting?

The problem is, the documentation says: "This is the same format as used 
by gl.lrectwrite() and the imgfile module."  This implies the byte order 
that you get on the SGI which is opposite of what you get on Intel. The 
code produced the correct results on the SGI, but not on Intel.

(By the way, I'm away from computers for a week starting tomorrow 
extremely early.)

-- 
Sjoerd Mullender <sjoerd@acm.org>

From troels at 2-10.org  Sat Dec 27 15:10:05 2003
From: troels at 2-10.org (Troels Therkelsen)
Date: Sat Dec 27 15:10:11 2003
Subject: [Python-Dev] 2.3.3 Compilation warnings...
Message-ID: <DJECKOIDDEODMIODJIPHEEKHCDAA.troels@2-10.org>

Hey all,

I don't know what the policy on compilation warnings are with the
development of Python, but I'd imagine that it's "no warnings".

I get the following warnings:

gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
I. -I./Include  -DPy_BUILD_CORE -o Python/errors.o Python/errors.c
Python/errors.c:602: warning: function declaration isn't a prototype

gcc -pthread  -Xlinker -export-dynamic -o python \
                Modules/python.o \
                libpython2.3.a -lpthread -ldl  -lutil   -lm
libpython2.3.a(posixmodule.o): In function `posix_tmpnam':
/usr/local/src/Python-2.3.3/./Modules/posixmodule.c:5833: the use of `tmpnam_r'
is dangerous, better use `mkstemp'
libpython2.3.a(posixmodule.o): In function `posix_tempnam':
/usr/local/src/Python-2.3.3/./Modules/posixmodule.c:5788: the use of `tempnam'
is dangerous, better use `mkstemp'

building 'readline' extension
gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -fno-strict-aliasin
g -I. -I/usr/local/src/Python-2.3.3/./Include -I/usr/local/include -I/usr/local/
src/Python-2.3.3/Include -I/usr/local/src/Python-2.3.3 -c
/usr/local/src/Python-2.3.3/Modules/readline.c -o
build/temp.linux-i686-2.3/readline.o
/usr/local/src/Python-2.3.3/Modules/readline.c: In function `flex_complete':
/usr/local/src/Python-2.3.3/Modules/readline.c:583: warning: implicit
declaration of function `completion_matches'
/usr/local/src/Python-2.3.3/Modules/readline.c:583: warning: return makes
pointer from integer without a cast

I guess the warnings in posixmodule has always been there, I just
haven't been paying much attention to it and I thought I might just
as well report all warnings.


System specs:

$ uname -a
Linux gateway 2.2.19 #22 Wed Jun 20 18:12:16 PDT 2001 i686 unknown

$ gcc -v
Reading specs from /usr/lib/gcc-lib/i386-slackware-linux/2.95.3/specs
gcc version 2.95.3 20010315 (release)

Regards,

Troels Therkelsen


From guido at python.org  Sat Dec 27 17:39:47 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Dec 27 17:39:53 2003
Subject: [Python-Dev] 2.3.3 Compilation warnings...
In-Reply-To: Your message of "Sat, 27 Dec 2003 21:10:05 +0100."
	<DJECKOIDDEODMIODJIPHEEKHCDAA.troels@2-10.org> 
References: <DJECKOIDDEODMIODJIPHEEKHCDAA.troels@2-10.org> 
Message-ID: <200312272239.hBRMdlo01094@c-24-5-183-134.client.comcast.net>

> I don't know what the policy on compilation warnings are with the
> development of Python, but I'd imagine that it's "no warnings".

Right, if possible.

> I get the following warnings:
> 
> gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -
> I. -I./Include  -DPy_BUILD_CORE -o Python/errors.o Python/errors.c
> Python/errors.c:602: warning: function declaration isn't a prototype
> 
> gcc -pthread  -Xlinker -export-dynamic -o python \
>                 Modules/python.o \
>                 libpython2.3.a -lpthread -ldl  -lutil   -lm
> libpython2.3.a(posixmodule.o): In function `posix_tmpnam':
> /usr/local/src/Python-2.3.3/./Modules/posixmodule.c:5833: the use of `tmpnam_r'
> is dangerous, better use `mkstemp'
> libpython2.3.a(posixmodule.o): In function `posix_tempnam':
> /usr/local/src/Python-2.3.3/./Modules/posixmodule.c:5788: the use of `tempnam'
> is dangerous, better use `mkstemp'

Note that this is the linker speaking, not the compiler.  The only way
to get rid of this warning is not to use those two entry points, which
means removing the functions os.tempnam() and os.tmpnam().  That will
eventually happy.  These functions have been deprecated already, but
for backwards compatibility, we need to provide them for a few more
releases.  In the mean time, I don't know how to tell the linker to
suppress the warnings.

> building 'readline' extension
> gcc -pthread -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -fPIC -fno-strict-aliasin
> g -I. -I/usr/local/src/Python-2.3.3/./Include -I/usr/local/include -I/usr/local/
> src/Python-2.3.3/Include -I/usr/local/src/Python-2.3.3 -c
> /usr/local/src/Python-2.3.3/Modules/readline.c -o
> build/temp.linux-i686-2.3/readline.o
> /usr/local/src/Python-2.3.3/Modules/readline.c: In function `flex_complete':
> /usr/local/src/Python-2.3.3/Modules/readline.c:583: warning: implicit
> declaration of function `completion_matches'
> /usr/local/src/Python-2.3.3/Modules/readline.c:583: warning: return makes
> pointer from integer without a cast

This seems a case of a missing function declaration in readline.h.
Unfortunately being compatible with all versions of GNU readline that
are around is very tricky.  If you have a patch, I'd consider it, but
it would have to be tested on a number of different platforms before
being accepted.  I don't have the time to do all the verification
myself.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Sat Dec 27 17:41:16 2003
From: guido at python.org (Guido van Rossum)
Date: Sat Dec 27 17:41:23 2003
Subject: [Python-Dev] Fixes for imageop module
In-Reply-To: Your message of "Sat, 27 Dec 2003 19:12:50 +0100."
	<3FEDCBA2.3030607@acm.org> 
References: <3FEC9C64.5080805@acm.org>
	<200312262249.hBQMnhf24306@c-24-5-183-134.client.comcast.net>
	<3FECBFB3.7090906@acm.org>
	<200312271749.hBRHnkN25210@c-24-5-183-134.client.comcast.net> 
	<3FEDCBA2.3030607@acm.org> 
Message-ID: <200312272241.hBRMfHb01333@c-24-5-183-134.client.comcast.net>

> > Hm.  Anybody who uses the imageop module currently on Linux will find
> > their programs broken.  The strings manipulated by imageop all end up
> > either being written to a file or handed to some drawing code, and
> > changing the byte order would definitely break things!
> 
> That's why I asked.
> 
> > So I don't think this is an acceptable change.  I take it that for
> > IRIX, the byte order implied by the old code is simply wrong?  Maybe
> > the module can be given a global (shrudder?) byte order setting that
> > you can change but that defaults to the old setting?
> 
> The problem is, the documentation says: "This is the same format as used 
> by gl.lrectwrite() and the imgfile module."  This implies the byte order 
> that you get on the SGI which is opposite of what you get on Intel. The 
> code produced the correct results on the SGI, but not on Intel.

Your fix would be okay if until now it was *only* used on IRIX with
gl.lrectwrite() and/or the imgfile module.  But how can we prove that?

> (By the way, I'm away from computers for a week starting tomorrow 
> extremely early.)

It's okay to way a week before making a decision.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From edcjones at erols.com  Sat Dec 27 18:13:49 2003
From: edcjones at erols.com (Edward C. Jones)
Date: Sat Dec 27 18:17:43 2003
Subject: [Python-Dev] Valgrind on Python?
Message-ID: <3FEE122D.6040702@erols.com>

Has anyone run valgrind on Python? I get a lot of messages, probably 
mostly incorrect.  Most are from "PyObject_Free 
(Objects/obmalloc.c:711)". I am using Python 2.3.2 and gcc 3.2.2 with Linux.



From skanigsvv at terland.ternopil.ua  Sat Dec 27 20:19:54 2003
From: skanigsvv at terland.ternopil.ua (Skanigsvv)
Date: Sat Dec 27 20:20:18 2003
Subject: [Python-Dev] Python-dev
In-Reply-To: <LK6A945B3E1JEG2G@python.org>
References: <LK6A945B3E1JEG2G@python.org>
Message-ID: <9GLL219D5I8FF0CF@terland.ternopil.ua>

An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031228/e5f01921/attachment-0001.html
-------------- next part --------------
A non-text attachment was scrubbed...
Name: v1.gif
Type: image/gif
Size: 7859 bytes
Desc: not available
Url : http://mail.python.org/pipermail/python-dev/attachments/20031228/e5f01921/v1-0001.gif
From andymac at bullseye.apana.org.au  Sat Dec 27 18:26:59 2003
From: andymac at bullseye.apana.org.au (Andrew MacIntyre)
Date: Sat Dec 27 20:49:33 2003
Subject: [Python-Dev] Re: [Python-checkins] python/dist/src/Objects
	listobject.c, 2.172, 2.173
In-Reply-To: <1072465591.31901.71.camel@localhost.localdomain>
References: <E1AZxGl-0008So-00@sc8-pr-cvs1.sourceforge.net>
	<1072465591.31901.71.camel@localhost.localdomain>
Message-ID: <20031228092926.D12899@bullseye.apana.org.au>

On Fri, 26 Dec 2003, Jeremy Hylton wrote:

> > Modified Files:
> > 	listobject.c
> > Log Message:
> > Revert previous two checkins to repair test failure.
> >
> > The special-case code that was removed could return a value indicating
> > success but leave an exception set.  test_fileinput failed in a debug
> > build as a result.
>
> If there's some value to the optimization I just removed, let's discuss
> it on python-dev.  I could have fixed the new code, but it seems a lot
> of hairy new code with magic constants that only covered a few corner
> cases.  The checkin message pointed to a python-list discussion that
> didn't have much real evidence in it.

While Duncan Booth's tests didn't properly account for setup overhead and
his coverage of the range of source element counts insufficient, the
performance difference between the two approaches is real and can be
quite significant - list_ass_slice() is a big win for large source lists,
and a non-trivial loss for small lists (>35% for 10 or fewer elements,
>14% up to 64 elements).

The change was an attempt to maximise the performance for all cases.

If performance of list() is not seen as particularly crucial, then I would
suggest that the special case calling of list_ass_slice() should probably
be removed completely.

The special casing of empty source sequences is not really important IMO,
but the timings looked good...

Some of the interest in this area comes from that class of Python user
who wants to make a shallow copy of a list, and looks for a copy() method
on list objects, and failing that finds copy.copy().  Some of them
eventually twig that list() also does this, and doesn't require importing
the copy module.  These users don't feel comfortable with the slice
approach, despite the fact that it is considerably faster.  In fact I just
noticed a posting to python-list/c.l.p from Alex Martelli in which he
suggests list() is not unreasonable to use for this purpose.  I don't have
any problem with the slice, myself.

While I tested the change extensively, I'll confess to not trying a debug
build :-(.

BTW, my ISP's dialin is playing silly buggers at the moment, so I'm not
seeing email in a timely fashion. :-(  That's not an excuse for the
PyDict_CheckExact goof, because I saw that checkin msg and still
managed not to see the problem, and hadn't gotten around to building from
a fresh CVS checkout :-(  Thanks for fixing that.

Regards,
Andrew.

--
Andrew I MacIntyre                     "These thoughts are mine alone..."
E-mail: andymac@bullseye.apana.org.au  (pref) | Snail: PO Box 370
        andymac@pcug.org.au             (alt) |        Belconnen  ACT  2616
Web:    http://www.andymac.org/               |        Australia

From tree at basistech.com  Sat Dec 27 20:55:52 2003
From: tree at basistech.com (Tom Emerson)
Date: Sat Dec 27 21:01:34 2003
Subject: [Python-Dev] Valgrind on Python?
In-Reply-To: <3FEE122D.6040702@erols.com>
References: <3FEE122D.6040702@erols.com>
Message-ID: <16366.14376.408291.541535@magrathea.basistech.com>

Yes: valgrind and pymalloc do not get along very well. If you either
setup Valgrind exceptions, or better, build without pymaloc, you (and
valgrind) will be much happier.

    -tree

-- 
Tom Emerson                                          Basis Technology Corp.
Software Architect                                 http://www.basistech.com
  "Beware the lollipop of mediocrity: lick it once and you suck forever"

From skip at manatee.mojam.com  Sun Dec 28 08:01:17 2003
From: skip at manatee.mojam.com (Skip Montanaro)
Date: Sun Dec 28 08:01:26 2003
Subject: [Python-Dev] Weekly Python Bug/Patch Summary
Message-ID: <200312281301.hBSD1HoM014593@manatee.mojam.com>


Bug/Patch Summary
-----------------

618 open / 4455 total bugs (+70)
220 open / 2504 total patches (+22)

New Bugs
--------

socketmodule.c does not pickup all symbols with MS visC++ (2003-12-14)
	http://python.org/sf/860134
PythonIDE does not save for PythonLauncher (2003-12-15)
	http://python.org/sf/860242
fileinput does not use universal input (2003-12-15)
	http://python.org/sf/860515
Core dump problems (2003-12-15)
	http://python.org/sf/860547
UnboundLocalError in cgitb.py (2003-12-16)
	http://python.org/sf/861340
PythonIDE on osx can't run script with commandline python (2003-12-19)
	http://python.org/sf/862941
ConfigParser.getint failure on 2.3.3 (2003-12-21)
	http://python.org/sf/864063
2.3.3 tests on cygwin (2003-12-22)
	http://python.org/sf/864374
doctest chokes on recursive members (2003-12-23)
	http://python.org/sf/864944
Python 2.3.3 won't build on MacOSX 10.2 (2003-12-23)
	http://python.org/sf/864985
Hangs when opening 2nd shell with subprocess under windows (2003-12-23)
	http://python.org/sf/865014
bsddb test_all.py - incorrect (2003-12-23)
	http://python.org/sf/865120
SimpleHTTPServer docs out of date (2003-12-26)
	http://python.org/sf/866222

New Patches
-----------

tests and processors patch for urllib2 (2003-12-02)
	http://python.org/sf/852995
PyErrr_Display and traceback.format_exception_only behaviour (2003-12-15)
	http://python.org/sf/860326
installing modules are explained in old filenames (2003-12-18)
	http://python.org/sf/862531
heapq: A way to change the compare function (2003-12-28)
	http://python.org/sf/866594

Closed Bugs
-----------

Over restricted type checking on eval() function (2000-09-22)
	http://python.org/sf/215126
smtpd.py needs documentation (2001-08-14)
	http://python.org/sf/450803
Confusions in formatfloat (2002-03-20)
	http://python.org/sf/532631
email package does not work with mailbox (2002-07-26)
	http://python.org/sf/586899
Another dealloc stack killer (2002-08-25)
	http://python.org/sf/600007
doctest fails with TypeError (2003-07-02)
	http://python.org/sf/764504
doctest.DocTestSuite does not support __test__ (2003-07-15)
	http://python.org/sf/772091
bsddb3 hash craps out with threads (2003-07-21)
	http://python.org/sf/775414
Windows _bsddb link warnings (2003-08-01)
	http://python.org/sf/781614
Minor FP bug in object.c (2003-08-15)
	http://python.org/sf/789290
Parser wart (2003-08-27)
	http://python.org/sf/796116
shutil.copyfile fails when dst exists read-only (2003-09-22)
	http://python.org/sf/810879
Proto 2 pickle vs dict subclass (2003-10-20)
	http://python.org/sf/826897
os.makedirs() cannot handle "." (2003-10-24)
	http://python.org/sf/829532
reading shelves is really slow (2003-11-26)
	http://python.org/sf/849662
Typo in Popen3 description (2003-11-28)
	http://python.org/sf/850818
Doc/README has broken link (2003-11-28)
	http://python.org/sf/850823
Missing BuildRequires in src rpm specfile (2003-11-28)
	http://python.org/sf/851020
socket.recv() raises MemoryError exception (WindowsXP ONLY) (2003-12-03)
	http://python.org/sf/853507
UserDict.DictMixin's comments should be a docstring (2003-12-07)
	http://python.org/sf/856072
Remove notion of deprecated string.{atol,atof} functions (2003-12-10)
	http://python.org/sf/857821
Pathological case segmentation fault in issubclass (2003-12-10)
	http://python.org/sf/858016
typo in doc (2003-12-14)
	http://python.org/sf/859810
typo in docs (2003-12-14)
	http://python.org/sf/859811

Closed Patches
--------------

658254: accept None for time.ctime() and friends (2003-01-06)
	http://python.org/sf/663482
doctest handles comments incorrectly (2003-01-15)
	http://python.org/sf/668500
iconv_codec 3rd generation (2003-04-13)
	http://python.org/sf/720585
test_timeout updates (2003-04-28)
	http://python.org/sf/728815
Port tests to unittest (Part 2) (2003-05-13)
	http://python.org/sf/736962
traceback module caches sources invalid (2003-05-13)
	http://python.org/sf/737473
ast-branch: multiple function fixes (2003-06-11)
	http://python.org/sf/752911
itertools roundrobin() (2003-06-17)
	http://python.org/sf/756253
add time elapsed to gc debug output (2003-06-25)
	http://python.org/sf/760990
A ForwardingHandler for logging (2003-09-10)
	http://python.org/sf/804180
call com_set_lineno more often (2003-11-28)
	http://python.org/sf/850789
zipimport.c is broken on 64-bit SusE AMD, here's a fix (2003-12-11)
	http://python.org/sf/858317
documentation bool change fix (2003-12-12)
	http://python.org/sf/859286
Get rid of compiler warnings on unicodeobject.c (2003-12-13)
	http://python.org/sf/859573

From barry at barrys-emacs.org  Sun Dec 28 14:48:23 2003
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun Dec 28 14:48:31 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.ne
 t>
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private>
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>
	<200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.net>
Message-ID: <6.0.1.1.2.20031228194701.023490c8@torment.chelsea.private>

At 24-12-2003 18:12, Guido van Rossum wrote:
>I've tried to look for a way to report OpenBSD bugs, but it appears
>the only way to report one is by using their sendbug command, which
>assumes you've got OpenBSD installed.  Would someone here be so kind
>as to report the bug we've found?  A pointer to Kurt's last message in
>this thread should be a good piece of evidence.
>
>--Guido van Rossum (home page: http://www.python.org/~guido/)

 From the "How to report a problem" page on www.openbsd.org they say mail
to <mailto:bugs@openbsd.org>bugs@openbsd.org.

Barry
-------------- next part --------------
An HTML attachment was scrubbed...
URL: http://mail.python.org/pipermail/python-dev/attachments/20031228/13980c19/attachment.html
From barry at barrys-emacs.org  Sun Dec 28 14:53:33 2003
From: barry at barrys-emacs.org (Barry Scott)
Date: Sun Dec 28 14:53:39 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <m3r7yu6j37.fsf@float.localdomain>
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private>
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>
	<m3r7yu6j37.fsf@float.localdomain>
Message-ID: <6.0.1.1.2.20031228195231.022a77c8@torment.chelsea.private>

You can use -fno-stack-protector to turn off the propolice code.

Why not do that until there is a working combination?

Barry



From perky at i18n.org  Mon Dec 29 03:17:24 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Mon Dec 29 03:17:31 2003
Subject: [Python-Dev] Optimization versus code bloat
Message-ID: <20031229081724.GA42076@i18n.org>

Hello!

I posted a patch[1] optimizing str.split with specialized splitter
as unicode.split do. The patch accellates str.split about 10~20%
and adds about 50 lines. I'm curious whether it's acceptable ratio
for python-dev moods.

[1] http://www.python.org/sf/866875

Hye-Shik

From python-dev at zesty.ca  Mon Dec 29 04:43:16 2003
From: python-dev at zesty.ca (Ka-Ping Yee)
Date: Mon Dec 29 04:43:31 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <20031220151629.GA24651@panix.com>
Message-ID: <Pine.LNX.4.33.0312290142580.8440-100000@server1.lfw.org>

I've been distracted by holiday events, but this requires a reply.

On Sat, 20 Dec 2003, Aahz wrote:
> Exactly.  From my observations of these discussions, there are
> essentially only two reasons for restricted execution:
>
> * To simplify things by reducing the potential solution space
>
> * To protect a system against a hostile attacker

There is a huge blind spot in your claim.  You forgot:

  * To limit the damage caused by a bug in your program

  * To make your programs have more predictable behaviour

Capabilities are about making clear what parts of your program
can and can't do.

> Supposedly there's a middle ground of untrusted but non-hostile code,
> but what's the point of providing support for that?

Have you ever used a library written by someone else?  Have you ever
found a bug in something you wrote yourself?

"Untrusted but non-hostile code" is what all of us write every day.


-- ?!ng


From guido at python.org  Mon Dec 29 09:52:46 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 29 09:53:02 2003
Subject: [Python-Dev] Optimization versus code bloat
In-Reply-To: Your message of "Mon, 29 Dec 2003 17:17:24 +0900."
	<20031229081724.GA42076@i18n.org> 
References: <20031229081724.GA42076@i18n.org> 
Message-ID: <200312291452.hBTEql011394@c-24-5-183-134.client.comcast.net>

> I posted a patch[1] optimizing str.split with specialized splitter
> as unicode.split do. The patch accellates str.split about 10~20%
> and adds about 50 lines. I'm curious whether it's acceptable ratio
> for python-dev moods.
> 
> [1] http://www.python.org/sf/866875

Works for me -- split is a pretty common operation!  I might even use
this speedup in the Parrot benchmark. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Mon Dec 29 09:56:06 2003
From: aahz at pythoncraft.com (Aahz)
Date: Mon Dec 29 09:56:10 2003
Subject: [Python-Dev] Re: Capabilities - published interfaces
In-Reply-To: <Pine.LNX.4.33.0312290142580.8440-100000@server1.lfw.org>
References: <20031220151629.GA24651@panix.com>
	<Pine.LNX.4.33.0312290142580.8440-100000@server1.lfw.org>
Message-ID: <20031229145606.GA24858@panix.com>

On Mon, Dec 29, 2003, Ka-Ping Yee wrote:
> On Sat, 20 Dec 2003, Aahz wrote:
>>
>> Exactly.  From my observations of these discussions, there are
>> essentially only two reasons for restricted execution:
>>
>> * To simplify things by reducing the potential solution space
>>
>> * To protect a system against a hostile attacker
> 
> There is a huge blind spot in your claim.  You forgot:
> 
>   * To limit the damage caused by a bug in your program
> 
>   * To make your programs have more predictable behaviour
> 
> Capabilities are about making clear what parts of your program
> can and can't do.

My understanding is that capabilities are not the same thing as
restricted execution; it's certainly the case that we could provide some
kind of capability model that doesn't meet the demands of restricted
execution.

While your points are valid, they are not points that I've seen brought
up before in the context of restricted execution.  There are many other
programming models that address those issues; why do you think that
restricted execution makes a particularly good choice?  (Say, as compared
to design-by-contract.)

>> Supposedly there's a middle ground of untrusted but non-hostile code,
>> but what's the point of providing support for that?
> 
> Have you ever used a library written by someone else?  Have you ever
> found a bug in something you wrote yourself?
> 
> "Untrusted but non-hostile code" is what all of us write every day.

Well, sure, but if that's the only issue in play, I don't think that the
stringent demands of restricted execution are needed.  The question is
whether the benefits of restricted execution are worth the effort
required (and possibly the performance penalty).



Not-quite tangent:

I was talking to someone a few days ago, an experienced programmer who
hasn't yet tried Python (and who I'm trying to persuade -- but he's an
old Lisp and Smalltalk hacker who hates B&D syntax).  He's also running
a web server at home; I commented that if I did that, I'd use a server
written in Python because it's more secure.  He said that he'd seen more
security alerts for Zope than for Apache, which forced me to explain
that Zope isn't a web server, it's an application framework that allows
people to run arbitrary Python code.

>From my POV, that's the context in which we either need to provide truly
robust restricted execution or just decide that we can't do it.  And as
I've pointed out before, restricted execution in the context of something
like a web server is almost certainly going to require something external
to Python in order to prevent excessive CPU and memory usage -- which is
an issue that has grown in difficulty given the new prevalence of
event-driven programming (instead of forking and threading).
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From martin at v.loewis.de  Mon Dec 29 10:25:40 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 10:25:55 2003
Subject: [Python-Dev] First version of Python MSI available
Message-ID: <3FF04774.9020404@v.loewis.de>

I have committed python/nondist/sandbox/msi to the CVS.
This is a MSI generator for Python on Windows, which I'd
like to propose as a replacement for Wise. A sample .msi
file is available at

http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi

This installer includes binaries compiled with MSVC 7.1
(aka VC.NET 2003).

The MSI file tries to copy most features and user interface
from the Wise installer, but adjusts to the feature set of MSI.
I'm aware of a few issues, but I'd like to request feedback
in any case.

Known issues:
- the C runtime DLL is not yet included. It turns out that
   the VC.NET merge module for msvcr71.dll is useless, so I
   have to extract the DLL from it and incorporate that;
   I'd also like to include some auto-detection of DLLs
   used.
- MSI "administrative installs" (network installs) are not
   tested yet; neither is advertisement (this should not work
   well, in any case, as the MSI file won't advertise the
   shortcuts)
- the package doesn't ask for confirmation when installing
   into an existing directory. Not sure how to do this with
   MSI.

Please let me know what you think,

Martin


From martin at v.loewis.de  Mon Dec 29 10:27:28 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 10:27:46 2003
Subject: [Python-Dev] Switching to VC.NET 2003
Message-ID: <3FF047E0.8070905@v.loewis.de>

In my own sandbox, I have prepared project files for VC.NET.

Unless there are any strong objections, I'd like to switch
over to VC.NET shortly, disabling VC6 builds. In the process,
I will also update the build dependencies to more recent
versions of several packages.

Regards,
Martin


From guido at python.org  Mon Dec 29 10:32:30 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 29 10:32:35 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: Your message of "Mon, 29 Dec 2003 16:27:28 +0100."
	<3FF047E0.8070905@v.loewis.de> 
References: <3FF047E0.8070905@v.loewis.de> 
Message-ID: <200312291532.hBTFWU011539@c-24-5-183-134.client.comcast.net>

> In my own sandbox, I have prepared project files for VC.NET.
> 
> Unless there are any strong objections, I'd like to switch
> over to VC.NET shortly, disabling VC6 builds. In the process,
> I will also update the build dependencies to more recent
> versions of several packages.

Will the resulting installer still install a working version of Python
on old versions of Windows, like win95 and win98?  I presume this will
require that the MSI support is present on the target system; does
this exist for Win95 etc?  How about Windows NT 4.0?  If not, what's
the oldest Windows version still supported?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From logistix at cathoderaymission.net  Mon Dec 29 11:09:10 2003
From: logistix at cathoderaymission.net (logistix)
Date: Mon Dec 29 10:54:29 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <200312291532.hBTFWU011539@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>


> In my own sandbox, I have prepared project files for VC.NET.
> 
> Unless there are any strong objections, I'd like to switch
> over to VC.NET shortly, disabling VC6 builds. In the process,
> I will also update the build dependencies to more recent
> versions of several packages.
> 

Are you using a Visual Studio "project" or a makefile based build? I 
believe you need a makefile based build if you want to let people develop 
with the free SDK from microsoft.

I have some notes here that I posted a while ago:

http://tinyurl.com/34ljh


From martin at v.loewis.de  Mon Dec 29 11:00:47 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 11:01:01 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <200312291532.hBTFWU011539@c-24-5-183-134.client.comcast.net>
References: <3FF047E0.8070905@v.loewis.de>
	<200312291532.hBTFWU011539@c-24-5-183-134.client.comcast.net>
Message-ID: <3FF04FAF.40209@v.loewis.de>

Guido van Rossum wrote:

 > Will the resulting installer still install a working version of Python
 > on old versions of Windows, like win95 and win98?

I don't know; in principle, it should (or atleast, we should be able
to make it work if desired).

 > I presume this will
 > require that the MSI support is present on the target system; does
 > this exist for Win95 etc? How about Windows NT 4.0?

Yes; there is a four-file bootstrap mechanism that we could
distribute. It consists of
1. a tiny setup.exe
2. an installer for installer for W9x
3. an installer for installer for WNT
4. the actual MSI file.

I believe the setup.exe driver needs to be parametrized
as part of the build process to hard-code the name of the
MSI file; I have no procedure yet to do this.

I would propose to make parts 1..3 optional downloads;
we can also (alternatively) point users to the
redistributable download at

http://www.microsoft.com/msdownload/platformsdk/instmsi.htm

With that procedure, they would explicitly need to install
installer, then reboot (I think), then install Python.

On some installations, installer might be present already,
but in a too-old version (I use the schema of Installer 2.0).
In principle, the already-present installer should find out
that it is too old, and suggest some sort of upgrade.

 > If not, what's the oldest Windows version still supported?

A different question is the version of shared libraries
that the Python binaries depend on. On the one hand, this
is msvcr71.dll, which is not present by default on any
Windows release, so we need to ship that.

Another issue is Winsock 2 (which, I believe, is in wsock32.dll).
A quick research indicates that this was present in Win95
already. I don't know whether all of the entry points that
we use was present already; confirmation appreciated. In
principle, usage of IPv6 could be a problem. In another principle,
the platform SDK arranges to transparently emulate the IPv6
API on systems where it is not natively available, by means
of GetProcAddress. This all needs testing.

Regards,
Martin


From martin at v.loewis.de  Mon Dec 29 11:06:39 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 11:06:52 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>
Message-ID: <3FF0510F.3070504@v.loewis.de>

logistix wrote:

 > Are you using a Visual Studio "project" or a makefile based build? I
 > believe you need a makefile based build if you want to let people
 > develop with the free SDK from microsoft.

I'm using project files, so users of the free compiler would have
to write their makefiles themselves.

It might be possible to write a generator that produces a makefile
automatically from the VC.NET project files; contributions
are welcome. Maintaining both project files and makefiles in parallel
is not feasible; dropping the project files is also unacceptable.

Regards,
Martin



From martin at v.loewis.de  Mon Dec 29 11:12:44 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 11:13:04 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF04FAF.40209@v.loewis.de>
References: <3FF047E0.8070905@v.loewis.de>	<200312291532.hBTFWU011539@c-24-5-183-134.client.comcast.net>
	<3FF04FAF.40209@v.loewis.de>
Message-ID: <3FF0527C.6010909@v.loewis.de>

Martin v. Loewis wrote:
> On some installations, installer might be present already,
> but in a too-old version (I use the schema of Installer 2.0).
> In principle, the already-present installer should find out
> that it is too old, and suggest some sort of upgrade.

Following up to myself: If this turns out to be a frequent
problem, we could probably go back to an older installer
scheme - in particular, if we don't want to fight Windows
File Protection (which is only transparently supported
in most recent installer versions).

In the sample installer, I put python24.dll into TARGETDIR,
not into System32Folder. I'd like to keep it that way,
as Microsoft discourages deployment into system32. Of course,
if there is a good reason to install python24.dll into
system32, changing the installer generator to do so would
be only a small effort.

Regards,
Martin


From pje at telecommunity.com  Mon Dec 29 11:36:06 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec 29 11:37:31 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF047E0.8070905@v.loewis.de>
Message-ID: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>

At 04:27 PM 12/29/03 +0100, Martin v. Loewis wrote:
>In my own sandbox, I have prepared project files for VC.NET.
>
>Unless there are any strong objections, I'd like to switch
>over to VC.NET shortly, disabling VC6 builds. In the process,
>I will also update the build dependencies to more recent
>versions of several packages.

Was the question of mingw32-built extensions ever resolved?  That is, will 
we be able to build extensions for the standard Windows Python using the 
distutils' "mingw32" compiler, as is possible with Python 2.2?

If this hasn't been resolved, and somebody can send me a binary for a 
.NET-built Python, I'll be happy to test.  I have several Pyrex and C 
extensions of varying vintage that I can use for such a test.


From tim.one at comcast.net  Mon Dec 29 11:38:33 2003
From: tim.one at comcast.net (Tim Peters)
Date: Mon Dec 29 11:38:40 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF0527C.6010909@v.loewis.de>
Message-ID: <LNBBLJKPBEHFEDALKOLCMEPPIAAB.tim.one@comcast.net>

[Martin v. Loewis]
> ...
>
> In the sample installer, I put python24.dll into TARGETDIR,
> not into System32Folder. I'd like to keep it that way,
> as Microsoft discourages deployment into system32. Of course,
> if there is a good reason to install python24.dll into
> system32, changing the installer generator to do so would
> be only a small effort.

We need to hear from Mark Hammond about this, but he's on vacation now.  The
current installer's "non-admin install" option leaves system folders alone,
and works fine for everything I routinely do on Windows.  The point to
stuffing the Python DLL into a system directory had (IIRC) something to do
with a Windows Service embedding a Python component via COM, in which case
"the virtual command line" that started the service has no idea where Python
might live on the box.  Or something like that -- or something not like that
at all.  That's why we need Mark <wink>.


From martin at v.loewis.de  Mon Dec 29 11:43:13 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 11:43:32 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
Message-ID: <3FF059A1.8090202@v.loewis.de>

Phillip J. Eby wrote:

> Was the question of mingw32-built extensions ever resolved?  That is, 
> will we be able to build extensions for the standard Windows Python 
> using the distutils' "mingw32" compiler, as is possible with Python 2.2?

I don't know; I don't use mingw32.

OTOH, I cannot see what the problem might be - except that you will
have to link with msvcr71.dll at the end, not with msvcrt40.dll.

> If this hasn't been resolved, and somebody can send me a binary for a 
> .NET-built Python, I'll be happy to test.

Sure: Please try my installer at

http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi

Notice that this doesn't include msvcr71.dll, which I could provide
in private.

It also does not include msvcr71.lib - I have no clue how you would
get a copy of that. But then, somehow, msvcrt40.lib (or is it
msvcrt40.a) must have gotten into mingw32 also.

Regards,
Martin



From martin at v.loewis.de  Mon Dec 29 11:45:52 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 11:46:17 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <LNBBLJKPBEHFEDALKOLCMEPPIAAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCMEPPIAAB.tim.one@comcast.net>
Message-ID: <3FF05A40.4070709@v.loewis.de>

Tim Peters wrote:

> The point to
> stuffing the Python DLL into a system directory had (IIRC) something to do
> with a Windows Service embedding a Python component via COM, in which case
> "the virtual command line" that started the service has no idea where Python
> might live on the box. 

Ah, ok. I'll figure out in the mean time how to incorporate msvcr71.dll
into the package in the first place; putting it then into a "shared 
component" (along with pythonxy.dll) should be trivial.

Regards,
Martin


From martin at v.loewis.de  Mon Dec 29 11:50:18 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 11:50:34 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF059A1.8090202@v.loewis.de>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<3FF059A1.8090202@v.loewis.de>
Message-ID: <3FF05B4A.2070706@v.loewis.de>

Martin v. Loewis wrote:
> It also does not include msvcr71.lib - I have no clue how you would
> get a copy of that. But then, somehow, msvcrt40.lib (or is it
> msvcrt40.a) must have gotten into mingw32 also.

Some googling shows there is hope:

http://sources.redhat.com/ml/cygwin-announce/2003-09/msg00089.html

2003-06-18  Earnie Boyd  <earnie@users.sf.net>

	* Makefile.in (LIBS): Add new MSVCRT libraries libmsvcr70 and
	libmsvcr71, including debug versions.
	(msvcr70.def, msvcr70d.def, msvcr71.def, msvcr71.def): New targets.

So atleast the part of linking with msvcr71.dll should be
possible. As I said, I cannot imagine what else might be a problem.

Regards,
Martin


From aleaxit at yahoo.com  Mon Dec 29 11:53:09 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Mon Dec 29 11:53:23 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF0510F.3070504@v.loewis.de>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>
	<3FF0510F.3070504@v.loewis.de>
Message-ID: <200312291753.09793.aleaxit@yahoo.com>

On Monday 29 December 2003 05:06 pm, Martin v. Loewis wrote:
   ...
> It might be possible to write a generator that produces a makefile
> automatically from the VC.NET project files; contributions

At the time of Visual Studio 6, such a "generator" had been written
by Microsoft, and was part of the VS6 package -- i.e., you could produce
and save a makefile for the project you had loaded.  Is this no more the case
for the current release of Visual Studio...?


Alex


From logistix at cathoderaymission.net  Mon Dec 29 12:14:53 2003
From: logistix at cathoderaymission.net (logistix)
Date: Mon Dec 29 12:00:38 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <200312291753.09793.aleaxit@yahoo.com>
Message-ID: <Pine.LNX.4.44.0312291109150.12470-100000@oblivion.cathoderaymission.net>

On Mon, 29 Dec 2003, Alex Martelli wrote:

> On Monday 29 December 2003 05:06 pm, Martin v. Loewis wrote:
>    ...
> > It might be possible to write a generator that produces a makefile
> > automatically from the VC.NET project files; contributions
> 
> At the time of Visual Studio 6, such a "generator" had been written
> by Microsoft, and was part of the VS6 package -- i.e., you could produce
> and save a makefile for the project you had loaded.  Is this no more the case
> for the current release of Visual Studio...?
> 
> 
> Alex
> 
> 
> _______________________________________________
> Python-Dev mailing list
> Python-Dev@python.org
> http://mail.python.org/mailman/listinfo/python-dev
> Unsubscribe: http://mail.python.org/mailman/options/python-dev/logistix%40cathoderaymission.net
> 

Yes, they removed that functionality (or at least I wasn't able to find 
it).  I used Visual Studio 6.0 to generate a good makefile in my tests.  
In VC 7.1, you can feed the project file into the compiler via a command 
line flag.  I think you needed to use the GUI to build in 6.0, so MS 
provided the makefile export feature for people who wanted to do automated 
builds and other stuff where you couldn't rely on the gui.  So the feature 
became obsolete from MS's perspective.

Unfortunately, I don't think the command-line compilers in the SDK will 
accept a Visual Studio project file.



From martin at v.loewis.de  Mon Dec 29 12:03:20 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 12:04:23 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <200312291753.09793.aleaxit@yahoo.com>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>
	<200312291753.09793.aleaxit@yahoo.com>
Message-ID: <3FF05E58.1070108@v.loewis.de>

Alex Martelli wrote:
> At the time of Visual Studio 6, such a "generator" had been written
> by Microsoft, and was part of the VS6 package -- i.e., you could produce
> and save a makefile for the project you had loaded.  Is this no more the case
> for the current release of Visual Studio...?

No. Microsoft has dropped that feature (perhaps they will restore it
in VS.NET 2005 or something :-(

However, it is now simpler to process project files; they are XML
files with a fairly obvious vocabulary (*). A friend of mine once
wrote a generator that produces GNU makefiles out of vcproj files;
such generators tend to be quite application-specific, as they
need to take custom build steps into account, and translate them
properly. Most likely, the generator we would use would not be
useful outside Python.

I would like to see a single makefile generated, instead of one
per project, as the VC6 generator would do; having so many
makefiles is painful. That makefile should have an "all" target
in addition to the targets for the individual projects.

Regards,
Martin

(*) OTOH, the "solution" files (formerly workspaces) don't
use XML syntax, and have changed between VS.NET 2002 and
VS.NET 2003. They would need to be considered as well, as they
contain the inter-project dependencies (including the list
of all projects in the solution).


From martin at v.loewis.de  Mon Dec 29 12:24:57 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 12:25:42 2003
Subject: [Python-Dev] str.ascii_lower
Message-ID: <3FF06369.9020508@v.loewis.de>

Looking at python.org/sf/866982, I find it troubling that
there are languages where "I".lower() != "i"
(for those of you not familiar with Turkish: the lower-case
letter of "I" is U+0131, LATIN SMALL LETTER DOTLESS I,
which is \xfd in iso-8859-9).

As a solution, I'd like to propose a new method ascii_lower,
which is locale-unaware and only works for bytes 65..90
(returning the byte itself for all other characters).

Similarly, ascii_upper might be needed "for symmetry";
I don't know whether the symmetry should extend beyond
those two.

This, in turn, should be used inside the codecs library
where encoding names are normalized to lower case.

What do you think?

Regards,
Martin


From guido at python.org  Mon Dec 29 12:37:37 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 29 12:37:42 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: Your message of "Mon, 29 Dec 2003 18:24:57 +0100."
	<3FF06369.9020508@v.loewis.de> 
References: <3FF06369.9020508@v.loewis.de> 
Message-ID: <200312291737.hBTHbbM11741@c-24-5-183-134.client.comcast.net>

> Looking at python.org/sf/866982, I find it troubling that
> there are languages where "I".lower() != "i"
> (for those of you not familiar with Turkish: the lower-case
> letter of "I" is U+0131, LATIN SMALL LETTER DOTLESS I,
> which is \xfd in iso-8859-9).
> 
> As a solution, I'd like to propose a new method ascii_lower,
> which is locale-unaware and only works for bytes 65..90
> (returning the byte itself for all other characters).
> 
> Similarly, ascii_upper might be needed "for symmetry";
> I don't know whether the symmetry should extend beyond
> those two.
> 
> This, in turn, should be used inside the codecs library
> where encoding names are normalized to lower case.
> 
> What do you think?

I never though there were locales possible that affected the mappings
inside ASCII either.

But shouldnt' this work just as well if it's only for encoding names
(which I'd hope would be ASCII themselves):

def ascii_lower(s):
    return str(unicode(s).lower())

The unicode() call converts ASCII to Unicode, which should always work
for encoding names, and the Unicode lower() is locale-independent.

This seems more elegant than adding yet more methods to the str type.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jepler at unpythonic.net  Mon Dec 29 12:40:08 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec 29 12:40:11 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: <3FF06369.9020508@v.loewis.de>
References: <3FF06369.9020508@v.loewis.de>
Message-ID: <20031229174008.GH6171@unpythonic.net>

On Mon, Dec 29, 2003 at 06:24:57PM +0100, Martin v. Loewis wrote:
> Looking at python.org/sf/866982, I find it troubling that
> there are languages where "I".lower() != "i"
> (for those of you not familiar with Turkish: the lower-case
> letter of "I" is U+0131, LATIN SMALL LETTER DOTLESS I,
> which is \xfd in iso-8859-9).

This post caused me to notice the following behavior.  Is it "right"?

>>> import locale
>>> locale.setlocale(locale.LC_CTYPE, "tr_TR")
'tr_TR'
>>> locale.getlocale()[1]  # Expected charset
'ISO8859-9'
>>> "I".lower()   # Expected behavior
'\xfd'
>>> u"I".lower()  # Python bug? (should be u'\u0131')
u'i'
>>> locale.setlocale(locale.LC_CTYPE, "tr_TR.UTF-8")
'tr_TR.UTF-8'
>>> "I".lower()   # C library bug? (should be "\xc4\xb1")*
'I'
>>> locale.setlocale(locale.LC_CTYPE, "en_US.UTF-8")
'en_US.UTF-8'
>>> "I".lower()   # (UTF-8 locale works properly in english)
'i'

Jeff
* RedHat 9, glibc-2.3.2-11.9

From skip at pobox.com  Mon Dec 29 09:01:28 2003
From: skip at pobox.com (Skip Montanaro)
Date: Mon Dec 29 12:42:06 2003
Subject: [Python-Dev] Optimization versus code bloat
In-Reply-To: <20031229081724.GA42076@i18n.org>
References: <20031229081724.GA42076@i18n.org>
Message-ID: <16368.13240.502549.132898@montanaro.dyndns.org>


    Hye-Shik> I posted a patch[1] optimizing str.split with specialized
    Hye-Shik> splitter as unicode.split do. The patch accellates str.split
    Hye-Shik> about 10~20% and adds about 50 lines. I'm curious whether it's
    Hye-Shik> acceptable ratio for python-dev moods.

    Hye-Shik> [1] http://www.python.org/sf/866875

Seems like a reasonable addition to me, especially if the same optimization
is performed for unicode objects.  I'm a bit confused about all the changes
to test_string.py though.  Since your change is only for performance it
seems to me that no changes to the test suite would have been necessary
unless it wasn't covering some critical tests.  Was that just a
rearrangement of that file or did you add some new test cases?  Whether or
not your optimization is accepted, if your test cases beef up the test
suite, I'd vote to add them.

Skip

From pje at telecommunity.com  Mon Dec 29 12:44:46 2003
From: pje at telecommunity.com (Phillip J. Eby)
Date: Mon Dec 29 12:46:08 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF059A1.8090202@v.loewis.de>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
Message-ID: <5.1.1.6.0.20031229123908.023122e0@telecommunity.com>

At 05:43 PM 12/29/03 +0100, Martin v. Loewis wrote:
OTOH, I cannot see what the problem might be - except that you will
>have to link with msvcr71.dll at the end, not with msvcrt40.dll.
>
>>If this hasn't been resolved, and somebody can send me a binary for a 
>>.NET-built Python, I'll be happy to test.
>
>Sure: Please try my installer at
>
>http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi
>
>Notice that this doesn't include msvcr71.dll, which I could provide
>in private.

Please do.


>It also does not include msvcr71.lib - I have no clue how you would
>get a copy of that. But then, somehow, msvcrt40.lib (or is it
>msvcrt40.a) must have gotten into mingw32 also.

Actually, one generates that by using dlltool and various other programs to 
extract a .def file from python2x.dll.  I haven't generated one in ages, so 
I'm going to have to dig back up how to do that.

The rest is handled by the distutils, pretty much.  I don't think there's 
actually any direct linking to the MS VC runtime, anyway, if one is only 
using Python API calls.  But I guess I'll find out.

Hopefully I'll have enough clock cycles to spare on Thursday to get 
everything set up and tested.


From guido at python.org  Mon Dec 29 12:47:39 2003
From: guido at python.org (Guido van Rossum)
Date: Mon Dec 29 12:47:44 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: Your message of "Mon, 29 Dec 2003 11:40:08 CST."
	<20031229174008.GH6171@unpythonic.net> 
References: <3FF06369.9020508@v.loewis.de>  
	<20031229174008.GH6171@unpythonic.net> 
Message-ID: <200312291747.hBTHldO11794@c-24-5-183-134.client.comcast.net>

> This post caused me to notice the following behavior.  Is it "right"?
> 
> >>> import locale
> >>> locale.setlocale(locale.LC_CTYPE, "tr_TR")
> 'tr_TR'
> >>> locale.getlocale()[1]  # Expected charset
> 'ISO8859-9'
> >>> "I".lower()   # Expected behavior
> '\xfd'
> >>> u"I".lower()  # Python bug? (should be u'\u0131')
> u'i'

Why?  Unicode strings are not affected by the locale.

> >>> locale.setlocale(locale.LC_CTYPE, "tr_TR.UTF-8")
> 'tr_TR.UTF-8'
> >>> "I".lower()   # C library bug? (should be "\xc4\xb1")*
> 'I'
> >>> locale.setlocale(locale.LC_CTYPE, "en_US.UTF-8")
> 'en_US.UTF-8'
> >>> "I".lower()   # (UTF-8 locale works properly in english)
> 'i'

I have no idea what adding UTF8 to the local means.  Is this something
that Python's locale-awareness does or is it simply recognized by the
C library?

--Guido van Rossum (home page: http://www.python.org/~guido/)

From martin at v.loewis.de  Mon Dec 29 12:59:18 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 13:00:16 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: <200312291737.hBTHbbM11741@c-24-5-183-134.client.comcast.net>
References: <3FF06369.9020508@v.loewis.de>
	<200312291737.hBTHbbM11741@c-24-5-183-134.client.comcast.net>
Message-ID: <3FF06B76.3010004@v.loewis.de>

Guido van Rossum wrote:

> But shouldnt' this work just as well if it's only for encoding names
> (which I'd hope would be ASCII themselves):
> 
> def ascii_lower(s):
>     return str(unicode(s).lower())

Certainly. Another implementation would be

lower_map = string.maketrans(string.ascii_upper,
string.ascii_lower)
def ascii_lower(str):
    return str.translate(lower_map)

> This seems more elegant than adding yet more methods to the str type.

Ok. I'll fix it in this direction, then. There would be two copies
of this function that I can see: One for the codecs (which uses
tolower() in C code), and one in the email module.

And yes, codec names must be pure ASCII.

Regards,
Martin


From martin at v.loewis.de  Mon Dec 29 13:04:56 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 13:05:06 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: <20031229174008.GH6171@unpythonic.net>
References: <3FF06369.9020508@v.loewis.de>
	<20031229174008.GH6171@unpythonic.net>
Message-ID: <3FF06CC8.5070705@v.loewis.de>

Jeff Epler wrote:

>>>>u"I".lower()  # Python bug? (should be u'\u0131')
> 
> u'i'

As Guido says: unicode.tolower is locale-inaware;
it uses the Unicode Consortium character properties
instead to determine the lower-case character.

>>>>"I".lower()   # C library bug? (should be "\xc4\xb1")*
> 
> 'I'

This is really a limitation of the C language, not of
the C library. The interface is

char tolower(char input);

so it can only accept and return a single char. Multi-byte
characters are not supported in that interface.

Traditionally, for characters that cannot be converted,
tolower returns its argument.

>>>>"I".lower()   # (UTF-8 locale works properly in english)
> 
> 'i'

This is because "i" is a single byte in UTF-8.

Regards,
Martin


From jepler at unpythonic.net  Mon Dec 29 13:09:27 2003
From: jepler at unpythonic.net (Jeff Epler)
Date: Mon Dec 29 13:09:30 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: <200312291747.hBTHldO11794@c-24-5-183-134.client.comcast.net>
References: <3FF06369.9020508@v.loewis.de>
	<20031229174008.GH6171@unpythonic.net>
	<200312291747.hBTHldO11794@c-24-5-183-134.client.comcast.net>
Message-ID: <20031229180927.GI6171@unpythonic.net>

I stand corrected about the behavior of Unicode in the presence of
locales.

On Mon, Dec 29, 2003 at 09:47:39AM -0800, Guido van Rossum wrote:
> > >>> locale.setlocale(locale.LC_CTYPE, "tr_TR.UTF-8")
> > 'tr_TR.UTF-8'
> > >>> "I".lower()   # C library bug? (should be "\xc4\xb1")*
> > 'I'
> > >>> locale.setlocale(locale.LC_CTYPE, "en_US.UTF-8")
> > 'en_US.UTF-8'
> > >>> "I".lower()   # (UTF-8 locale works properly in english)
> > 'i'
> 
> I have no idea what adding UTF8 to the local means.  Is this something
> that Python's locale-awareness does or is it simply recognized by the
> C library?

"A locale name is typically of the form language[_territory]
[.code-set][@modifier]" -- man setlocale() on my system

RedHat 9 made a halfhearted attempt to use UTF-8 as the encoding for all
locales.  So it sets LANG=en_US.UTF-8 by default.  In theory,
tr_TR.UTF_8 should be the Turkish locale with UTF-8 characters, but it
behaves incorrectly by having "I".lower() == "I".

Well, since my earlier post combined a misunderstanding of how Python
works with a possible C library bug, I guess I raised two non-issues.
Sorry for wasting everyone's time.

Jeff

From martin at v.loewis.de  Mon Dec 29 13:09:40 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 13:09:59 2003
Subject: [Python-Dev] str.ascii_lower
In-Reply-To: <200312291747.hBTHldO11794@c-24-5-183-134.client.comcast.net>
References: <3FF06369.9020508@v.loewis.de>
	<20031229174008.GH6171@unpythonic.net>
	<200312291747.hBTHldO11794@c-24-5-183-134.client.comcast.net>
Message-ID: <3FF06DE4.1010406@v.loewis.de>

Guido van Rossum wrote:

> I have no idea what adding UTF8 to the local means.  Is this something
> that Python's locale-awareness does or is it simply recognized by the
> C library?

It's recognized by the C library, both on Windows and Unix. The full
format is

<lang>_<country>@<modifierlist>.<charset>

By default, each <lang> implies a default <charset>, which can be
overridden. On Linux, <charset> is a IANA name (e.g. de_DE.ISO-8859-15);
on Windows, it is a code page (e.g. german_Germany.1252).

Regards,
Martin


From paoloinvernizzi at dmsware.com  Mon Dec 29 15:30:07 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Mon Dec 29 15:30:28 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF05E58.1070108@v.loewis.de>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>
	<3FF05E58.1070108@v.loewis.de>
Message-ID: <3FF08ECF.4010302@dmsware.com>


Martin v. Loewis wrote:
> No. Microsoft has dropped that feature (perhaps they will restore it
> in VS.NET 2005 or something :-(
> 
> However, it is now simpler to process project files; they are XML
> files with a fairly obvious vocabulary (*). A friend of mine once
> wrote a generator that produces GNU makefiles out of vcproj files;
> such generators tend to be quite application-specific, as they
> need to take custom build steps into account, and translate them
> properly. Most likely, the generator we would use would not be
> useful outside Python.

I can suggest the fact that Scons can handle multiple builders (cygwin, 
mingw, VC6, VC7, VC7.1) easily and for VC it can generate the proper 
Project files?

Can be a solution to support a single scons (makefile like) file and 
from that generate the proper project files?

Users that wants to keep working with the IDE can do that!

A free solution for a user that want to build python in a win32 
environment without Visual Studio (whatever edition) could be:

- Download python (from 1.5.2 above) and scons (as a make replacement)
- Build python with the free Microsoft SDK compiler via scons.

Scons has also an integrated autoconf-like environment, so it's possible 
to have a path to unify the classic configure/make *nix way with the 
win32 one.

BTW, scons makefiles are just python files! Think only about the *update 
the python build number* problem that arose some times ago! It's trivial 
if handled by a builder script with-python-steroids!

---
Paolo Invernizzi

From martin at v.loewis.de  Mon Dec 29 16:55:38 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 16:56:00 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF08ECF.4010302@dmsware.com>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>	<3FF05E58.1070108@v.loewis.de>
	<3FF08ECF.4010302@dmsware.com>
Message-ID: <3FF0A2DA.4010205@v.loewis.de>

Paolo Invernizzi wrote:
> Can be a solution to support a single scons (makefile like) file and 
> from that generate the proper project files?

That might be an option, but not the one I'm working on; it isn't even
one that I'm particularly interested in. If I were maintaining the
Windows port (which I'm not), I would dislike the solution because it
would mean I have to learn scons...

> Users that wants to keep working with the IDE can do that!

... and it would also mean that I *cannot* work with the IDE: I cannot
make modifications to the projects in the IDE, because I would have
to make the modifications in the scons files, not in the VC projects.

> A free solution for a user that want to build python in a win32 
> environment without Visual Studio (whatever edition) could be:
> 
> - Download python (from 1.5.2 above) and scons (as a make replacement)
> - Build python with the free Microsoft SDK compiler via scons.

Such a free solution is already available, through cygwin/mingw.

However, this is free software, so if you are interested, feel free
to contribute. If you want your contribution be accepted, be prepared
to answer "yes" to the question "Are you going to maintain the build
process for the next three years"?

If somebody volunteers to maintain such a build process, I would not
object to having this process in parallel to the VC build process
(to which I'm willing to contribute, for the next three years).

Regards,
Martin


From paoloinvernizzi at dmsware.com  Mon Dec 29 17:23:01 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Mon Dec 29 17:23:01 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF0A2DA.4010205@v.loewis.de>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>	<3FF05E58.1070108@v.loewis.de>	<3FF08ECF.4010302@dmsware.com>
	<3FF0A2DA.4010205@v.loewis.de>
Message-ID: <3FF0A945.2020107@dmsware.com>

Martin v. Loewis wrote:

> If I were maintaining the
> Windows port (which I'm not), I would dislike the solution because it
> would mean I have to learn scons...

It is only an alternative to learning a new tool for converting VC 
project files to nmake specific makefile.

> I cannot
> make modifications to the projects in the IDE, because I would have
> to make the modifications in the scons files, not in the VC projects.

That's true. You have to make the modifications in the scons file, but 
that modifications (usually adding or removing some c module, or I'm 
missing something?) can be used by VC/cygwin/mgwin/*nix user... potentially.

> Such a free solution is already available, through cygwin/mingw.

But that leave you with the original problem... makefiles for the free 
microsoft compiler...

> However, this is free software, so if you are interested, feel free
> to contribute. If you want your contribution be accepted, be prepared
> to answer "yes" to the question "Are you going to maintain the build
> process for the next three years"?

> If somebody volunteers to maintain such a build process, I would not
> object to having this process in parallel to the VC build process
> (to which I'm willing to contribute, for the next three years).

I'm not actually in the position to reply "yes" to that question, I have 
too few spare time... but you are tempting me... :-)

It was just an alternative suggestion that come in my mind after having 
read previous threads.

I've done some month ago a scratch scons script for VC6 and VC7 
compilation of the core python dll, pythonw.exe and python.exe...

I must take a look to my messy sandbox at works!

---
Paolo Invernizzi

From perky at i18n.org  Mon Dec 29 19:34:08 2003
From: perky at i18n.org (Hye-Shik Chang)
Date: Mon Dec 29 19:34:14 2003
Subject: [Python-Dev] Optimization versus code bloat
In-Reply-To: <16368.13240.502549.132898@montanaro.dyndns.org>
References: <20031229081724.GA42076@i18n.org>
	<16368.13240.502549.132898@montanaro.dyndns.org>
Message-ID: <20031230003408.GA17100@i18n.org>

On Mon, Dec 29, 2003 at 08:01:28AM -0600, Skip Montanaro wrote:
> 
>     Hye-Shik> I posted a patch[1] optimizing str.split with specialized
>     Hye-Shik> splitter as unicode.split do. The patch accellates str.split
>     Hye-Shik> about 10~20% and adds about 50 lines. I'm curious whether it's
>     Hye-Shik> acceptable ratio for python-dev moods.
> 
>     Hye-Shik> [1] http://www.python.org/sf/866875
> 
> Seems like a reasonable addition to me, especially if the same optimization
> is performed for unicode objects.  I'm a bit confused about all the changes
> to test_string.py though.  Since your change is only for performance it
> seems to me that no changes to the test suite would have been necessary
> unless it wasn't covering some critical tests.  Was that just a
> rearrangement of that file or did you add some new test cases?

Nah. Because I added one more specialized splitter, I should add
unittests for that subsequently. So we need to test each test cases
for every specialized splitters (whitespace, string and a character).

Thanks for your review! :-)


Hye-Shik

From martin at v.loewis.de  Mon Dec 29 20:00:36 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Mon Dec 29 20:00:53 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF0A945.2020107@dmsware.com>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>	<3FF05E58.1070108@v.loewis.de>	<3FF08ECF.4010302@dmsware.com>
	<3FF0A2DA.4010205@v.loewis.de> <3FF0A945.2020107@dmsware.com>
Message-ID: <3FF0CE34.4060409@v.loewis.de>

Paolo Invernizzi wrote:
>> If I were maintaining the
>> Windows port (which I'm not), I would dislike the solution because it
>> would mean I have to learn scons...
> 
> 
> It is only an alternative to learning a new tool for converting VC 
> project files to nmake specific makefile.

It's different. That tool would have to be written from scratch,
which would be a large effort for a single person. Learning scons
would be a relatively small effort for a relatively large group
of people (not only immediate maintainers of the Windows port,
but anybody contributing to it, also).

> That's true. You have to make the modifications in the scons file, but 
> that modifications (usually adding or removing some c module, or I'm 
> missing something?) can be used by VC/cygwin/mgwin/*nix user... 
> potentially.

The more typical modification is the addition of an extension module,
which means adding a new target. That becomes more and more tricky
these days, as these new targets often have prerequisites that are
difficult to build themselves (e.g. finding an available copy of
OpenSSL, invoking Perl to build OpenSSL, invoking nmake to build
zlib, etc).

I somewhat doubt that the scons-to-vcproj generator gets such
complex build dependencies correct - at which point the maintainers
would need to look into the inner workings of scons to find out if
it really can represent the build step in all supported "output
build procedures", and how to make scons do it correctly.

> But that leave you with the original problem... makefiles for the free 
> microsoft compiler...

That, of course, is not a problem for the majority of the
Python-Windows contributors, who already have a copy of VC 7.1
(thanks to a generous offer earlier this year).

So for whoever this is a problem, they would need to find
a solution. They need to find a way today already
(as Python does  not provide nmake files out of the box
at the moment,  either). The solution does not need
to materialize itself next week, as the release of
Python 2.4 is still months away.

> I've done some month ago a scratch scons script for VC6 and VC7 
> compilation of the core python dll, pythonw.exe and python.exe...
> 
> I must take a look to my messy sandbox at works!

You could also maintain this "outside" of the standard Python
distribution, as some kind of "drop-in" zip file. That would
put no obligation onto you - your users could either use it,
or leave it (and we could advertise it as an alternative
somewhere).

Regards,
Martin


From pf_moore at yahoo.co.uk  Tue Dec 30 06:45:00 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Dec 30 06:44:54 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
References: <3FF047E0.8070905@v.loewis.de>
	<5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
Message-ID: <zndapm0z.fsf@yahoo.co.uk>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> At 04:27 PM 12/29/03 +0100, Martin v. Loewis wrote:
>>In my own sandbox, I have prepared project files for VC.NET.
>>
>>Unless there are any strong objections, I'd like to switch
>>over to VC.NET shortly, disabling VC6 builds. In the process,
>>I will also update the build dependencies to more recent
>>versions of several packages.
>
> Was the question of mingw32-built extensions ever resolved?  That is,
> will we be able to build extensions for the standard Windows Python
> using the distutils' "mingw32" compiler, as is possible with Python
> 2.2?

It is (pretty close) to being solved - recent versions of mingw32
allow building with msvcr71 via a -lmsvcr71 flag. However, last time I
tried this, the generated linker commands didn't quite work, and
needed a bit of fiddling.

I have the latest mingw, though, so I can easily try this.

Paul
-- 
This signature intentionally left blank


From pf_moore at yahoo.co.uk  Tue Dec 30 06:49:36 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Dec 30 06:50:22 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<3FF059A1.8090202@v.loewis.de>
Message-ID: <vfnypltb.fsf@yahoo.co.uk>

"Martin v. Loewis" <martin@v.loewis.de> writes:

> OTOH, I cannot see what the problem might be - except that you will
> have to link with msvcr71.dll at the end, not with msvcrt40.dll.

There's an issue with mingw using malloc/free from msvcrt in its
startup code - by the time msvcr71 gets linked in, the startup code
has already resolved these two to msvcrt. I believe this is (nearly)
resolved. Also, I've never seen a real problem caused by this, just
dire hearsay about problems using incompatible runtimes...

>> If this hasn't been resolved, and somebody can send me a binary for
>> a .NET-built Python, I'll be happy to test.
>
> Sure: Please try my installer at
>
> http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi

Does this install cleanly alongside a "production" Python 2.3? Ie,
will it leave the meaning of the "python" command, and command line
associations for .py files, etc, unchanged?

As a test install, I'd like it to have no effect on my main Python (I
have no test machine to install on separately).

> Notice that this doesn't include msvcr71.dll, which I could provide
> in private.

Not a problem for me - I have this.

> It also does not include msvcr71.lib - I have no clue how you would
> get a copy of that. But then, somehow, msvcrt40.lib (or is it
> msvcrt40.a) must have gotten into mingw32 also.

The appropriate library is indeed supplied with mingw.

Paul.
-- 
This signature intentionally left blank


From pf_moore at yahoo.co.uk  Tue Dec 30 07:02:00 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Dec 30 07:01:54 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<3FF059A1.8090202@v.loewis.de>
	<5.1.1.6.0.20031229123908.023122e0@telecommunity.com>
Message-ID: <r7ympl8n.fsf@yahoo.co.uk>

"Phillip J. Eby" <pje@telecommunity.com> writes:

> The rest is handled by the distutils, pretty much.  I don't think
> there's actually any direct linking to the MS VC runtime, anyway, if
> one is only using Python API calls.  But I guess I'll find out.

One thing that I imagine does need looking at is modifying distutils
to check whether Python was built with VC7.1, and if so add a
-lmsvcr71 flag to the gcc command line when compiling with mingw. This
can be hacked by hand, but only at the expense of making setup.py
version-specific (or doing your own test in setup.py).

I don't know if Martin has already done this, but it needs doing. I'm
not a distutils expert, but I am willing to look at it in the longer
term.

BTW, on another note, which 3rd party package developers have access
to MSVC7.1, or can build with Mingw? Has anyone surveyed this? I, for
one, have installed the following packages which have extension
modules involved:

    cx_Oracle
    mxBase
    PIL
    pygame (not used much)
    win32all
    twisted
    wxPython
    pyXML
    Metakit

If there wasn't a Windows binary version for 2.4 produced, this would
cause me problems.

At the very least, I'd suggest a warning post on c.l.p and
c.l.p.announce, something to the effect of "Python 2.4 will be built
with MSVC 7.1. Extension developers supplying binary distributions for
Windows will need some way of building MSVC 7.1 compatible modules
(MSVC 7.1 itself, or a recent version of the free Mingw compiler
package) to continue providing binries."

Paul.
-- 
This signature intentionally left blank


From skip at pobox.com  Tue Dec 30 10:03:55 2003
From: skip at pobox.com (Skip Montanaro)
Date: Tue Dec 30 10:04:05 2003
Subject: [Python-Dev] urllib2 doesn't grok URLs w/ user/passwd
Message-ID: <16369.37851.229594.332869@montanaro.dyndns.org>

SF seems to be down for some unscheduled reason.  Posting here just so I
don't completely forget about it should I exit my web browser before SF is
back up...

urllib2.urlopen("http://foo@www.python.org/") fails (at least in part)
because it fails to separate the username and password from the hostname.
Trying to open http://foo:bar@www.python.org/ reveals other shortcomings in
its url parsing.  It seems to me the syntactic bits shouldn't be difficult
to resolve using urllib.spluituser().  I'm much less clear what to do with
the username and password once they've been separated from the hostname.

Skip


From theller at python.net  Tue Dec 30 11:10:09 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec 30 10:10:55 2003
Subject: [Python-Dev] Re: [Python-checkins] python/nondist/sandbox/msi
 msi.py, 1.1.1.1, 1.2 msilib.py, 1.1.1.1, 1.2
In-Reply-To: <E1AbLOP-00042f-00@sc8-pr-cvs1.sourceforge.net>
	(loewis@users.sourceforge.net's
	message of "Tue, 30 Dec 2003 07:02:45 -0800")
References: <E1AbLOP-00042f-00@sc8-pr-cvs1.sourceforge.net>
Message-ID: <y8sui8wu.fsf@python.net>

loewis@users.sourceforge.net writes:

> Update of /cvsroot/python/python/nondist/sandbox/msi
> In directory sc8-pr-cvs1:/tmp/cvs-serv15532
>
> Modified Files:
> 	msi.py msilib.py 

>       root.start_component("TARGETDIR", default_feature)
>       root.add_file("PCBuild/w9xpopen.exe") # XXX: separate component to only install on W9x

Currently in the Wise script, w9xpopen.exe is also installed on
NT/2k/XP, and I would like to leave it this way.  py2exe needs it to
create programs using popen that also run on w9x.

Thomas


From theller at python.net  Tue Dec 30 11:25:17 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec 30 10:26:01 2003
Subject: [Python-Dev] First version of Python MSI available
In-Reply-To: <3FF04774.9020404@v.loewis.de> (Martin v. Loewis's message of
	"Mon, 29 Dec 2003 16:25:40 +0100")
References: <3FF04774.9020404@v.loewis.de>
Message-ID: <oetqi87m.fsf@python.net>

"Martin v. Loewis" <martin@v.loewis.de> writes:

> I have committed python/nondist/sandbox/msi to the CVS.
> This is a MSI generator for Python on Windows, which I'd
> like to propose as a replacement for Wise. A sample .msi
> file is available at
>
> http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi
>
> This installer includes binaries compiled with MSVC 7.1
> (aka VC.NET 2003).
>
> The MSI file tries to copy most features and user interface
> from the Wise installer, but adjusts to the feature set of MSI.
> I'm aware of a few issues, but I'd like to request feedback
> in any case.

I still have MSVC 7.0 (not 7.1) installed, and trying to build an
extension with distutils raises this error:

running build_ext
Traceback (most recent call last):
  File "setup.py", line 332, in ?
    options = options,
  File "c:\python24\lib\distutils\core.py", line 149, in setup
    dist.run_commands()
  File "c:\python24\lib\distutils\dist.py", line 907, in run_commands
    self.run_command(cmd)
  File "c:\python24\lib\distutils\dist.py", line 927, in run_command
    cmd_obj.run()
  File "c:\python24\lib\distutils\command\build.py", line 107, in run
    self.run_command(cmd_name)
  File "c:\python24\lib\distutils\cmd.py", line 333, in run_command
    self.distribution.run_command(command)
  File "c:\python24\lib\distutils\dist.py", line 927, in run_command
    cmd_obj.run()
  File "c:\python24\lib\distutils\command\build_ext.py", line 243, in run
    force=self.force)
  File "c:\python24\lib\distutils\ccompiler.py", line 1177, in new_compiler
    return klass (None, dry_run, force)
  File "c:\python24\lib\distutils\msvccompiler.py", line 206, in __init__
    self.__macros = MacroExpander(self.__version)
  File "c:\python24\lib\distutils\msvccompiler.py", line 112, in __init__
    self.load_macros(version)
  File "c:\python24\lib\distutils\msvccompiler.py", line 128, in load_macros
    self.set_macro("FrameworkSDKDir", net, "sdkinstallrootv1.1")
  File "c:\python24\lib\distutils\msvccompiler.py", line 118, in set_macro
    self.macros["$(%s)" % macro] = d[key]
KeyError: 'sdkinstallrootv1.1'
> c:\python24\lib\distutils\msvccompiler.py(118)set_macro()
-> self.macros["$(%s)" % macro] = d[key]

Here is a dump of some variable:

(Pdb) print d
{'dbgmanageddebugger': 'C:\\Programme\\Gemeinsame Dateien\\Microsoft Shared\\VS7Debug\\vs7jit.exe PID %d APPDOM %d EXTEXT "%s" EVTHDL %d', 'dbgjitdebuglaunchsetting': 2, 'installroot': 'C:\\WINDOWS\\Microsoft.NET\\Framework\\', 'sdkinstallroot': 'C:\\Programme\\Microsoft Visual Studio .NET\\FrameworkSDK\\'}
(Pdb) print key
sdkinstallrootv1.1
(Pdb) print macro
FrameworkSDKDir
(Pdb)


Thomas


From theller at python.net  Tue Dec 30 11:27:40 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec 30 10:28:22 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <vfnypltb.fsf@yahoo.co.uk> (Paul Moore's message of "Tue, 30
	Dec 2003 11:49:36 +0000")
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<3FF059A1.8090202@v.loewis.de> <vfnypltb.fsf@yahoo.co.uk>
Message-ID: <k74ei83n.fsf@python.net>

Paul Moore <pf_moore@yahoo.co.uk> writes:

> "Martin v. Loewis" <martin@v.loewis.de> writes:
>
>> Sure: Please try my installer at
>>
>> http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi
>
> Does this install cleanly alongside a "production" Python 2.3? Ie,
> will it leave the meaning of the "python" command, and command line
> associations for .py files, etc, unchanged?

No, it changes the associations for .py files. It would be useful to
have the option to create no associations.

As a workaround, installing the MSI file, and afterwards simply
reinstalling python-2.3.3.exe seems to work fine.

Thomas


From barry at python.org  Tue Dec 30 10:41:25 2003
From: barry at python.org (Barry Warsaw)
Date: Tue Dec 30 10:41:33 2003
Subject: [Python-Dev] Head broken when --enable-unicode=ucs4 is used
Message-ID: <1072798884.9216.77.camel@anthem>

I've been building Python with --enable-unicode=ucs4 ever since I
upgraded to RedHat 9.  IIUC, this option is required to get Python and
RH9's default Tk to play nicely together.

The Python cvs trunk (i.e. 2.4) is broken when using this option.  The
pain starts immediately:

gcc -pthread -c -fno-strict-aliasing -DNDEBUG -g -O3 -Wall -Wstrict-prototypes -I. -I./Include  -DPy_BUILD_CORE -o Modules/python.o Modules/python.c
In file included from Include/Python.h:82,
                 from Modules/python.c:3:
Include/unicodeobject.h:73:2: #error Must define Py_UNICODE_SIZE
In file included from Include/Python.h:82,
                 from Modules/python.c:3:
Include/unicodeobject.h:131: parse error before "Py_UNICODE"
Include/unicodeobject.h:131: warning: type defaults to `int' in declaration of `Py_UNICODE'
Include/unicodeobject.h:131: warning: data definition has no type or storage class
Include/unicodeobject.h:370: parse error before "Py_UNICODE"
Include/unicodeobject.h:370: warning: no semicolon at end of struct or union
Include/unicodeobject.h:375: parse error before '}' token
Include/unicodeobject.h:375: warning: type defaults to `int' in declaration of `PyUnicodeObject'
Include/unicodeobject.h:375: warning: data definition has no type or storage class
[...continuing on for seemingly ever...]

Without this switch, the compilation appears to work.  I don't have time
right now to investigate further, but if no one else can reproduce (or
fix) this, I'll try to help out later.

-Barry



From aleaxit at yahoo.com  Tue Dec 30 11:11:44 2003
From: aleaxit at yahoo.com (Alex Martelli)
Date: Tue Dec 30 11:11:51 2003
Subject: [Python-Dev] urllib2 doesn't grok URLs w/ user/passwd
In-Reply-To: <16369.37851.229594.332869@montanaro.dyndns.org>
References: <16369.37851.229594.332869@montanaro.dyndns.org>
Message-ID: <200312301711.44687.aleaxit@yahoo.com>

On Tuesday 30 December 2003 04:03 pm, Skip Montanaro wrote:
> SF seems to be down for some unscheduled reason.  Posting here just so I
> don't completely forget about it should I exit my web browser before SF is
> back up...
>
> urllib2.urlopen("http://foo@www.python.org/") fails (at least in part)
> because it fails to separate the username and password from the hostname.
> Trying to open http://foo:bar@www.python.org/ reveals other shortcomings in
> its url parsing.  It seems to me the syntactic bits shouldn't be difficult
> to resolve using urllib.spluituser().  I'm much less clear what to do with
> the username and password once they've been separated from the hostname.

Presumably they need to be kept somewhere and sent in the Authorization
header in case the server returns a 401 error and challenge (or a proxy 
returns a 407 error and challenge) -- or maybe the Authorization header
(with the base 64 encoding of user:pass) can be sent even as part of the
first request to speed things up (assuming an authorization scheme of
Basic).  RFC 2617, I believe.  urllib2's architecture delegates authorization
to separate components, of course, so I guess the userid and password
should just be handed over to such components if they're present, but I
haven't looked into that in detail.


Alex


From kdart at kdart.com  Tue Dec 30 13:01:42 2003
From: kdart at kdart.com (Keith Dart)
Date: Tue Dec 30 13:01:46 2003
Subject: [Python-Dev] feedback on Enum class
Message-ID: <1072807302.11336.37.camel@leviathan.kdart.com>

Greetings everyone. I hope you all had a great Christmas holiday. 8-)

I am posting this Enum (named number) class to this list for possible
feedback. It can be used as a regular integer but when printed
(stringified) it yeilds its name. The hash also makes named numbers with
the same integer value into unique dictionary keys. That is the part I
am not sure about... Comments appreciated. 
 

class Enum(int):
	__slots__ = ("_name")
	def __new__(cls, val, name):
		v = int.__new__(cls, val)
		v._name = str(name)
		return v
	def __str__(self):
		return self._name
	def __repr__(self):
		return "%s(%d, %r)" % (self.__class__.__name__, self,self._name)
	def __hash__(self):
		return int.__hash__(self) + hash(self._name)




-- 
-- ------------------------------------------------------------------------- 
Keith Dart
<mailto:kdart@kdart.com>
<http://www.kdart.com/>  
----------------------------------------------------------------------------
Public key ID: B08B9D2C Public key: <http://www.kdart.com/~kdart/public.key>
============================================================================
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: application/pgp-signature
Size: 189 bytes
Desc: This is a digitally signed message part
Url : http://mail.python.org/pipermail/python-dev/attachments/20031230/df76d679/attachment.bin
From tismer at stackless.com  Tue Dec 30 13:38:29 2003
From: tismer at stackless.com (Christian Tismer)
Date: Tue Dec 30 13:38:35 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF047E0.8070905@v.loewis.de>
References: <3FF047E0.8070905@v.loewis.de>
Message-ID: <3FF1C625.7070103@stackless.com>

Martin v. Loewis wrote:
> In my own sandbox, I have prepared project files for VC.NET.
> 
> Unless there are any strong objections, I'd like to switch
> over to VC.NET shortly, disabling VC6 builds. In the process,
> I will also update the build dependencies to more recent
> versions of several packages.

Ach du Sch***e.
I don't want to buy another M$ compiler.
Is there a freeware alternative?

ciao - chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From paoloinvernizzi at dmsware.com  Tue Dec 30 13:41:34 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Tue Dec 30 13:41:36 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF0CE34.4060409@v.loewis.de>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>	<3FF05E58.1070108@v.loewis.de>	<3FF08ECF.4010302@dmsware.com>	<3FF0A2DA.4010205@v.loewis.de>
	<3FF0A945.2020107@dmsware.com> <3FF0CE34.4060409@v.loewis.de>
Message-ID: <bssgsq$93p$1@sea.gmane.org>

Martin v. Loewis wrote:

> The more typical modification is the addition of an extension module,
> which means adding a new target. That becomes more and more tricky
> these days, as these new targets often have prerequisites that are
> difficult to build themselves (e.g. finding an available copy of
> OpenSSL, invoking Perl to build OpenSSL, invoking nmake to build
> zlib, etc).

Ok. For what I have understood reading the PCbuild/readme.txt (I admit I 
never tried to build on win32 an *hard* module, like tk or bdb, or ssl), 
the building process of that modules is still mostly a manual job...
"follow the Windows instructions for building the Sleepycat software"

> I somewhat doubt that the scons-to-vcproj generator gets such
> complex build dependencies correct - at which point the maintainers
> would need to look into the inner workings of scons to find out if
> it really can represent the build step in all supported "output
> build procedures", and how to make scons do it correctly.

The VC project generated by scons simply binds the IDE build command to 
scons itself, so when the IDE need to build something, it actually 
launch scons, which can handle a lot of mutual dependend target easily.

Well, summarizing all, I can try some *non trivial* build process, like 
the python exes plus core dll, plus out-of-the-box modules, plus OpenSSL 
or bdb, and see what can be done with it...

I'll post something as soon as I've done some tries.

---
Paolo Invernizzi



From pf_moore at yahoo.co.uk  Tue Dec 30 13:56:24 2003
From: pf_moore at yahoo.co.uk (Paul Moore)
Date: Tue Dec 30 13:56:13 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>
	<3FF059A1.8090202@v.loewis.de>
	<vfnypltb.fsf@yahoo.co.uk> <k74ei83n.fsf@python.net>
Message-ID: <3cb2xhgn.fsf@yahoo.co.uk>

Thomas Heller <theller@python.net> writes:

> No, it changes the associations for .py files. It would be useful to
> have the option to create no associations.

Agreed, that would be useful. (More generally, if there's anything
else needed to install a second Python version which is never used by
default, but only used when explicitly requested, that would be useful
too.)

Hmm, for example, does the installer set the registry key
HKLM\Software\Microsoft\Windows\CurrentVersion\App Paths\python.exe

That's what makes the command "python" work at the console prompt.

Maybe just a single option in the installer, "Make this the default
version of Python", which is true by default, would be best. This
could control file associations, App Paths, and anything else related.

> As a workaround, installing the MSI file, and afterwards simply
> reinstalling python-2.3.3.exe seems to work fine.

Yup, that's a good workaround.

Paul.
-- 
This signature intentionally left blank


From guido at python.org  Tue Dec 30 14:28:52 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 30 14:26:16 2003
Subject: [Python-Dev] urllib2 doesn't grok URLs w/ user/passwd
In-Reply-To: Your message of "Tue, 30 Dec 2003 17:11:44 +0100."
	<200312301711.44687.aleaxit@yahoo.com> 
References: <16369.37851.229594.332869@montanaro.dyndns.org>  
	<200312301711.44687.aleaxit@yahoo.com> 
Message-ID: <200312301928.hBUJSqu01449@c-24-5-183-134.client.comcast.net>

> On Tuesday 30 December 2003 04:03 pm, Skip Montanaro wrote:
> > SF seems to be down for some unscheduled reason.  Posting here just so I
> > don't completely forget about it should I exit my web browser before SF is
> > back up...
> >
> > urllib2.urlopen("http://foo@www.python.org/") fails (at least in part)
> > because it fails to separate the username and password from the hostname.
> > Trying to open http://foo:bar@www.python.org/ reveals other shortcomings in
> > its url parsing.  It seems to me the syntactic bits shouldn't be difficult
> > to resolve using urllib.spluituser().  I'm much less clear what to do with
> > the username and password once they've been separated from the hostname.
> 
> Presumably they need to be kept somewhere and sent in the Authorization
> header in case the server returns a 401 error and challenge (or a proxy 
> returns a 407 error and challenge) -- or maybe the Authorization header
> (with the base 64 encoding of user:pass) can be sent even as part of the
> first request to speed things up (assuming an authorization scheme of
> Basic).

This is what the ever-popular old urllib does.

> RFC 2617, I believe.  urllib2's architecture delegates authorization
> to separate components, of course, so I guess the userid and password
> should just be handed over to such components if they're present, but I
> haven't looked into that in detail.

Me neither.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Tue Dec 30 14:34:25 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 30 14:31:50 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: Your message of "Tue, 30 Dec 2003 19:38:29 +0100."
	<3FF1C625.7070103@stackless.com> 
References: <3FF047E0.8070905@v.loewis.de>  <3FF1C625.7070103@stackless.com> 
Message-ID: <200312301934.hBUJYPL01492@c-24-5-183-134.client.comcast.net>

> I don't want to buy another M$ compiler.
> Is there a freeware alternative?

Ten core Python developers received a free VC7.1 copy from Microsoft.
Maybe someone who hasn't opened theirs yet is willing to step down?

MS also alluded to a free downloadable compiler, but I haven't tracked
it down yet.  And of course theer's always mingw (or whatever it's
called).

--Guido van Rossum (home page: http://www.python.org/~guido/)

From aahz at pythoncraft.com  Tue Dec 30 14:59:55 2003
From: aahz at pythoncraft.com (Aahz)
Date: Tue Dec 30 14:59:59 2003
Subject: [Python-Dev] feedback on Enum class
In-Reply-To: <1072807302.11336.37.camel@leviathan.kdart.com>
References: <1072807302.11336.37.camel@leviathan.kdart.com>
Message-ID: <20031230195954.GA26455@panix.com>

On Tue, Dec 30, 2003, Keith Dart wrote:
> 
> I am posting this Enum (named number) class to this list for possible
> feedback. It can be used as a regular integer but when printed
> (stringified) it yeilds its name. The hash also makes named numbers with
> the same integer value into unique dictionary keys. That is the part I
> am not sure about... Comments appreciated. 

This should probably go to comp.lang.python.
-- 
Aahz (aahz@pythoncraft.com)           <*>         http://www.pythoncraft.com/

Weinberg's Second Law: If builders built buildings the way programmers wrote 
programs, then the first woodpecker that came along would destroy civilization.

From theller at python.net  Tue Dec 30 16:17:19 2003
From: theller at python.net (Thomas Heller)
Date: Tue Dec 30 15:17:17 2003
Subject: [Python-Dev] First version of Python MSI available
In-Reply-To: <3FF04774.9020404@v.loewis.de> (Martin v. Loewis's message of
	"Mon, 29 Dec 2003 16:25:40 +0100")
References: <3FF04774.9020404@v.loewis.de>
Message-ID: <ekum2eg0.fsf@python.net>

"Martin v. Loewis" <martin@v.loewis.de> writes:

> I have committed python/nondist/sandbox/msi to the CVS.
> This is a MSI generator for Python on Windows, which I'd
> like to propose as a replacement for Wise. A sample .msi
> file is available at
>
> http://www.dcl.hpi.uni-potsdam.de/home/loewis/python2.4.0.msi
>
> This installer includes binaries compiled with MSVC 7.1
> (aka VC.NET 2003).
>
> The MSI file tries to copy most features and user interface
> from the Wise installer, but adjusts to the feature set of MSI.
> I'm aware of a few issues, but I'd like to request feedback
> in any case.

The installer does not install pyconfig.h into Python24\include.

After copying this file manually (and installing VC7.1 - puh) I'm now
able to build extensions with distutils.

Also distutils\command\wininst.exe is not installed.

Thomas


From kbk at shore.net  Tue Dec 30 15:17:29 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Dec 30 15:17:36 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <200312301934.hBUJYPL01492@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Tue, 30 Dec 2003 11:34:25 -0800")
References: <3FF047E0.8070905@v.loewis.de> <3FF1C625.7070103@stackless.com>
	<200312301934.hBUJYPL01492@c-24-5-183-134.client.comcast.net>
Message-ID: <87znda12na.fsf@hydra.localdomain>

Guido van Rossum <guido@python.org> writes:

> Ten core Python developers received a free VC7.1 copy from Microsoft.
> Maybe someone who hasn't opened theirs yet is willing to step down?

Maybe there are a few more available?
-- 
KBK

From guido at python.org  Tue Dec 30 15:28:23 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 30 15:25:49 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: Your message of "Tue, 30 Dec 2003 15:17:29 EST."
	<87znda12na.fsf@hydra.localdomain> 
References: <3FF047E0.8070905@v.loewis.de> <3FF1C625.7070103@stackless.com>
	<200312301934.hBUJYPL01492@c-24-5-183-134.client.comcast.net> 
	<87znda12na.fsf@hydra.localdomain> 
Message-ID: <200312302028.hBUKSNN01582@c-24-5-183-134.client.comcast.net>

> > Ten core Python developers received a free VC7.1 copy from Microsoft.
> > Maybe someone who hasn't opened theirs yet is willing to step down?
> 
> Maybe there are a few more available?

I'm not going to ask.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From jjl at pobox.com  Tue Dec 30 16:55:45 2003
From: jjl at pobox.com (John J Lee)
Date: Tue Dec 30 16:56:05 2003
Subject: [Python-Dev] urllib2 doesn't grok URLs w/ user/passwd
In-Reply-To: <200312301711.44687.aleaxit@yahoo.com>
References: <16369.37851.229594.332869@montanaro.dyndns.org>
	<200312301711.44687.aleaxit@yahoo.com>
Message-ID: <Pine.LNX.4.58.0312302129080.472@alice>

On Tue, 30 Dec 2003, Alex Martelli wrote:

> On Tuesday 30 December 2003 04:03 pm, Skip Montanaro wrote:
> > SF seems to be down for some unscheduled reason.  Posting here just so I
> > don't completely forget about it should I exit my web browser before SF is
> > back up...
> >
> > urllib2.urlopen("http://foo@www.python.org/") fails (at least in part)
> > because it fails to separate the username and password from the hostname.
> > Trying to open http://foo:bar@www.python.org/ reveals other shortcomings in
[...]
> Presumably they need to be kept somewhere and sent in the Authorization
> header in case the server returns a 401 error and challenge (or a proxy
> returns a 407 error and challenge) -- or maybe the Authorization header
[...]

urllib2 already knows about this syntax for proxy auth
(ProxyHandler.proxy_open).  If somebody fixes this (assuming it needs
fixing -- I haven't read what the standards say), note that the Basic
authentication logic is duplicated, and perhaps broken in one place, which
should probably be fixed at the same time:

http://www.google.com/groups?threadm=87d6dq2jx0.fsf%40pobox.com

If nobody does it first, I'll fix it *eventually* (I'm lazy about
configuring a local proxy and web server...).

BTW, is somebody planning a 2.3.4 (or 2.2.4)?  When?  That would motivate
me to fix urllib2 bugs sooner.


John

From martin at v.loewis.de  Tue Dec 30 17:48:08 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 30 17:48:52 2003
Subject: [Python-Dev] First version of Python MSI available
In-Reply-To: <ekum2eg0.fsf@python.net>
References: <3FF04774.9020404@v.loewis.de> <ekum2eg0.fsf@python.net>
Message-ID: <3FF200A8.5090604@v.loewis.de>

Thomas Heller wrote:

> The installer does not install pyconfig.h into Python24\include.
[...]
> Also distutils\command\wininst.exe is not installed.

Thanks, will fix.

Regards,
Martin


From martin at v.loewis.de  Tue Dec 30 17:52:07 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 30 17:52:26 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF1C625.7070103@stackless.com>
References: <3FF047E0.8070905@v.loewis.de> <3FF1C625.7070103@stackless.com>
Message-ID: <3FF20197.1090802@v.loewis.de>

Christian Tismer wrote:

> Ach du Sch***e.
> I don't want to buy another M$ compiler.
> Is there a freeware alternative?

Depends on what you want to do. If all you want
to do is build extensions, this should be possible
with mingw. Actually, with some work, it might be
possible to build all of Python with mingw.

As for freeware alternatives: Linux is a good
alternative to Windows, too :-)

Regards,
Martin


From martin at v.loewis.de  Tue Dec 30 18:05:57 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 30 18:06:11 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <vfnypltb.fsf@yahoo.co.uk>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>	<3FF059A1.8090202@v.loewis.de>
	<vfnypltb.fsf@yahoo.co.uk>
Message-ID: <3FF204D5.4010501@v.loewis.de>

Paul Moore wrote:

> Does this install cleanly alongside a "production" Python 2.3? Ie,
> will it leave the meaning of the "python" command, and command line
> associations for .py files, etc, unchanged?

No. It will overwrite the associations, and, upon uninstallation,
it will entirely remove them.

OTOH, it does install into a separate folder, and it creates
shortcuts in a separate start menu folder.

> As a test install, I'd like it to have no effect on my main Python (I
> have no test machine to install on separately).

You could generate such an installer yourself, setting a variable
in msi.py. However, I can make installation of the shortcuts optional,
in future builds of the installer (to be enabled through a public
MSI property).

Regards,
Martin


From martin at v.loewis.de  Tue Dec 30 18:08:57 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 30 18:10:00 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3cb2xhgn.fsf@yahoo.co.uk>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>	<3FF059A1.8090202@v.loewis.de>	<vfnypltb.fsf@yahoo.co.uk>
	<k74ei83n.fsf@python.net> <3cb2xhgn.fsf@yahoo.co.uk>
Message-ID: <3FF20589.2030702@v.loewis.de>

Paul Moore wrote:

> Hmm, for example, does the installer set the registry key
> HKLM\Software\Microsoft\Windows\CurrentVersion\App Paths\python.exe

Yes, it does.

> Maybe just a single option in the installer, "Make this the default
> version of Python", which is true by default, would be best. This
> could control file associations, App Paths, and anything else related.

Authoring new user interface interface is a tedious task.
It is much easier to just have a public property, so you would
have to install it as

msiexec.exe python2.4.0.msi DEFAULTPYTHON=1

Regards,
Martin


From martin at v.loewis.de  Tue Dec 30 18:13:36 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 30 18:13:54 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <r7ympl8n.fsf@yahoo.co.uk>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>	<5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>	<3FF059A1.8090202@v.loewis.de>	<5.1.1.6.0.20031229123908.023122e0@telecommunity.com>
	<r7ympl8n.fsf@yahoo.co.uk>
Message-ID: <3FF206A0.3070002@v.loewis.de>

Paul Moore wrote:
> I don't know if Martin has already done this, but it needs doing. I'm
> not a distutils expert, but I am willing to look at it in the longer
> term.

I haven't changed any Python file at all in the process of compiling
with VC 7.1. However, Python (since 2.3) indicates the compiler used
to build it in sys.version; I believe this could be used as an
indication to find out whether it was build with VC6 or VC7.1 (dunno
whether it could also tell apart 7.0 and 7.1).

> If there wasn't a Windows binary version for 2.4 produced, this would
> cause me problems.

Python 2.4 is still months ahead, I expect more copies of VC 7.1 being
available by that time (unless Microsoft releases the next compiler
version before that - but they are unlikely to do so until the very
end of 2004).

> At the very least, I'd suggest a warning post on c.l.p and
> c.l.p.announce, something to the effect of "Python 2.4 will be built
> with MSVC 7.1. Extension developers supplying binary distributions for
> Windows will need some way of building MSVC 7.1 compatible modules
> (MSVC 7.1 itself, or a recent version of the free Mingw compiler
> package) to continue providing binries."

I expect the word will spread quickly; again, there is plenty of
time to prepare for that.

Regards,
Martin


From martin at v.loewis.de  Tue Dec 30 18:17:01 2003
From: martin at v.loewis.de (Martin v. Loewis)
Date: Tue Dec 30 18:17:12 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <bssgsq$93p$1@sea.gmane.org>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>	<3FF05E58.1070108@v.loewis.de>	<3FF08ECF.4010302@dmsware.com>	<3FF0A2DA.4010205@v.loewis.de>	<3FF0A945.2020107@dmsware.com>
	<3FF0CE34.4060409@v.loewis.de> <bssgsq$93p$1@sea.gmane.org>
Message-ID: <3FF2076D.7060806@v.loewis.de>

Paolo Invernizzi wrote:
> Ok. For what I have understood reading the PCbuild/readme.txt (I admit I 
> never tried to build on win32 an *hard* module, like tk or bdb, or ssl), 
> the building process of that modules is still mostly a manual job...
> "follow the Windows instructions for building the Sleepycat software"

It depends: yes for tk and sleepycat, no for ssl and bzip2.

> I'll post something as soon as I've done some tries.

Please keep us posted. It would be good if it involved no more manual
intervention than the current process; it would be a plus if the manual
intervention would be reduced.

It would also be a plus if you could figure out some of the pending
issues while you are at it, like how to stop the linker warnings when
linking _bsddb.pyd :-)

Regards,
Martin


From paoloinvernizzi at dmsware.com  Tue Dec 30 18:47:52 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Tue Dec 30 18:47:53 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF206A0.3070002@v.loewis.de>
References: <5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>	<5.1.1.6.0.20031229113056.02e151a0@telecommunity.com>	<3FF059A1.8090202@v.loewis.de>	<5.1.1.6.0.20031229123908.023122e0@telecommunity.com>	<r7ympl8n.fsf@yahoo.co.uk>
	<3FF206A0.3070002@v.loewis.de>
Message-ID: <bst2r4$sli$1@sea.gmane.org>

Martin v. Loewis wrote:

> I believe this could be used as an
> indication to find out whether it was build with VC6 or VC7.1 (dunno
> whether it could also tell apart 7.0 and 7.1).

For a version built with VC7.1 the sys.version is:

   2.4a0 (#0, Dec 15 2003, 17:22:56) [MSC v.1310 32 bit (Intel)]

If I remember well, for VC7.0 it's

[MSC v.1300 bit (Intel)]

---
Paolo Invernizzi


From paoloinvernizzi at dmsware.com  Tue Dec 30 18:54:59 2003
From: paoloinvernizzi at dmsware.com (Paolo Invernizzi)
Date: Tue Dec 30 18:55:00 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
In-Reply-To: <3FF2076D.7060806@v.loewis.de>
References: <Pine.LNX.4.44.0312291005350.11619-100000@oblivion.cathoderaymission.net>	<3FF0510F.3070504@v.loewis.de>	<200312291753.09793.aleaxit@yahoo.com>	<3FF05E58.1070108@v.loewis.de>	<3FF08ECF.4010302@dmsware.com>	<3FF0A2DA.4010205@v.loewis.de>	<3FF0A945.2020107@dmsware.com>	<3FF0CE34.4060409@v.loewis.de>
	<bssgsq$93p$1@sea.gmane.org> <3FF2076D.7060806@v.loewis.de>
Message-ID: <bst38g$t8v$1@sea.gmane.org>

Martin v. Loewis wrote:

> Please keep us posted. It would be good if it involved no more manual
> intervention than the current process; it would be a plus if the manual
> intervention would be reduced.

Ok.

> It would also be a plus if you could figure out some of the pending
> issues while you are at it, like how to stop the linker warnings when
> linking _bsddb.pyd :-)

Ok ;-)

--
Paolo Invernizzi


From trentm at ActiveState.com  Tue Dec 30 19:27:32 2003
From: trentm at ActiveState.com (Trent Mick)
Date: Tue Dec 30 19:33:07 2003
Subject: [Python-Dev] "What's New" in docs
In-Reply-To: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04BF@au3010avexu1.global.avaya.com>;
	from tdelaney@avaya.com on Fri, Dec 12, 2003 at 10:30:39AM +1100
References: <338366A6D2E2CA4C9DAEAE652E12A1DEFE04BF@au3010avexu1.global.avaya.com>
Message-ID: <20031230162732.B12400@ActiveState.com>

[Delaney, Timothy C (Timothy) wrote]
> What's New in...
>     Python 2.3
>         1 PEP 218: A Standard Set Datatype
>     Python 2.2
>         1 PEP 234: Iterators

(Behind the times here I know but:) ActivePython's chm does this. The
TOC has:

    What's New
        Release Notes
        What's New in Python 2.3?
        What's New in Python 2.2?
        What's New in Python 2.1?
        The NEWS file

Trent

-- 
Trent Mick
TrentM@ActiveState.com

From kbk at shore.net  Tue Dec 30 20:31:22 2003
From: kbk at shore.net (Kurt B. Kaiser)
Date: Tue Dec 30 20:31:27 2003
Subject: [Python-Dev] OpenBSD anyone?
In-Reply-To: <200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.net>
	(Guido van Rossum's message of "Wed, 24 Dec 2003 10:12:39 -0800")
References: <LNBBLJKPBEHFEDALKOLCEEMGHPAB.tim.one@comcast.net>
	<m34qvrwow1.fsf@float.localdomain>
	<200312231620.hBNGK8Q20710@c-24-5-183-134.client.comcast.net>
	<6.0.1.1.2.20031224173051.022dfec8@torment.chelsea.private>
	<200312241747.hBOHlMZ22008@c-24-5-183-134.client.comcast.net>
	<200312241812.hBOICdK22045@c-24-5-183-134.client.comcast.net>
Message-ID: <87brppdb85.fsf@hydra.localdomain>

Guido van Rossum <guido@python.org> writes:

> Would someone here be so kind as to report the bug we've found?

Bug submitted with cut-down:

http://cvs.openbsd.org/cgi-bin/query-pr-wrapper?full=yes&numbers=3622

It has been verified that this is fixed at OpenBSD 3.4.

Therefore it's only a problem with 3.3 because there was no stack
protector prior to that.

Workaround when using -g option is to add -fno-stack-protector or -O1.

I don't think any change to Python is warranted. 

-- 
KBK

From raymond.hettinger at verizon.net  Tue Dec 30 20:40:12 2003
From: raymond.hettinger at verizon.net (Raymond Hettinger)
Date: Tue Dec 30 20:40:41 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
Message-ID: <003901c3cf3f$08c19720$e841fea9@oemcomputer>

Dmitry Vasiliev has submitted a C implementation for the bisect module.

My thoughts are to accept it with the following changes:

* leave the original code intact for teaching purposes
* import the C version from _bisect 
* make sure the C code is not list specific and will work with any 
  container supporting __len__(), __getitem__(), and insert().  

Do you guys have any objections?


Raymond Hettinger


From guido at python.org  Tue Dec 30 20:47:31 2003
From: guido at python.org (Guido van Rossum)
Date: Tue Dec 30 20:44:57 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: Your message of "Tue, 30 Dec 2003 20:40:12 EST."
	<003901c3cf3f$08c19720$e841fea9@oemcomputer> 
References: <003901c3cf3f$08c19720$e841fea9@oemcomputer> 
Message-ID: <200312310147.hBV1lV602218@c-24-5-183-134.client.comcast.net>

> Dmitry Vasiliev has submitted a C implementation for the bisect module.
> 
> My thoughts are to accept it with the following changes:
> 
> * leave the original code intact for teaching purposes
> * import the C version from _bisect 
> * make sure the C code is not list specific and will work with any 
>   container supporting __len__(), __getitem__(), and insert().  
> 
> Do you guys have any objections?

Only that I wished you had done the same for heapq.py.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From tim.one at comcast.net  Tue Dec 30 22:23:34 2003
From: tim.one at comcast.net (Tim Peters)
Date: Tue Dec 30 22:23:40 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: <200312310147.hBV1lV602218@c-24-5-183-134.client.comcast.net>
Message-ID: <LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>

[Raymond]
>> Dmitry Vasiliev has submitted a C implementation for the bisect
>> module.
>>
>> My thoughts are to accept it with the following changes:
>>
>> * leave the original code intact for teaching purposes

+1

>> * import the C version from _bisect

+1

>> * make sure the C code is not list specific and will work with any
>>   container supporting __len__(), __getitem__(), and insert().

-0.6.  "Gratuitous hyper-generality", IMO -- I'm willing to bet nobody has
ever written a production class, with that collection of methods, that would
also be suitable for using with bisect.  "list subtype" is good enough, and
will leave the code clearer, quicker, and more maintainable.

>> Do you guys have any objections?

Six tenths of one, as above <wink>.

[Guido]
> Only that I wished you had done the same for heapq.py.

It's not too late.  IIRC, neither you nor I could make time to review the
heapq plans before the C code got checked in.  I was able to make a little
time after, and mourned the loss of the educational value of the Python
version (to Raymond, in pvt).

That's a big enough point that it deserves public airing, though:  everyone
(except Guido, I guess) please remember what it was like when you first
learned Python!  Mounds and mounds of library code written in Python, doing
interesting things with interesting techniques, and written at a high enough
level that you weren't wholly buried under details.  Python's C code is
generally clean and educational too, but has a *much* smaller potential
audience than our pure-Python modules.  Nobody can sanely accuse me of not
caring about speed (see a decade of speed-obsessed Python checkins for
counterexamples <wink>), but I cry a little each time a piece of the system
gets recoded in C.  Keeping the original Python code around is a nice
compromise (pioneered, IIRC, by string.py, a loooong time ago).


From python at rcn.com  Tue Dec 30 23:27:03 2003
From: python at rcn.com (Raymond Hettinger)
Date: Tue Dec 30 23:28:13 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
Message-ID: <005801c3cf56$703a1e60$e841fea9@oemcomputer>

[Raymond]
> Dmitry Vasiliev has submitted a C implementation for the bisect 
> module.
> 
> My thoughts are to accept it with the following changes:
> 
> * leave the original code intact for teaching purposes
> * import the C version from _bisect
> * make sure the C code is not list specific and will work with any 
>   container supporting __len__(), __getitem__(), and insert().  
> 
> Do you guys have any objections?

> [Guido]
> > Only that I wished you had done the same for heapq.py.

[Tim]
> It's not too late.  IIRC, neither you nor I could make time to review
the
> heapq plans before the C code got checked in.  I was able to make a
little
> time after, and mourned the loss of the educational value of the
Python
> version (to Raymond, in pvt).

I would be happy to bring back the python version and make it coexist
with
the C version.  IMO, it was a programming pearl unto itself and efforts
were already made to preserve the extensive module docstring and make
the
code match the original line for line with the variable names and logic
intact.

The bisect module warrants even more consideration because it is more
basic 
and more venerable. 
 


[Tim]
> Nobody can sanely accuse me of not
> caring about speed (see a decade of speed-obsessed Python checkins for
> counterexamples <wink>), but I cry a little each time a piece of the
> system
> gets recoded in C.  Keeping the original Python code around is a nice
> compromise (pioneered, IIRC, by string.py, a loooong time ago).

Also, with the PyPy project, there is a chance that cleanly coded pure
python modules will survive the test of time longer than their C
counterparts.  In the itertools module, I documented pure python 
equivalents to make the documentation more specific, to enchance
understanding, and to make it easier to backport.  For the most 
part, that effort was successful.

The only downside is the double maintenance problem of keeping the two
in sync.  Given the stability of bisect, that is not much of an issue.


Raymond Hettinger


#################################################################
#################################################################
#################################################################
#####
#####
#####
#################################################################
#################################################################
#################################################################

From skip at pobox.com  Wed Dec 31 00:08:04 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 31 00:08:13 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
References: <200312310147.hBV1lV602218@c-24-5-183-134.client.comcast.net>
	<LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
Message-ID: <16370.22964.429991.454763@montanaro.dyndns.org>


    Tim> Nobody can sanely accuse me of not caring about speed (see a decade
    Tim> of speed-obsessed Python checkins for counterexamples <wink>), but
    Tim> I cry a little each time a piece of the system gets recoded in C.
    Tim> Keeping the original Python code around is a nice compromise
    Tim> (pioneered, IIRC, by string.py, a loooong time ago).

It would be nice to keep the old Python implementations around for two
reasons.  First, there's the obvious educational value.  Second, the PyPy
folks will know where to find it when they need it.  I think a Demo/Lib
subdirectory (with appropriate README) would be a reasonable place to put
such code out to pasture.  It could be made available in source
distributions but never installed.

Skip


From tjreedy at udel.edu  Wed Dec 31 00:49:59 2003
From: tjreedy at udel.edu (Terry Reedy)
Date: Wed Dec 31 00:50:00 2003
Subject: [Python-Dev] Re: urllib2 doesn't grok URLs w/ user/passwd
References: <16369.37851.229594.332869@montanaro.dyndns.org><200312301711.44687.aleaxit@yahoo.com>
	<Pine.LNX.4.58.0312302129080.472@alice>
Message-ID: <bsto22$tig$1@sea.gmane.org>


"John J Lee" <jjl@pobox.com> wrote in message
news:Pine.LNX.4.58.0312302129080.472@alice...
> http://www.google.com/groups?threadm=87d6dq2jx0.fsf%40pobox.com
>
> If nobody does it first, I'll fix it *eventually* (I'm lazy about
> configuring a local proxy and web server...).
>
> BTW, is somebody planning a 2.3.4  #yes
>(or 2.2.4)? #no
>  When?        #April-May
>  That would motivate me to fix urllib2 bugs sooner.

I suspect more backported bug fixes will motivate a release.

tjr





From anthony at interlink.com.au  Wed Dec 31 02:49:35 2003
From: anthony at interlink.com.au (Anthony Baxter)
Date: Wed Dec 31 02:50:08 2003
Subject: [Python-Dev] Re: urllib2 doesn't grok URLs w/ user/passwd 
In-Reply-To: <bsto22$tig$1@sea.gmane.org> 
Message-ID: <200312310749.hBV7naqk011830@localhost.localdomain>


>>> "Terry Reedy" wrote
> > BTW, is somebody planning a 2.3.4  #yes
> >(or 2.2.4)? #no
> >  When?        #April-May
> >  That would motivate me to fix urllib2 bugs sooner.
> 
> I suspect more backported bug fixes will motivate a release.

I plan a 2.3.4 in the April-May-June timeframe. This will _probably_
be the final release of 2.3 before 2.4, although we could have a 2.3.5
if there's a need for it at the end of the year (shortly after 2.4 is
done would be my pick). I don't know that we'd have a need for many
more 2.3 releases after that - particularly since, at the moment at
least, 2.4 seems to be continuing down 2.3's conservative path as far as
potentially dangerous changes.

Note that, aside from major bug fixes, I'm not particularly driven by
volume of bug fixes for selecting a release schedule - mostly it's driven
by my own personal estimation of when I'm likely to have time available
to do the release. Each release I do seems to take less time (yay for 
automation! boo that the major part of the automation seems to have gone
missing somehow in the laptop theft/recovery of data from tape.... ah 
well).

Anthony
-- 
Anthony Baxter     <anthony@interlink.com.au>   
It's never too late to have a happy childhood.


From 4q89v20mku at msn.com  Wed Dec 31 16:33:44 2003
From: 4q89v20mku at msn.com (Cara Curtis)
Date: Wed Dec 31 08:39:02 2003
Subject: [Python-Dev] Refill your medication online! yyjw l
Message-ID: <xal7mq--3l$ae$81q0vc10vjjse1@bpvn7l7.q1jfw>

Rated Top Rated Online Store.

HOT & NEW - Levitra/Lipitor/Nexium

Weekly speciasls on all our drugs.

-Zocor
-Soma
-Ambien
-Phentermine
-V1agra
-Discount Generic's on ALL
-MORE

Next day discrete shipping on all products!

http://www.rxstoreusa.biz/shopping
















Please, I wish to receive no more discounts on valuable items.
http://www.rxstoreusa.biz/a.html


jet

djjdnjqj nldxi
From barry at python.org  Wed Dec 31 09:07:24 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 31 09:07:30 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: <LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
References: <LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
Message-ID: <1072879643.28895.241.camel@anthem>

On Tue, 2003-12-30 at 22:23, Tim Peters wrote:

> That's a big enough point that it deserves public airing, though:  everyone
> (except Guido, I guess) please remember what it was like when you first
> learned Python!  Mounds and mounds of library code written in Python, doing
> interesting things with interesting techniques, and written at a high enough
> level that you weren't wholly buried under details.  Python's C code is
> generally clean and educational too, but has a *much* smaller potential
> audience than our pure-Python modules.  Nobody can sanely accuse me of not
> caring about speed (see a decade of speed-obsessed Python checkins for
> counterexamples <wink>), but I cry a little each time a piece of the system
> gets recoded in C.  Keeping the original Python code around is a nice
> compromise (pioneered, IIRC, by string.py, a loooong time ago).

Yes, yes, yes.  I'd be bummed if modules like pickle.py got lost or out
of sync with its C cousin.

-Barry



From barry at python.org  Wed Dec 31 09:10:23 2003
From: barry at python.org (Barry Warsaw)
Date: Wed Dec 31 09:10:32 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: <16370.22964.429991.454763@montanaro.dyndns.org>
References: <200312310147.hBV1lV602218@c-24-5-183-134.client.comcast.net>
	<LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
	<16370.22964.429991.454763@montanaro.dyndns.org>
Message-ID: <1072879822.28895.245.camel@anthem>

On Wed, 2003-12-31 at 00:08, Skip Montanaro wrote:

> It would be nice to keep the old Python implementations around for two
> reasons.  First, there's the obvious educational value.  Second, the PyPy
> folks will know where to find it when they need it.  I think a Demo/Lib
> subdirectory (with appropriate README) would be a reasonable place to put
> such code out to pasture.  It could be made available in source
> distributions but never installed.

The problem is that relegating stuff to Demo would very likely lead to
bitrot, which is not a good thing.  Another advantage of having current,
working (and unit tested) pure-Python alternatives is that they are much
easier to modify if you're experimenting with new features, or bug
fixes, etc.

OTOH, some of the really ancient Python modules will occasionally need
attention to bring them up to modern coding standards, idioms, and
practices.

-Barry



From tismer at stackless.com  Wed Dec 31 09:44:10 2003
From: tismer at stackless.com (Christian Tismer)
Date: Wed Dec 31 09:44:16 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <3FF20197.1090802@v.loewis.de>
References: <3FF047E0.8070905@v.loewis.de> <3FF1C625.7070103@stackless.com>
	<3FF20197.1090802@v.loewis.de>
Message-ID: <3FF2E0BA.9090105@stackless.com>

Martin v. Loewis wrote:

[CT doesn't want to spend money on a new compiler]

> Depends on what you want to do. If all you want
> to do is build extensions, this should be possible
> with mingw. Actually, with some work, it might be
> possible to build all of Python with mingw.
> 
> As for freeware alternatives: Linux is a good
> alternative to Windows, too :-)

The problem is: On Windows, and with VC++6.0, I really can
fly, while on Linux I'm so-so.
It would be a shame if I had to pass Stackless Windows
support over to someone else, just because we changed compilers ;-)

best wishes for the new year -- chris
-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From tismer at stackless.com  Wed Dec 31 09:45:28 2003
From: tismer at stackless.com (Christian Tismer)
Date: Wed Dec 31 09:45:33 2003
Subject: [Python-Dev] Switching to VC.NET 2003
In-Reply-To: <200312301934.hBUJYPL01492@c-24-5-183-134.client.comcast.net>
References: <3FF047E0.8070905@v.loewis.de> <3FF1C625.7070103@stackless.com>
	<200312301934.hBUJYPL01492@c-24-5-183-134.client.comcast.net>
Message-ID: <3FF2E108.9060101@stackless.com>

Guido van Rossum wrote:

>>I don't want to buy another M$ compiler.
>>Is there a freeware alternative?
> 
> 
> Ten core Python developers received a free VC7.1 copy from Microsoft.
> Maybe someone who hasn't opened theirs yet is willing to step down?
> 
> MS also alluded to a free downloadable compiler, but I haven't tracked
> it down yet.  And of course theer's always mingw (or whatever it's
> called).

If there is a way I can provide Windows binaries, and I can do
some interactive debugging, I'm willing to switch tools, sure.

ciao - chris

-- 
Christian Tismer             :^)   <mailto:tismer@stackless.com>
Mission Impossible 5oftware  :     Have a break! Take a ride on Python's
Johannes-Niemeyer-Weg 9a     :    *Starship* http://starship.python.net/
14109 Berlin                 :     PGP key -> http://wwwkeys.pgp.net/
work +49 30 89 09 53 34  home +49 30 802 86 56  mobile +49 173 24 18 776
PGP 0x57F3BF04       9064 F4E1 D754 C2FF 1619  305B C09C 5A3B 57F3 BF04
      whom do you want to sponsor today?   http://www.stackless.com/



From skip at pobox.com  Wed Dec 31 09:56:03 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 31 09:56:14 2003
Subject: [Python-Dev] SF patch 864863: Bisect C implementation
In-Reply-To: <1072879822.28895.245.camel@anthem>
References: <200312310147.hBV1lV602218@c-24-5-183-134.client.comcast.net>
	<LNBBLJKPBEHFEDALKOLCEEJGIBAB.tim.one@comcast.net>
	<16370.22964.429991.454763@montanaro.dyndns.org>
	<1072879822.28895.245.camel@anthem>
Message-ID: <16370.58243.130247.574887@montanaro.dyndns.org>


    Barry> The problem is that relegating stuff to Demo would very likely
    Barry> lead to bitrot, which is not a good thing.  

True.  Note, however, that since these files would be in a source
distribution there's no reason they couldn't be factored into "make test"
somehow.

    Barry> Another advantage of having current, working (and unit tested)
    Barry> pure-Python alternatives is that they are much easier to modify
    Barry> if you're experimenting with new features, or bug fixes, etc.

Agreed.

Skip



From mcherm at mcherm.com  Wed Dec 31 10:12:34 2003
From: mcherm at mcherm.com (Michael Chermside)
Date: Wed Dec 31 10:12:38 2003
Subject: [Python-Dev] Re: Switching to VC.NET 2003
Message-ID: <1072883554.3ff2e762a975e@mcherm.com>

[Paul Moore:]
> Maybe just a single option in the installer, "Make this the default
> version of Python", which is true by default, would be best. This
> could control file associations, App Paths, and anything else related.

[Martin v Loewis:]
> Authoring new user interface interface is a tedious task.
> It is much easier to just have a public property, so you would
> have to install it as
> 
> msiexec.exe python2.4.0.msi DEFAULTPYTHON=1

Yes, but it's a tedious task undertaken by a single individual who
knows something about how msi files work. Whereas a checkbox in
the installer is easily understood by any user, including the vast
majority who wouldn't know what "msi" meant, much less how to
run execute one by hand with public properties specified.

Not that I'm going to complain about how you're handling this...
after all, you're the one volunteering your time. But if you're
interested in hearing random opinions from others, I am one who
would find it awefully nice if the MS installer had an option to
install a non-default version of Python. Then someday it would be
much easier for me to keep around copies of Python 2.4, 2.4.1,
2.4.2, 2.5, 2.5.1, and 2.6 all on the same machine.

If-the-PSF-were-paying-your-salary-THEN-I'd-complain

-- Michael Chermside


From barry at python.org  Wed Dec 31 11:36:20 2003
From: barry at python.org (Barry A. Warsaw)
Date: Wed Dec 31 11:36:25 2003
Subject: [Python-Dev] base64.py support for RFC 3548
Message-ID: <16370.64260.941889.163153@gargle.gargle.HOWL>


Hi all,

I found I needed more complete RFC 3548 support for a project I'm
working on.  This RFC contains the most up-to-date specification of
Base64, Base32, and Base16, including a description of alternative
alphabets, etc.

I didn't want to create any new modules, so I modified the base64.py
module to include a bunch of new functions, leaving the old API
unchanged (i.e. encode(), decode(), encodestring(), decodestring()).

I have new functions with docstrings, and an updated unit test.  If
you folks like the API I've chosen and are generally in favor of my
changes, I'll update the documentation as well.

The patch is available here:

http://sourceforge.net/tracker/index.php?func=detail&aid=868496&group_id=5470&atid=305470

Currently assigned to Tim for review.

-Barry

From barry at python.org  Wed Dec 31 11:41:11 2003
From: barry at python.org (Barry A. Warsaw)
Date: Wed Dec 31 11:41:15 2003
Subject: [Python-Dev] regrtest.py mods for code coverage
Message-ID: <16370.64551.432152.383782@gargle.gargle.HOWL>


Hi all,

I wanted to improve the code coverage for base64.py (see previous
message, re: RFC 3548 support) in the test_base64.py unit test.  So I
did a fairly simpleminded adaptation of Zope3's test.py -T code
coverage flag to regrtest.py.  Very handy for upping coverage of a
module's code!

The patch is available here:

http://sourceforge.net/tracker/index.php?func=detail&aid=868499&group_id=5470&atid=305470

Currently assigned to Jeremy for review.  Please let me know what you
think.

-Barry

From guido at python.org  Wed Dec 31 12:54:15 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 12:54:25 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
Message-ID: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>

While it's still 2003 in most of the US and Europe (and happy new year
to the folks in Asia, Australia and New Zealand! :-), I present the
official Pie-thon Parrot Benchmark:

  ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz

I'll quote from the README.txt file:

"""
This is a benchmark to be run in front of a live audience at OSCON
2004 between Python and Parrot.  The bytecode must be Python 2.3
bytecode frozen in December 2003 (which is almost over as I write this
:-).

For some more background, see the python-dev thread around
  http://mail.python.org/pipermail/python-dev/2003-December/040977.html

The benchmark here is intended to make Dan Sugalski's life difficult:
there are some standard benchmark thingies (simple random algorithms
using basic data types) but also a lot of play with odd corners of the
language definition, and Python's extremely dynamic object model:
funky descriptors, mutable classes, that sort of thing.  The benchmark
is supposed to run with either Python 2.3 or Python 2.4.

[...]

On timing: there's a requirement that the benchmark runs for at least
30 seconds.  It currently runs for nearly a minute on my home box,
which is a four-year-old 650 MHz Pentium box.  If the contest hardware
is so fast that it runs under a minute, there's a number in b.py that
can be cranked up to increase the number of runs.  (It takes 27
seconds on my recent work desktop, and on my screaming IBM T40 laptop
it runs in 15 seconds, so I suspect that we should at least double the
number of runs from 2 to 4.)
"""

I'd be happy to answer any questions.  Let the competition begin!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 13:12:13 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 13:12:48 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010201bc18c1a93cb2@[172.24.18.98]>

At 9:54 AM -0800 12/31/03, Guido van Rossum wrote:
>While it's still 2003 in most of the US and Europe (and happy new year
>to the folks in Asia, Australia and New Zealand! :-), I present the
>official Pie-thon Parrot Benchmark:
>
>   ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz

Woohoo! The announcement's forwarded on to p6i for the denizens there 
to grovel over. I fully expect many hours of pain and misery ahead to 
make this work. :)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Wed Dec 31 13:27:51 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 13:27:57 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Wed, 31 Dec 2003 13:12:13 EST."
	<a06010201bc18c1a93cb2@[172.24.18.98]> 
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>  
	<a06010201bc18c1a93cb2@[172.24.18.98]> 
Message-ID: <200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>

> >   ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
> 
> Woohoo! The announcement's forwarded on to p6i for the denizens there 
> to grovel over. I fully expect many hours of pain and misery ahead to 
> make this work. :)

Enjoy. :-)  Is there a online archive where I can watch the fun
without signing up?

BTW I forgot to mention that parts of the test are self-checking, but
other parts require comparing of the output.  The file 'out' is all
you should care about.  And if you want the bytecode, try this in a
Unix shell:

  $ python2.3 -O
  [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
  Type "help", "copyright", "credits" or "license" for more information.
  >>> import b
  >>> ^D
  $

You should now have optimized bytecode in the 8 files b*.pyo.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 13:40:18 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 13:40:49 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net> 
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010202bc18c805ba84@[172.24.18.98]>

At 10:27 AM -0800 12/31/03, Guido van Rossum wrote:
>  > >   ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
>>
>>  Woohoo! The announcement's forwarded on to p6i for the denizens there
>>  to grovel over. I fully expect many hours of pain and misery ahead to
>>  make this work. :)
>
>Enjoy. :-)  Is there a online archive where I can watch the fun
>without signing up?

Yep. There's a news<->mail gateway for all the perl.org lists with 
web archive, so you can read via a newsreader at 
news://nntp.perl.org/perl.perl6.internals or webbed at 
http://www.nntp.perl.org/group/perl.perl6.internals. It's a bit of a 
rough web archive, unfortunately, but the gateway does maintain 
threads. (Assuming the participants aren't using a broken mail 
program. Alas, I sometimes do)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From dan at sidhe.org  Wed Dec 31 14:06:10 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 14:06:31 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net> 
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010203bc18ce6f3b32@[172.24.18.98]>

At 10:27 AM -0800 12/31/03, Guido van Rossum wrote:
>  > >   ftp://ftp.python.org/pub/python/parrotbench/parrotbench.tgz
>>
>>  Woohoo! The announcement's forwarded on to p6i for the denizens there
>>  to grovel over. I fully expect many hours of pain and misery ahead to
>>  make this work. :)
>
>Enjoy. :-)  Is there a online archive where I can watch the fun
>without signing up?
>
>BTW I forgot to mention that parts of the test are self-checking, but
>other parts require comparing of the output.  The file 'out' is all
>you should care about.  And if you want the bytecode, try this in a
>Unix shell:
>
>   $ python2.3 -O
>   [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
>   Type "help", "copyright", "credits" or "license" for more information.
>   >>> import b
>   >>> ^D
>   $

Hrm. On my OS X laptop:

lir:~/Desktop/parrotbench dan$ python -O
Python 2.3 (#1, Sep 13 2003, 00:49:11)
[GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>>  import b
--> iteration 0
--> b0
Traceback (most recent call last):
   File "<stdin>", line 1, in ?
   File "b.py", line 12, in ?
     b0.main()
   File "b0.py", line 893, in main
     checkoutput(4201300315)
   File "b0.py", line 763, in checkoutput
     check(strhash(outputtext), n)
   File "b0.py", line 3, in check
     raise AssertionError("%.30r != %.30r" % (a, b))
AssertionError: 503203581L != 4201300315L
>>>

I'll try generating the bytecode on a linux box, but something seems 
amiss somewhere. (Perhaps just in my understanding of what should be 
happening)
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Wed Dec 31 14:31:02 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 14:31:08 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Wed, 31 Dec 2003 14:06:10 EST."
	<a06010203bc18ce6f3b32@[172.24.18.98]> 
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net> 
	<a06010203bc18ce6f3b32@[172.24.18.98]> 
Message-ID: <200312311931.hBVJV2o05897@c-24-5-183-134.client.comcast.net>

(Thanks for the p6i archives link BTW; Googling for "p6i" wasn't too
helpful since its true name is perl6.internals. :-)

> >   $ python2.3 -O
> >   [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
> >   Type "help", "copyright", "credits" or "license" for more information.
> >   >>> import b
> >   >>> ^D
> >   $
> 
> Hrm. On my OS X laptop:
> 
> lir:~/Desktop/parrotbench dan$ python -O
> Python 2.3 (#1, Sep 13 2003, 00:49:11)
> [GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
> Type "help", "copyright", "credits" or "license" for more information.
> >>>  import b
> --> iteration 0
> --> b0
> Traceback (most recent call last):
>    File "<stdin>", line 1, in ?
>    File "b.py", line 12, in ?
>      b0.main()
>    File "b0.py", line 893, in main
>      checkoutput(4201300315)
>    File "b0.py", line 763, in checkoutput
>      check(strhash(outputtext), n)
>    File "b0.py", line 3, in check
>      raise AssertionError("%.30r != %.30r" % (a, b))
> AssertionError: 503203581L != 4201300315L
> >>>
> 
> I'll try generating the bytecode on a linux box, but something seems 
> amiss somewhere. (Perhaps just in my understanding of what should be 
> happening)

No, this means that the benchmark fails on that platform.

But you got quite far (the first 5 sections of b0.py worked without a
hitch).

I notice you're using the original Python 2.3 release thjat came with
Panther.  It's a bit old.  If upgrading to Python 2.3.3 doesn't help,
can you send me the output of

  python b0.py

?  Send it in private mail, it's rather large. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 16:16:34 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 16:18:25 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312311931.hBVJV2o05897@c-24-5-183-134.client.comcast.net>
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net> 
	<a06010203bc18ce6f3b32@[172.24.18.98]>
	<200312311931.hBVJV2o05897@c-24-5-183-134.client.comcast.net>
Message-ID: <a06010205bc18ebb31724@[172.24.18.98]>

At 11:31 AM -0800 12/31/03, Guido van Rossum wrote:
>(Thanks for the p6i archives link BTW; Googling for "p6i" wasn't too
>helpful since its true name is perl6.internals. :-)
>
>>  >   $ python2.3 -O
>>  >   [GCC 3.2.2 20030222 (Red Hat Linux 3.2.2-5)] on linux2
>>  >   Type "help", "copyright", "credits" or "license" for more information.
>>  >   >>> import b
>>  >   >>> ^D
>>  >   $
>>
>  > Hrm. On my OS X laptop:

[Failure snipped]

>  >
>>  I'll try generating the bytecode on a linux box, but something seems
>>  amiss somewhere. (Perhaps just in my understanding of what should be
>>  happening)
>
>No, this means that the benchmark fails on that platform.
>
>But you got quite far (the first 5 sections of b0.py worked without a
>hitch).
>
>I notice you're using the original Python 2.3 release thjat came with
>Panther.  It's a bit old.  If upgrading to Python 2.3.3 doesn't help,

Turns out not to. I built and installed the 2.3.3 kit and it still 
fails, albeit differently.

>can you send me the output of
>
>   python b0.py
>
>?  Send it in private mail, it's rather large. :-)

Sent under a separate cover. Traceback (for those folks following 
along at home) is:

lir:~/Desktop/parrotbench dan$ /usr/local/bin/python2.3 
b0.py >b0_output_darwin.lst
Traceback (most recent call last):
   File "b0.py", line 924, in ?
     main()
   File "b0.py", line 893, in main
     checkoutput(4201300315)
   File "b0.py", line 763, in checkoutput
     check(strhash(outputtext), n)
   File "b0.py", line 3, in check
     raise AssertionError("%.30r != %.30r" % (a, b))
AssertionError: 2772929724L != 4201300315L
lir:~/Desktop/parrotbench dan$


I did get bytecode on a linux system, so I've something to work with, 
though it'll be tough to test times as we go on. (Though I figure if 
I can beat the 1.4GHz Celeron system with my 600MHz G3 laptop I'll be 
doing good :)

I'll give this a shot on the 10.2 OS X server box I have at home, 
just for chuckles. (I can set it up as an automated build server if 
you want, which is all I use it for right now)

-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From skip at pobox.com  Wed Dec 31 18:33:27 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 31 18:33:37 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <a06010205bc18ebb31724@[172.24.18.98]>
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
	<a06010203bc18ce6f3b32@[172.24.18.98]>
	<200312311931.hBVJV2o05897@c-24-5-183-134.client.comcast.net>
	<a06010205bc18ebb31724@[172.24.18.98]>
Message-ID: <16371.23751.389244.816299@montanaro.dyndns.org>


    >> > Hrm. On my OS X laptop:

    Dan> [Failure snipped]

    >> I notice you're using the original Python 2.3 release thjat came with
    >> Panther.  It's a bit old.  If upgrading to Python 2.3.3 doesn't help,

    Dan> Turns out not to. I built and installed the 2.3.3 kit and it still 
    Dan> fails, albeit differently.

Works for me (Mac OS X 10.2.8, Python 2.3.2, GCC 3.3, parrotbench directory
after cvs up around 5:30PM central time):

    montanaro:parrotbench% python2.3 -O
    Python 2.3.2 (#1, Dec 17 2003, 17:22:01) 
    [GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import b
    --> iteration 0
    --> b0
    3141592653
    3141592653
    --> b1
    False
    False
    0
    --> b2
    ...
    --> b4
    [0, 1, 2, 3, 4, 5, 6, 7, 8, 9]
    --> b5
    OK.
    --> b6
    --> All done.

Did you make sure you didn't have any .py[co] files before starting?

Skip

From guido at python.org  Wed Dec 31 18:34:07 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 18:34:14 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Wed, 31 Dec 2003 11:31:02 PST."
Message-ID: <200312312334.hBVNY7506326@c-24-5-183-134.client.comcast.net>

Dan reported a problem with the benchmark on Mac OS X:

> > Hrm. On my OS X laptop:
> > 
> > lir:~/Desktop/parrotbench dan$ python -O
> > Python 2.3 (#1, Sep 13 2003, 00:49:11)
> > [GCC 3.3 20030304 (Apple Computer, Inc. build 1495)] on darwin
> > Type "help", "copyright", "credits" or "license" for more information.
> > >>>  import b
> > --> iteration 0
> > --> b0
> > Traceback (most recent call last):
> >    File "<stdin>", line 1, in ?
> >    File "b.py", line 12, in ?
> >      b0.main()
> >    File "b0.py", line 893, in main
> >      checkoutput(4201300315)
> >    File "b0.py", line 763, in checkoutput
> >      check(strhash(outputtext), n)
> >    File "b0.py", line 3, in check
> >      raise AssertionError("%.30r != %.30r" % (a, b))
> > AssertionError: 503203581L != 4201300315L
> > >>>

After some off-line email exchanges I think I have a fix for this
behavior, which must have to do with a different length of the
addresses shown in the default repr(), e.g. "<Foo object at 0xffff>".

Version 1.0.1 of the benchmark is at the same place as before:

  ftp://python.org/pub/python/parrotbench/parrotbench.tgz

(You can tell whether you have the fixed version by looking at first
line of README.txt; if it says "Parrot benchmark 1.0.1" you do.)

I haven't heard back from Dan, but I assume that the fix works.

Happy New Year everyone!

--Guido van Rossum (home page: http://www.python.org/~guido/)

From guido at python.org  Wed Dec 31 18:45:00 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 18:45:07 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Wed, 31 Dec 2003 17:33:27 CST."
	<16371.23751.389244.816299@montanaro.dyndns.org> 
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
	<a06010203bc18ce6f3b32@[172.24.18.98]>
	<200312311931.hBVJV2o05897@c-24-5-183-134.client.comcast.net>
	<a06010205bc18ebb31724@[172.24.18.98]> 
	<16371.23751.389244.816299@montanaro.dyndns.org> 
Message-ID: <200312312345.hBVNj0v06358@c-24-5-183-134.client.comcast.net>

>     Dan> Turns out not to. I built and installed the 2.3.3 kit and it still 
>     Dan> fails, albeit differently.
> 
> Works for me (Mac OS X 10.2.8, Python 2.3.2, GCC 3.3, parrotbench directory
> after cvs up around 5:30PM central time):

Check the README.txt.  Does it say 1.0.1?  Then you have my fixed version!

I hope so, then I can go home. :-)
 
>     montanaro:parrotbench% python2.3 -O
>     Python 2.3.2 (#1, Dec 17 2003, 17:22:01) 
>     [GCC 3.3 20030304 (Apple Computer, Inc. build 1493)] on darwin
>     Type "help", "copyright", "credits" or "license" for more information.
>     >>> import b
>     --> iteration 0
>     --> b0
>     3141592653
>     3141592653
>     --> b1
>     False
>     False
      ^^^^^
This actually means that you failed the stack overflow test.  It should
work when you run it like this:

  python -O b.py

Or you could do sys.setrecursionlimit(1001) before importing b in the
interactive session.

But that's unrelated to Dan's problem (which happens before b0
completes).  And even if that test fails, the .pyo files will be
correct.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 18:49:36 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 18:50:03 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312312334.hBVNY7506326@c-24-5-183-134.client.comcast.net>
References: <200312312334.hBVNY7506326@c-24-5-183-134.client.comcast.net>
Message-ID: <a0601020bbc1910f7eee6@[172.24.18.98]>

At 3:34 PM -0800 12/31/03, Guido van Rossum wrote:
>After some off-line email exchanges I think I have a fix for this
>behavior, which must have to do with a different length of the
>addresses shown in the default repr(), e.g. "<Foo object at 0xffff>".
>
>Version 1.0.1 of the benchmark is at the same place as before:
>
>   ftp://python.org/pub/python/parrotbench/parrotbench.tgz
>
>(You can tell whether you have the fixed version by looking at first
>line of README.txt; if it says "Parrot benchmark 1.0.1" you do.)
>
>I haven't heard back from Dan, but I assume that the fix works.

Yup, works just fine. (You caught me during the east coast commute home :)

>Happy New Year everyone!

Ho, ho, ho!
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From allison at sumeru.stanford.EDU  Wed Dec 31 18:56:00 2003
From: allison at sumeru.stanford.EDU (Dennis Allison)
Date: Wed Dec 31 18:56:49 2003
Subject: [Python-Dev] Are we collecting benchmark results across machines
In-Reply-To: <200312312334.hBVNY7506326@c-24-5-183-134.client.comcast.net>
Message-ID: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU>


I suspect there are other folks who have run the pie-con benchmarks on
their machines.  Perhaps we should construct a chart.  Times below are in
seconds.

~60	Guido's ancient 650Mhz Pentium
~27	Guido's desktop at work
~15	IBM T40 laptop

88.550	dual 450 MHz Pentium 2
22.340	Athlon XP2800 (1243.363 MHz clock)

The latter two times are for Python 2.3.3 built out of the box using the
Makefile (make time) shipped with the parrotbenchmarks ftp file on an
unloaded machine. The times reported are the user time from the time
triplet.



From skip at pobox.com  Wed Dec 31 18:56:25 2003
From: skip at pobox.com (Skip Montanaro)
Date: Wed Dec 31 18:57:06 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312312345.hBVNj0v06358@c-24-5-183-134.client.comcast.net>
References: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.net>
	<a06010201bc18c1a93cb2@[172.24.18.98]>
	<200312311827.hBVIRpb05769@c-24-5-183-134.client.comcast.net>
	<a06010203bc18ce6f3b32@[172.24.18.98]>
	<200312311931.hBVJV2o05897@c-24-5-183-134.client.comcast.net>
	<a06010205bc18ebb31724@[172.24.18.98]>
	<16371.23751.389244.816299@montanaro.dyndns.org>
	<200312312345.hBVNj0v06358@c-24-5-183-134.client.comcast.net>
Message-ID: <16371.25129.466173.864597@montanaro.dyndns.org>


    >> Works for me (Mac OS X 10.2.8, Python 2.3.2, GCC 3.3, parrotbench
    >> directory after cvs up around 5:30PM central time):

    Guido> Check the README.txt.  Does it say 1.0.1?  Then you have my fixed
    Guido> version!

Yes.

    Guido> I hope so, then I can go home. :-)

I saw your checkin after my cvs up, so yes, you probably did fix the
problem.  Time to go find some champagne.

Skip

From guido at python.org  Wed Dec 31 19:14:16 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 19:14:29 2003
Subject: [Python-Dev] Re: Are we collecting benchmark results across machines
In-Reply-To: Your message of "Wed, 31 Dec 2003 15:56:00 PST."
	<Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU> 
References: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU> 
Message-ID: <200401010014.i010EGt06465@c-24-5-183-134.client.comcast.net>

> I suspect there are other folks who have run the pie-con benchmarks on
> their machines.  Perhaps we should construct a chart.  Times below are in
> seconds.
> 
> ~60	Guido's ancient 650Mhz Pentium
> ~27	Guido's desktop at work
> ~15	IBM T40 laptop
> 
> 88.550	dual 450 MHz Pentium 2
> 22.340	Athlon XP2800 (1243.363 MHz clock)
> 
> The latter two times are for Python 2.3.3 built out of the box using the
> Makefile (make time) shipped with the parrotbenchmarks ftp file on an
> unloaded machine. The times reported are the user time from the time
> triplet.

Perhaps we can turn this into a benchmark comparison chart.  In
particular, in my experience pystone is a pretty good indicator of
system performance even if it's a lousy benchmark.  I'll report the
pystone numbers for those same three systems:

home desktop: 10438.4 pystones/second
work desktop: 17421.6 pystones/second
laptop:       30198.1 pystones/second

Multiplying the pystone numbers with the parrotbench times should give
a constant if the benchmarks are equivalent.  But this doesn't look
good: I get

home desktop: 626304
work desktop: 470383
laptop:       452972

The home desktop is a clear outlier; it's more than twice as slow on
the parrot benchmark, but only 2/3rds slower on pystone...

(This is begging for a spreadsheet. :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 19:16:31 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 19:16:54 2003
Subject: [Python-Dev] Are we collecting benchmark results across machines
In-Reply-To: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU>
References: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU>
Message-ID: <a0601020cbc19173d675c@[172.24.18.98]>

At 3:56 PM -0800 12/31/03, Dennis Allison wrote:
>I suspect there are other folks who have run the pie-con benchmarks on
>their machines.  Perhaps we should construct a chart.  Times below are in
>seconds.
>
>~60	Guido's ancient 650Mhz Pentium
>~27	Guido's desktop at work
>~15	IBM T40 laptop
>
>88.550	dual 450 MHz Pentium 2
>22.340	Athlon XP2800 (1243.363 MHz clock)

Well, add in:

real    1m20.412s
user    1m1.580s
sys     0m2.100s

for this somewhat creaky 600MHz G3 iBook...
-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Wed Dec 31 19:26:03 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 19:26:09 2003
Subject: [Python-Dev] Are we collecting benchmark results across machines
In-Reply-To: Your message of "Wed, 31 Dec 2003 19:16:31 EST."
	<a0601020cbc19173d675c@[172.24.18.98]> 
References: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU> 
	<a0601020cbc19173d675c@[172.24.18.98]> 
Message-ID: <200401010026.i010Q3B06511@c-24-5-183-134.client.comcast.net>

> Well, add in:
> 
> real    1m20.412s
> user    1m1.580s
> sys     0m2.100s
> 
> for this somewhat creaky 600MHz G3 iBook...

I've been looking at user times only, but on that box the discrepancy
between user and real time is enormous!

It also suggests that a 600 MHz G3 and a 650 P3 are pretty close, and
contrary to what Apple seems to claim, the G3's MHz rating isn't worth
much more than a P3's MHz rating.

Could you run pystone too?

  python -c 'from test.pystone import main; main(); main(); main()'

and then report the smallest pystones/second value.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 19:34:43 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 19:34:56 2003
Subject: [Python-Dev] Are we collecting benchmark results across machines
In-Reply-To: <200401010026.i010Q3B06511@c-24-5-183-134.client.comcast.net>
References: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU> 
	<a0601020cbc19173d675c@[172.24.18.98]>
	<200401010026.i010Q3B06511@c-24-5-183-134.client.comcast.net>
Message-ID: <a0601020dbc191a3b1ac2@[172.24.18.98]>

At 4:26 PM -0800 12/31/03, Guido van Rossum wrote:
>  > Well, add in:
>>
>>  real    1m20.412s
>>  user    1m1.580s
>>  sys     0m2.100s
>>
>>  for this somewhat creaky 600MHz G3 iBook...
>
>I've been looking at user times only, but on that box the discrepancy
>between user and real time is enormous!

Yeah, that's not uncommon. I'm not sure if there's a problem with the 
time command, or if it's something else. This is a laptop (currently 
on battery power, though with reduced performance turned off) and 
it's got a 100MHz main memory bus, which is definitely a limiting 
factor once code slips out of L1 cache. It's faster than my Gameboy, 
but some days I wonder how much... :)

>It also suggests that a 600 MHz G3 and a 650 P3 are pretty close, and
>contrary to what Apple seems to claim, the G3's MHz rating isn't worth
>much more than a P3's MHz rating.

Possibly. I'm not sure it's necessarily the best machine for that 
comparison, given the power/performance tradeoffs with laptops. I may 
give it a whirl on one of the G3 desktop machines around here to see 
how much bus speed matters. (I won't be surprised, given the likely 
working set for an interpreter, to find that bus speed makes more of 
a difference than CPU speed)

>Could you run pystone too?
>
>   python -c 'from test.pystone import main; main(); main(); main()'
>
>and then report the smallest pystones/second value.

Smallest is:

Pystone(1.1) time for 50000 passes = 6.61
This machine benchmarks at 7564.3 pystones/second

-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Wed Dec 31 19:39:05 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 19:39:11 2003
Subject: [Python-Dev] Are we collecting benchmark results across machines
In-Reply-To: Your message of "Wed, 31 Dec 2003 19:34:43 EST."
	<a0601020dbc191a3b1ac2@[172.24.18.98]> 
References: <Pine.LNX.4.10.10312311544210.27868-100000@sumeru.stanford.EDU>
	<a0601020cbc19173d675c@[172.24.18.98]>
	<200401010026.i010Q3B06511@c-24-5-183-134.client.comcast.net> 
	<a0601020dbc191a3b1ac2@[172.24.18.98]> 
Message-ID: <200401010039.i010d5U06553@c-24-5-183-134.client.comcast.net>

> >Could you run pystone too?
> >
> >   python -c 'from test.pystone import main; main(); main(); main()'
> >
> >and then report the smallest pystones/second value.
> 
> Smallest is:
> 
> Pystone(1.1) time for 50000 passes = 6.61
> This machine benchmarks at 7564.3 pystones/second

Thanks -- though I made a mistake -- I should've asked for the largest
pystone value -- larger is better for pystone, unlike for running
times.  Anyway, this value is reasonable given your parrotbench time.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Wed Dec 31 19:44:49 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Wed Dec 31 19:41:14 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200312311754.hBVHsFx05650@c-24-5-183-134.client.comcast.ne
 t>
Message-ID: <5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch>

At 09:54 31.12.2003 -0800, Guido van Rossum wrote:


>I'd be happy to answer any questions.  Let the competition begin!

wondering whether the correctness checks are too much dependent on details 
of the representation of objects, especially the order of keys in a dict 
repr etc... 


From guido at python.org  Wed Dec 31 19:50:33 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 19:50:41 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Thu, 01 Jan 2004 01:44:49 +0100."
	<5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch> 
References: <5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch> 
Message-ID: <200401010050.i010oXA06606@c-24-5-183-134.client.comcast.net>

> wondering whether the correctness checks are too much dependent on details 
> of the representation of objects, especially the order of keys in a dict 
> repr etc... 

Probably. :-(

I could probably fix that by subclassing dict with something whose
repr() sorts the keys -- it doesn't print the few large dicts that it
uses, so this should be fine.  The same approach could be used for
fixing all repr()-related issues.  I'll leave it to Dan to decide if
he wants this to be fixed ASAP or if he's okay with putting it off
until later -- I imagine it will take a while before he's got it
running on Parrot well enough to hit the first assert...

(There's also an issue with the distinction between repr() of int and
long, which will disappear in Python 3.0, and repr() of str
vs. unicode, which already doesn't exist in Jython.  So I predict
that the benchmark currently doesn't have a chance of passing on
Jython.  (But it would be interesting to time it anyway -- simply
disable the raises in b0.check() and in b5.check().)

--Guido van Rossum (home page: http://www.python.org/~guido/)

From dan at sidhe.org  Wed Dec 31 20:16:00 2003
From: dan at sidhe.org (Dan Sugalski)
Date: Wed Dec 31 20:16:23 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200401010050.i010oXA06606@c-24-5-183-134.client.comcast.net>
References: <5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch>
	<200401010050.i010oXA06606@c-24-5-183-134.client.comcast.net>
Message-ID: <a0601020ebc19247f82de@[172.24.18.98]>

At 4:50 PM -0800 12/31/03, Guido van Rossum wrote:
>  > wondering whether the correctness checks are too much dependent on details
>>  of the representation of objects, especially the order of keys in a dict
>>  repr etc...
>
>Probably. :-(
>
>I could probably fix that by subclassing dict with something whose
>repr() sorts the keys -- it doesn't print the few large dicts that it
>uses, so this should be fine.  The same approach could be used for
>fixing all repr()-related issues.  I'll leave it to Dan to decide if
>he wants this to be fixed ASAP or if he's okay with putting it off
>until later -- I imagine it will take a while before he's got it
>running on Parrot well enough to hit the first assert...

Heh. I'm OK with either leaving it as-is if the current ordering is 
considered correct, or altering the benchmark to guarantee ordering. 
(Or having the output compared by an external program to guarantee 
correctness, which is fine as well)

-- 
                                         Dan

--------------------------------------"it's like this"-------------------
Dan Sugalski                          even samurai
dan@sidhe.org                         have teddy bears and even
                                       teddy bears get drunk

From guido at python.org  Wed Dec 31 20:21:27 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 20:21:34 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Wed, 31 Dec 2003 20:16:00 EST."
	<a0601020ebc19247f82de@[172.24.18.98]> 
References: <5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch>
	<200401010050.i010oXA06606@c-24-5-183-134.client.comcast.net> 
	<a0601020ebc19247f82de@[172.24.18.98]> 
Message-ID: <200401010121.i011LRP06664@c-24-5-183-134.client.comcast.net>

> >> wondering whether the correctness checks are too much dependent
> >> on details of the representation of objects, especially the order
> >> of keys in a dict repr etc...
> >
> >Probably. :-(
> >
> >I could probably fix that by subclassing dict with something whose
> >repr() sorts the keys -- it doesn't print the few large dicts that it
> >uses, so this should be fine.  The same approach could be used for
> >fixing all repr()-related issues.  I'll leave it to Dan to decide if
> >he wants this to be fixed ASAP or if he's okay with putting it off
> >until later -- I imagine it will take a while before he's got it
> >running on Parrot well enough to hit the first assert...
> 
> Heh. I'm OK with either leaving it as-is if the current ordering is 
> considered correct, or altering the benchmark to guarantee ordering. 
> (Or having the output compared by an external program to guarantee 
> correctness, which is fine as well)

The ordering is supposed to be an unimportant detail.  The external
program solution won't work due to the nature of the benchmark.  I'll
fix the benchmark some time next week.  The fix should be fairly
simple and localized.

--Guido van Rossum (home page: http://www.python.org/~guido/)

From pedronis at bluewin.ch  Wed Dec 31 20:39:00 2003
From: pedronis at bluewin.ch (Samuele Pedroni)
Date: Wed Dec 31 20:35:31 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: <200401010050.i010oXA06606@c-24-5-183-134.client.comcast.ne
 t>
References: <Your message of "Thu, 01 Jan 2004 01:44:49 +0100."
	<5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch>
	<5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch>
Message-ID: <5.2.1.1.0.20040101023157.02851b70@pop.bluewin.ch>

At 16:50 31.12.2003 -0800, Guido van Rossum wrote:

>(There's also an issue with the distinction between repr() of int and
>long, which will disappear in Python 3.0, and repr() of str
>vs. unicode, which already doesn't exist in Jython.  So I predict
>that the benchmark currently doesn't have a chance of passing on
>Jython.  (But it would be interesting to time it anyway -- simply
>disable the raises in b0.check() and in b5.check().)

Jython is still not up to run it. But yes it would be an interesting thing.

Indeed in any case some other things wrt representation would be a problem 
still:

Jython 2.2a0 on java1.4.0_02 (JIT: null)
Type "copyright", "credits" or "license" for more information.
 >>> u'a'
'a'
 >>> def f(): pass
...
 >>> f
<function f 1>
 >>> class X:
...  pass
...
 >>> X
<class __main__.X 2>
 >>>
...

In general <...> reprs vary.




From guido at python.org  Wed Dec 31 23:44:01 2003
From: guido at python.org (Guido van Rossum)
Date: Wed Dec 31 23:44:07 2003
Subject: [Python-Dev] Pie-thon benchmark code ready
In-Reply-To: Your message of "Thu, 01 Jan 2004 02:39:00 +0100."
	<5.2.1.1.0.20040101023157.02851b70@pop.bluewin.ch> 
References: <Your message of "Thu, 01 Jan 2004 01:44:49 +0100."
	<5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch>
	<5.2.1.1.0.20040101014159.03a31f50@pop.bluewin.ch> 
	<5.2.1.1.0.20040101023157.02851b70@pop.bluewin.ch> 
Message-ID: <200401010444.i014i1o06815@c-24-5-183-134.client.comcast.net>

> Jython is still not up to run it. But yes it would be an interesting thing.

Maybe you could find a sponsor for Jython development...?

> Indeed in any case some other things wrt representation would be a problem 
> still:
> 
> Jython 2.2a0 on java1.4.0_02 (JIT: null)
> Type "copyright", "credits" or "license" for more information.
>  >>> u'a'
> 'a'
>  >>> def f(): pass
> ...
>  >>> f
> <function f 1>
>  >>> class X:
> ...  pass
> ...
>  >>> X
> <class __main__.X 2>
>  >>>
> ...
> 
> In general <...> reprs vary.

But I've seen enough people write code that parses <...> reprs in some
way to make me think that maybe they should be standardized somewhat,
at least to the point where different Python implementations should
not differ gratuitously.  E.g. Jython could be much closer to CPython
by inserting 'at 0x'.  It's not like standardizing this would close
off an important implementation freedom for other Python
implementation.  (I won't go as far as requiring that the number
should be the same as hex(id(x)). :-)

--Guido van Rossum (home page: http://www.python.org/~guido/)