[Python-Dev] python-dev Summary for 2003-01-01 through 2003-01-15
Brett Cannon
brett@python.org
Thu, 16 Jan 2003 14:51:14 -0800 (PST)
Here is the rough draft. Have a look and let me know where I botched
things.
+++++++++++++++++++++++++++++++++++++++++++++++++++++
python-dev Summary for 2003-01-01 through 2002-01-15
+++++++++++++++++++++++++++++++++++++++++++++++++++++
======================
Summary Announcements
======================
This summary has been written in the first-person. Aahz had suggested I
write in third-person, but I just can't stand it. =)
A new project named `Minimal Python`_ whose goal is to write a Python
interpreter using a minimal amount of C and maximizing the amount of code
written in Python. As of now it is just a mailing list but some
interesting points have already been made and it has some major Python
programmers involved with it so it will more than likely succeed.
Leaving out `Broken strptime in Python 2.3a1 & CVS`_ for two reasons.
One, the only lesson in that thread is that when C does not give you a
good standard, rewrite the code in Python and come up with one. Two,
since the whole thing revolves around my code I would be more biased than
I normally am. =)
Also, some of the threads have notes with them mentioning how I could not
find them in the Mailman archive.
.. _Minimal Python: http://codespeak.net/mailman/listinfo/pypy-dev
.. _Broken strptime in Python 2.3a1 & CVS:
http://mail.python.org/pipermail/python-dev/2003-January/031983.html
===================================================
`PEP 303: Extend divmod() for Multiple Divisors`__
===================================================
__ http://mail.python.org/pipermail/python-dev/2002-December/031511.html
Splinter threads:
- map, filter, reduce, lambda (can't find thread in Mailman
archive)
As mentioned in the `last summary`__, this thread sparked a huge response
to the built-ins in Python (if you don't remember what all the built-ins
are, just execute ``dir(__builtins__)`` in the interpreter). It started
with Christian Tismer calling for the deprecation of ``divmod()`` since
he "found that divmod compared to / and % is at least 50 percent *slower*
to compute", primarily because ``divmod()`` is a method and the overhead
that comes from being that. Guido responded and agree that, in hindsight,
"Maybe it wasn't such a good idea". But since it isn't broken he is not
for replacing it since "At this point, any change causes waste". Besides,
Tim Peters came up with the great compromise of removing " divmod() when
[Tim's] dead. Before then, it stays."
Guido would rather have the energy spent on adding the ability for the
language to recognize that it "is a built-in so it can generate more
efficient code". This led to Christian suggesting changing Python so that
built-ins like ``divmod()`` that can easily be written in Python be done
so and thus remove the C version without any backwards-compatibility
issues. Guido's response was to "first make some steps towards the better
compiler [Guido] alluded to. Then we can start cutting."
Tim in an email said that "If we have to drop a builtin, I never liked
reduce <wink>". This then led to Barry Warsaw saying he could stand to
lose ``apply()``. Then Raymond Hettinger nominated ``buffer()`` and
``intern()`` as things that could stand to go away. Michael Hudson
suggested ``map()`` and ``filter()``.
Guido said ``apply()`` could stand getting a PendingDeprecationWarning
since the new calling conventions (``fxn(*args, **kwargs)``) replace
``apply()``'s use. He almost mentioned how he wanted something like
``foo(1, 2, 3, *args, 5, 6, 7, **kw)`` since the ``5, 6, 7`` is currently
not supported.
But the call for the removal of ``lambda`` is what really set things off.
People in support of the removal said that nested scoping removed a huge
reason for having ``lambda``. But functional programmers said that they
wouldn't let ``lambda`` be taken away. It was agreed it had its uses, but
the name was not relevant to its use and its syntax is non-standard and
should be changed in Py3K. Someone suggested renaming it ``def``, but
Guido said no since "Python's parser is intentionally simple-minded and
doesn't like having to look ahead more than one token" and using ``def``
for ``lambda`` would break that design. Barry suggested ``anon`` which
garnered some support.
It was also suggested to have ``lambda`` use parentheses to hold its
argument list like a good little function.
And then all of this led to a discussion over features in Python that are
not necessarily easy to read but are powerful. Some said that features
such as list comprehensions, the ``*/**`` calling syntax, etc., were not
good since they are not syntactically obvious. This was argued against by
saying they are quite useful and are not needed for basic programming.
.. _last summary:
http://www.python.org/dev/summary/2002-12-16_2002-12-31.html
==================
`Holes in time`__
==================
__ http://mail.python.org/pipermail/python-dev/2003-January/031836.html
Tim Peters brought up an inherent problem with the new datetime_ type and
dealing with timezones and daylight savings. Imagine when DST starts; the
clock jumps from 1:59 to 3:00. Now what happens if someone enters a time
that does not use DST? The implementation has to handle the possibility
of something inputting 2:00 and instantly pushing forward an hour. But
what is worse is when DST ends. You go from 1:59 back to 1:00! That
means you pass over everything from 1:00 to 1:59 twice during that day.
Since it is flat-out impossible to tell, what should be done in this case?
Originally ``datetime`` raised ValueError. It seems to be staying that
way (after a *very* lengthy discussion clarifying the whole situation).
If you want to read a paper on the history of time go to
http://www.naggum.no/lugm-time.html as recommended by Neil Schemenauer.
.. _datetime: http://www.python.org/dev/doc/devel/lib/module-datetime.html
===========================================
`no expected test output for test_sort?`__
===========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/031889.html
This thread was originally started by Skip Montanaro to try to figure out
why ``test_sort`` was failing to catch an error when run through
``regrtest.py``. But Skip quickly realized ``regrtest.py`` was using the
wrong directories.
What makes this thread worth mentioning is a question I posed about
whether it was worth converting tests over to PyUnit. Guido said not for
the sake of converting, but if you were going to be improving upon the
tests, then moving a testing suite over to PyUnit was a good idea. It was
also said that all new testing suites should use PyUnit (although doctest
is still acceptable, just not preferred). Walter Drwald started `patch
#662807`_ to use for converting tests to PyUnit. So give a hand if you
care to.
.. _patch #662807: http://www.python.org/sf/662807
============
binutils
============
(Can't find in Mailman archive)
Andrew Koenig told python-dev of a problem with binutils_ 2.13 way back in
the `2002-089-16 through 2002-09-01 summary`_. Well, it appears that
2.13.2 fixes the issues.
.. _2002-089-16 through 2002-09-01 summary:
http://www.python.org/dev/summary/2002-08-16-2002-09-01.html
===========================
The meaning of __all__
===========================
(Can't find in Mailman archive)
Jack Jansen asked what the proper use of ``__all__``; a discussion on the
PyObjC_ mailing list sparked this question. The answer is that
``__all__`` is used for the public API of a module.
.. _PyObjC: http://pyobjc.sourceforge.net/
======================================
PEP 301 implementation checked in
======================================
(Can't find in Mailman archive)
AM Kuchling announced that the `PEP 301`_ implementation by Richard Jones.
The web server handling everything is at `amk.ca`_ off of a DSL line so
don't hit it too hard. An example of how to change a Distutils
``setup.py`` file is given in the e-mail.
.. _PEP 301: http://www.python.org/peps/pep-0301.html
.. _amk.ca: http://www.amk.ca/cgi-bin/pypi.cgi
=======================================
bz2 problem deriving subclass in C
=======================================
(Can't find in Mailman archive)
This thread was originally about a patch by Neal Norwitz and how to
properly deal with deallocating in C code. Apparently this has nipped
several people in the bum and has changed since the introduction of
new-style classes. It appears that if you are deallocating a base type,
you call its ``tp_dealloc()`` function; if it is a new-style class, you
call ``self->ob_type->tp_free()``.
====================
`Cross compiling`__
====================
__ http://mail.python.org/pipermail/python-dev/2003-January/031848.html
(Link only part of thread; rest missing)
Splinter threads:
- `Bastion too
<http://mail.python.org/pipermail/python-dev/2003-January/031851.html>`__
- `What attempts at security should/can Python implement?
<http://mail.python.org/pipermail/python-dev/2003-January/031871.html>`__
- `Whither rexec?
<http://mail.python.org/pipermail/python-dev/2003-January/031842.html>`__
- `tainting
<http://mail.python.org/pipermail/python-dev/2003-January/031960.html>`__
Originally a thread about cross-compilation on OS X to Windows by Timothy
Wood, this ended up changing into a thread about security in Python.
As has come up multiple times before, rexec_'s lack of actual security for
new-style classes came up. This then led to a question of whether
Bastion_ was secure as well. The answer was no and have now been
subsequently sabotaged on purpose by Guido; they now require you to
manually edit the code to allow them to work.
A discussion on how to make Python secure came up. Various points were
made such as how to deal with introspection and how people might try to
get around being locked out of features. Tainting was also mentioned for
strings.
If you need some security in your app, take a look at mxProxy_. Its
design is similar to how Zope3_ handles security.
.. _rexec: http://www.python.org/doc/current/lib/module-rexec.html
.. _Bastion: http://www.python.org/doc/current/lib/module-Bastion.html
.. _mxProxy: http://www.egenix.com/files/python/mxProxy.html
.. _Zope3:
http://dev.zope.org/Wikis/DevSite/Projects/ComponentArchitecture/FrontPage
===========================
`tutor(function|module)`__
===========================
__ http://mail.python.org/pipermail/python-dev/2003-January/031893.html
(Only part of thread; rest missing from archive)
Christopher Blunk proposed the idea of having a built-in ``examples()``
function or something similar that would spit out example code on how to
use something. There was a discussion as to whether it was reasonable for
Python to try to keep up a group of examples in the distribution. It was
agreed, though, that more code examples would be good to have *somewhere*.
One place is in the Demos_ directory of the source distribution. Problem
with that is it is only available in the source distribution and it is not
vigorously maintained. Both of these are solvable, but it didn't look
like anyone was taking the reigns to cause this to happen.
Another was to push people to give to an online repository (such as the
`Python Cookbook`_) to keep it centralized and accessible by anyone.
Problem with that is it creates a dependency on something outside of
PythonLab's control.
The last option was adding it to the module documentation. This is the
safest since it means it becomes permanent and everyone will have access
to it. But it has the same drawback as all of these other suggestions;
you can't access it in the interpreter as easily as you can ``help()``.
The thread ended with no conclusion beyond cleaning up the ``Demos``
directory and adding more examples to the official documentation would be
nice.
.. _Python Cookbook: http://aspn.activestate.com/ASPN/Python/Cookbook/
=========================================
`PEP 297: Support for System Upgrades`__
=========================================
__ http://mail.python.org/pipermail/python-dev/2003-January/031838.html
MA Lemburg asked if Guido would be willing to let an implementation of
`PEP 297`_ into Python 2.3. He said yes and would also be willing to add
it to 2.2.3 and even 2.1.4 if that version ever comes about.
A discussion of how to handle the implementation began. The agreed upon
directory name became ``site-package-<Python-version>`` with ``<Python
version>`` being the full version: major.minor.micro. ``site.py`` will be
modified so that if the current version of Python is different from the
version of the directory that it won't add it to ``sys.path``.
There was the brief idea of deleting the directory as soon as the version
did not match, but that was thought of being harsh and unneeded.
.. _PEP 297: http://www.python.org/peps/pep-0297.html
================================
`PyBuffer* vs. array.array()`__
================================
__ http://mail.python.org/pipermail/python-dev/2003-January/031841.html
(Only part of thread; rest missing from archive)
Splinter threads:
- `Slow String Repeat
<http://mail.python.org/pipermail/python-dev/2003-January/031895.html>`__
Bill Bumgarner asked whether ``PyBuffer*`` or ``array.array()`` was the
proper thing to use because one is like a huge string and the other is
unsigned ints. Guido basically said to just use strings like everyone
else for byte storage. Bill said he would just live with the dichotomy.
This thread also brought up a question of performance for the code
``array.array('B').fromlist([0 for x in range(0, width*height*3)])``. Was
cool to watch various people take stabs as speeding it up. Written in
Python, the fast way was to change it to ``array.array('B',
[0])*width*height*3``; doing several decent-sized calls minimized
``memcpy()`` calls in the underlying C code. Raymond Hettinger came up
with a fast way to do it in C.
==========================
`new features for 2.3?`__
==========================
__ http://mail.python.org/pipermail/python-dev/2003-January/031837.html
Neal Norwitz wanted to get the tarfile_ module into Python 2.3. This led
to Guido saying he wanted to get Docutils_ and better pickling for
new-style classes in as well.
The Docutils idea was shot down because it was stated it currently is not
ready for inclusion. 2.4 is probably the version being shot for.
The pickle idea raised security ideas (this is becoming a trend for some
reason on the list lately). In case you didn't know, you shouldn't run
pickled code you don't trust.
.. _tarfile: http://python.org/sf/651082
================
`sys.path[0]`__
================
__ http://mail.python.org/pipermail/python-dev/2003-January/031896.html
Thomas Heller had a question about the documented handling of
``sys.path[0]``. This quickly led to the point that the value, when
specified and not the empty string, is a relative path; not necessarily
the desired result. And so `patch #664376`_ was opened to make it be
absolute.
.. _patch #664376: http://www.python.org/sf/664376
===================
`Misc. warnings`__
===================
__ http://mail.python.org/pipermail/python-dev/2003-January/031884.html
MA Lemburg noticed that ``test_ossaudiodev`` and ``test_bsddb3`` were not
expected skips on his Linux box. He wondered why they were not exepected
to be skipped on his platform. This led to a discussion over the expected
skip set and how it is decided what is going to be skipped (experts on the
various OSs make the call).
==============================
`Raising string exceptions`__
==============================
__ http://mail.python.org/pipermail/python-dev/2003-January/031907.html
It looks like string exceptions are going to raise
PendingDeprecationWarning; you have been warned.
======================
`PEP 290 revisited`__
======================
__ http://mail.python.org/pipermail/python-dev/2003-January/032003.html
Kevin Altis was cleaning up wxPython_ and all the references to the
``string`` module and pointed out that the stdlib probably could stand to
have a similar cleaning as well. Guido said, though, that it is better to
choose a module you are familiar with and do a complete style clean-up
instead of doing a library-wide clean-up of a single thing.
.. _wxPython: http://www.wxpython.org/
============================
`Assignment to __class__`__
============================
__ http://mail.python.org/pipermail/python-dev/2003-January/032016.html
Kevin Jacobs got bitten by the change to non-heap objects that prevents
the assigning to ``__class__`` (it was recently backported to the 2.2
branch). He asked if there was any way to bring it back; Guido said no
because of various layout issues in the C struct and such.
=================================
`Parallel pyc construction`__
=================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032060.html
Paul Dubois ran into an problem on a 384 processor system with .pyc file
corruption. He wanted a way to just skip the creation of .pyc files
entirely since his programs run for so long that initial start-up times
are inconsequential and that is the point of .pyc files. Neal Norwitz is
working on `patch #602345`_ to implement a command-line option.
.. _patch #602345: http://www.python.org/sf/602345
==============================================
`Extension modules, Threading, and the GIL`__
==============================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032017.html
This thread was briefly mentioned in the `last summary`_. But it had
changed into a discussion of Python external thread API. Mark Hammond
commented how he had bumped up against its limitations in multiple
contexts. Mark thought it would be useful to define an API where one
wanted the ability to say "I'm calling out to/in from an external API".
This would be a large project, though, and so Mark was not up to doing it
on his own. David Abrahams, though, stepped up and said he was willing to
help. So did Anthony Baxter.
Tim Peters had his own twist by wanting to be able to call from a thread
into a Python thread without knowing a single thing about that state of
Python at that point, e.g. don't even know that that interpreter is done
initializing. Mark saw the way to go about solving all of this by writing
a PEP, write a new C API to solve this issue that is optional and used by
extension modules wanting to use the new features, and then define a new
TLS (Thread Local Storage; thank you acronymfinder.com). Mark thought the
TLS would only be needed for thread-state. Someone suggested ACE_ as a
TSS (Thread State Storage) implementation which can cover all the
platforms that Python needs covered. But Mark suggested coming up with a
"pluggable TLS" design.
.. _ACE: http://doc.ece.uci.edu/Doxygen/Beta/html/ace/classACE__TSS.html
============================================
`Interop between datetime and mxDateTime`__
============================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032100.html
MA Lemburg expressed interest in trying to get mxDateTime_ to play nice
with datetime_. The idea of having a base type like the new basestring in
Python 2.3 was suggested. But there was pickling issues with datetime, MA
wanting to keep mxDateTime compatible with Python 1.5.2, and getting
everyone to work in a consistent way through the base type. Another
instance where interfaces would be handy.
.. _mxDateTime: http://www.lemburg.com/files/python/mxDateTime.html
===========================
`properties on modules?`__
===========================
__ http://mail.python.org/pipermail/python-dev/2003-January/032106.html
Neil Schemenauer wanted to be able to make module variables be properties.
Samuele Pedroni stepped up and said the issue for this to work was "about
instance-level property apart the module thing details". He also pointed
out that it would mean you could directly import a property which would
cause unexpected results, thus killing the idea. Guido was disappointed
he didn't get to wield his BDFL pronouncement stick and smack the idea
down <wink>.
======================================
`Fwd: Re: PEP 267 improvement idea`__
======================================
__ http://mail.python.org/pipermail/python-dev/2003-January/032165.html
Oren Tirosh forwarded an e-mail from python-list@python.org to python-dev
about `PEP 267`_ and various strategies. One was using negative keys for
signaling guaranteed key lookup failure. Oren also played with inlining.
He also said that since interned strings are now out of Python as of 2.3
(thanks to Oren) that the namespace could "have only interned strings as
entry keys and avoid the need for a second search pass on first failed
lookup". He even considered creating a special dict type for namespaces.
.. _PEP 267: http://www.python.org/peps/pep-0267.html