I posted last week about a need-for-speed patch that broke PEP 302
compliance, and asked if it should be fixed or reverted. I got exactly one
response which said "yes, it should be fixed or reverted", which
unfortunately didn't answer my question as to which one we should do. :)
If we don't revert it, there are two ways to fix it. One is to just change
PEP 302 so that the behavior is unbroken by definition. :) The other is
to actually go ahead and fix it by adding PathImporter and NullImporter
types to import.c, along with a factory function on sys.path_hooks to
create them. (This would've been the PEP-compliant way to implement the
So, "fix" by documentation, fix by fixing, or fix by reverting? Which
should it be?
From the initial bugreport
Various parts of cgi.FieldStorage call its
"read_lines_to_outerboundary", "read_lines" and
"skip_lines" methods. These methods use the
"readline" method of the file object that represents an
input stream. The input stream is typically data
supplied by an untrusted source (such as a user
uploading a file from a web browser). The input data
is not required by the RFC 822/1521/1522/1867
specifications to contain any newline characters. For
example, it is within the bounds of the specification
to supply a a multipart/form-data input stream with a
"file-data" part that consists of a 2GB string composed
entirely of "x" characters (which happens to be
something I did that led me to noticing this bug).
This bug has been around for about a year but I just worked up a
patch yesterday that applies OK against current SVN. It's attached
to the issue. Would someone be so kind as to check it in? Guido has
already reviewed it, I believe.
Currently, we have two running tracker demos online:
These installation are in various forms of demo mode and
"pre-release" (meaning that the configuration is still not
complete). They both use the sample data that Fredrik Lundh
produced at some point, so don't be surprised that they
are behind SF wrt. content.
While these might not be in the final form of operation,
I think users should already try to use them, to find
out which one they like best.
Discussions/Comments can be sent to infrastructure(a)python.org,
however, for reports/reviews, please use the Wiki at
You'll notice that it also lists Trac and Malone, however,
it seems that there is no progress on importing SF data
It seems that the pre-2.5 struct module has some additional
undocumented behavior that didn't percolate into the new version:
Python 2.4 and previous will coerce floats to integers when necessary
as such without any kind of complaint:
$ python2.4 -c "import struct; print repr(struct.pack('>H',
Python 2.5 refuses to coerce float to int:
$ python2.5 -c "import struct; print repr(struct.pack('>H',
Traceback (most recent call last):
File "<string>", line 1, in <module>
File "/Users/bob/src/python/Lib/struct.py", line 63, in pack
TypeError: unsupported operand type(s) for &: 'float' and 'long'
The available options are to:
1. Reinstate the pre-2.5 weirdness
2. Reinstate the pre-2.5 weirdness with a DeprecationWarning
3. Break existing code that relies on undocumented behavior (seems
more like a bug than lack of specification)
Either 2 or 3 seems reasonable to me, with a preference for 3 because
none of my code depends on old bugs in the struct module :)
As far as precedent goes, the array module *used* to coerce floats
silently, but it's had a DeprecationWarning since at least Python 2.3
(but perhaps even earlier). Maybe it's time to promote that warning
to an exception for Python 2.5?
 The pre-2.5 behavior should really be considered a bug, the
documentation says "Return a string containing the values v1, v2, ...
packed according to the given format. The arguments must match the
values required by the format exactly." I wouldn't consider arbitrary
floating point numbers to match the value required by an integer
format exactly. Floats are not in general interchangeable with
integers in Python anyway (e.g. list indexes, etc.).
There is an oversight in the design of __index__() that only just
surfaced :-( It is responsible for the following behavior, on a 32-bit
machine with >= 2GB of RAM:
>>> s = 'x' * (2**100) # works!
This is because PySequence_Repeat(v, w) works by applying w.__index__ in
order to call v->sq_repeat. However, __index__ is defined to clip the
result to fit in a Py_ssize_t. This means that the above problem exists
with all sequences, not just strings, given enough RAM to create such
sequences with 2147483647 items.
For reference, in 2.4 we correctly get an OverflowError.
Argh! What should be done about it?
Is it intentional that Python 2.5 is (currently) shipping with
distutils 2.4.0, while Python 2.4 (at least 2.4.1, 2.4.2 and 2.4.3)
shipped with distutils 2.4.1? Judging from my own tests, distutils
2.4.1 fixed several bugs that some of my test suites depend on (the
fixes, not the bugs ; ).
I'm trying to write a test for my Socket Timeouts patch , which fixes
signal handling (notably Ctl-C == SIGINT == KeyboarInterrupt) on socket
operations using a timeout. I don't see a portable way to send a signal,
and asking the test runner to press Ctl-C is a non-starter. A "real"
signal is needed to interrupt the select() (or equivalent) call, because
that's what wasn't being handled correctly. The bug should happen on the
other platforms I don't know how to test on.
Is there a portable way to send a signal? SIGINT would be best, but
another signal (such as SIGALRM) would do, I think.
If not, should I write the test to only work on systems implementing
SIGALRM, the signal I'm using now, or implementing kill(), or what?
I suggest that there be a third beta release and that we then wait just
a bit before going final.
The bugs that were found and fixed in the first two beta releases
suggest that Py2.5 is not yet as stable as we would like. Over the next
few days, I'll try to run it on as much third-party code as possible.
That would have detected the recently surfaced grammar error a little
bit earlier (the one where "for x, in listOfTuples" would not unpack).
The release process itself is going well but I don't think the pervasive
AST changes have been fully shaken-out yet.
Here is a first stab at writing up guidelines for people to follow
when reporting bug. If this goes well I will also do ones for
patches, committing, and PEPs.
These sets of guidelines are to help you file a bug report for the Python
programming language on SourceForge_. If your bug is not for the
language but for a third-party application, please report the bug to
*Please make sure to follow every step as it will make the lives
of the Python developers much easier!!!*
Get a SourceForge account
In order to file a bug report, you must have an account_ on SourceForge_. We
realize some people would like to have anonymous bug reporting for various
reasons (anonymity, ease or reporting, etc.). But SourceForge does not support
anonymous reporting. Plus, by registering, you are notified by email when any
action is been taken on your report. This can be very important if a Python
developer needs more information from you about the bug.
Start a new bug
You must be logged into SourceForge to file a bug! See `Get a SourceForge
account`_ if you do not have one.
Go to the `SourceForge bug page`_ to start a new bug report. There you will
find a link called `Submit New`_. Click on it and it will allow you
to fill out a new
Once you click on the link, you are presented with a page that has several
fields for you to fill in. Here is what to do for each field:
Set this to the area that the bug is related to (e.g.,
documentation, build, etc.).
Usually this is set the major.minor version of Python that you
found the bug in.
* Assigned To
Leave this alone
Leave this alone
A one-line describing the problem so as to make it easy for
developers to spot whether they have the expertise needed to work on
* Detailed Description
Following sections of this document discuss what should go in here.
* Check to Upload and Attach a File
If you are going to upload a file, you *must* check this box.
* <File Location Field>
Click the Browse button to upload any file to accompany your bug
report (usually a succinct way to reproduce the bug).
* File Description
A one-line describing the file; no date info is needed since the
upload is timestamped.
Specify Python version
It is important that we have the most accurate version number of the
interpreter you are using in order to best diagnose the issue. There are two
ways to get us the version information.
If you can run your Python interpreter, execute the following lines at an
interpreter and paste the result into the ``Detailed Description``
field of the bug report::
>>> import sys
>>> print sys.version
If you are running a version of Python newer than 2.4 and are working from a
source checkout of Python, the please also report the Subversion revision
number for the root of your checkout::
python/trunk$ svnversion .
If your bug is preventing you from running the interpreter, execute Python with
teh ``-V`` command-line flag and paste the output::
python/trunk$ python -V
Special settings for your Python interpreter
Sometimes your environment influences a bug and thus needs to be
reported to help find the problem. This means we need to have
* Operating System
* Environment Variables
If this is set and might be causing the issue, please either
upload the file or state what it does.
If your bug is on Windows and involves importing, please
report if this environment variable is set or not.
If you have andy third-party packages installed that may be
contributing to the bug, please report those.
* Custom Patches
Any differences between your code and the code the Python
developers work off of needs to be reported.
Sample code to reproduce bug
If you can, please upload a file the demonstrates the bug. The more
succinct the better! And please do not forget to check the upload
checkbox in the bug report.
At this point you should have a detailed bug report for developers to
work off of. Click the ``Submit`` button and read on to see what you
should do after the bug is reported.
Respond to requests from developers
No matter how detailed the bug report, there is always the off-chance
that a developer will need more information to fix a bug.
Please be prompt in replying to requests for information by submitting
a reply on the bug report.
You may be asked to test out a patch. It is very important that you
help with this, especially if the bug is not reproducible by the
developer working on it.
Write a patch!
If you are so inclined, patches for your own bug reports are always
helpful! Please make sure to reference the tracker item # in the
.. _SourceForge: http://www.sourceforge.net/
.. _SourceForge bug page:
.. _account: http://sourceforge.net/account/newuser_emailverify.php
.. _Submit New: