I am going through and running the entire test suite using importlib
to ferret out incompatibilities. I have found a bunch, although all
rather minor (raising a different exception typically; not even sure
they are worth backporting as anyone reliant on the old exceptions
might get a nasty surprise in the next micro release), and now I am
down to my last failing test suite: test_import.
Ignoring the execution bit problem (http://bugs.python.org/issue6526
but I have no clue why this is happening), I am bumping up against
TestPycRewriting.test_incorrect_code_name. Turns out that import
resets co_filename on a code object to __file__ before exec'ing it to
create a module's namespace in order to ignore the file name passed
into compile() for the filename argument. Now I can't change
co_filename from Python as it's a read-only attribute and thus can't
match this functionality in importlib w/o creating some custom code to
allow me to specify the co_filename somewhere (marshal.loads() or some
My question is how important is this functionality? Do I really need
to go through and add an argument to marshal.loads or some new
function just to set co_filename to something that someone explicitly
set in a .pyc file? Or I can let this go and have this be the one
place where builtins.__import__ and importlib.__import__ differ and
just not worry about it?
At 02:57 PM 8/31/2009 -0700, Brett Cannon wrote:
>Ignoring that 'new' is not in Python 3.x (luckily 'types' is), I want
>a proper solution that doesn't require reconstructing every code
>object that I happen to import.
Practicality beats purity. ;-) (Especially if it allows importlib
to run on older Pythons.)
Also, surely you're not worried about *performance* here?
At 01:39 PM 8/31/2009 -0700, Brett Cannon wrote:
>On Mon, Aug 31, 2009 at 12:59, Brett Cannon<brett(a)python.org> wrote:
> > On Mon, Aug 31, 2009 at 12:27, Antoine Pitrou<solipsis(a)pitrou.net> wrote:
> >> Brett Cannon <brett <at> python.org> writes:
> >>> I will plan to take this approach then;
> >>> http://bugs.python.org/issue6811 will track all of this. Since this is
> >>> a 3.2 thing I am not going to rush to implement this.
> >> I still don't understand what the point is of this complicated
> approach (adding
> >> an argument to marshal.load()) compared to the simple and obvious approach
> >> (making co_filename mutable).
> > If we add the argument to marshal.load* we can eventually drop the
> > file location string from marshal data entirely by requiring people to
> > specify the filename to use when the code object is created. Making
> > co_filename mutable simply doesn't allow for this case unless we
> > decide a default value should be used instead.
>I should also mention that I am +0 on the marshal.load* change. I
>could be convinced to try to pursue a mutable co_filenme direction,
>but considering the BDFL likes the marshal.load* approach and it opens
>up the possibility of compacting the marshal format I am leaning
>towards sticking with this initial direction.
Why not just try the code I posted earlier, that doesn't need a
mutable attribute OR an API change?
At 09:33 AM 8/31/2009 -0700, Guido van Rossum wrote:
>Of course, tracking down all the code objects in the return value of
>marshal.load*() might be a bit tricky -- API-wise I still think that
>making it an argument to marshal.load*() might be simpler. Also it
>would preserve the purity of code objects.
Or maybe we could just do something like this:
from new import code
def with_changed_filename(code_ob, filename):
if not isinstance(ob, code):
ob.co_argcount, ob.co_nlocals, ob.co_stacksize,
map(remap, ob.co_consts), ob.co_names,
ob.co_name, ob.co_firstlineno, ob.co_lnotab, ob.co_freevars,
Granted, this takes a bit more memory than an in-place modification,
but it's immediately usable and at least works wherever new.code is available.
(I've not tested the above, so it may not work. I seem to recall the
last time I wrote something like this there was something tricky
about handling co_freevars and co_cellvars; I think you may need to
omit them if empty, or convert them to None, or from None to an empty
tuple or some such rigamarole. And a 3.x version is left as an
exercise for the reader. ;-) )
Would anyone object if I removed the deletion of of
sys.setdefaultencoding in site.py?
I'm guessing "yes!" so thought I'd state my reasons now:
This deletion appears to be pretty flimsy; reload(sys) and you have it
back. Which is lucky, because I need it after it's been deleted...
Why? Well, because you can no longer put sitecustomize.py in a
project-specific location (http://bugs.python.org/issue1734860) and
because for some projects the only way I can deal with encoded strings
sensibly is to use setdefaultencoding, in my case at the start of a
script generated by zc.buildout's zc.recipe.egg (I *know* all the
encodings in this project are utf-8, but I don't want to go playing
whack-a-mole with whatever modules this rather large project uses that
haven't been made properly unicode aware).
Yes, it needs to be used as early as possible, and the docs should say
this, but deleting it seems to be petty in terms of stopping its use
when sitecustomize.py is too early and too system-wide and spraying
.decode('utf-8')'s all over a code base made up of a load of eggs
managed by buildout simply isn't feasible...
Simplistix - Content Management, Batch Processing & Python Consulting
I'm being bitten by this issue:
I'm not sure I agree with Daniel's closing of it so thought I'd ask here...
Am I right in thinking that the general idea is that "the current
working directory at the time of invoking a script or interpreter ends
up on the python path" or should I be thinking "the directory that a
script exists in should end up on the python path"?
If the latter, then what happens in the case of just starting up an
If neither, then how come when I have two .py files in a directory, I
can import one as a module from the other?
In any case, as a parting comment, http://bugs.python.org/issue1232023
seems to have been committed with no tests and the only documentation
being a one liner in the NEWS.txt file. Was there other discussion of this?
(Incidentally, export PYTHONPATH= or its Windows equivalent circumvents
whatever the patch was trying to achieve, so the change doesn't seem to
make sense anyway...)
Simplistix - Content Management, Batch Processing & Python Consulting
I have implemented the simple zip decryption in C (yes, the much loathed
weak password based classical
PKWARE encryption, which incidentally is the only one currently supported in
It performs fast, as one would expect, as compared to the current all-python
Does it sound worthy enough to create a patch for and integrate into python
Senior Undergraduate, Department of Computer Science and Engineering
Indian Institute of Technology Bombay
Here's a module "timed_command" I wrote a while ago and is generally
useful and might be a good addition to the standard library. It is
like commands.getstatusoutput but lets you run a command with an
optional timeout. Useful for systems programming where a sub-process
might hang. Only works on POSIX, but could perhaps be modified to run
on other platforms (I don't have the knowledge of Windows to do this).
If you would like to add this to the library, I relinquish all rights
Here's a link to the source repository:
The python documentation says that the read() and write() methods on array
objects have been deprecated since 1.5.1. I assume this is because their
semantics are almost the exact opposite of read() and write() on a file-like
object; array.read() reads data from a file into the array and array.write()
writes data from the array to a file.
This causes fatal confusion in code that checks for the existence of read()
and write() to determine whether an object is file-like. Code such as
What is the timeline for removing these methods from array? It has been 11