Python-Dev August 2009

python-dev@python.org

103 participants
73 discussions

how important is setting co_filename for a module being imported to what __file__ is set to?

by Brett Cannon

I am going through and running the entire test suite using importlib to ferret out incompatibilities. I have found a bunch, although all rather minor (raising a different exception typically; not even sure they are worth backporting as anyone reliant on the old exceptions might get a nasty surprise in the next micro release), and now I am down to my last failing test suite: test_import. Ignoring the execution bit problem (http://bugs.python.org/issue6526 but I have no clue why this is happening), I am bumping up against TestPycRewriting.test_incorrect_code_name. Turns out that import resets co_filename on a code object to __file__ before exec'ing it to create a module's namespace in order to ignore the file name passed into compile() for the filename argument. Now I can't change co_filename from Python as it's a read-only attribute and thus can't match this functionality in importlib w/o creating some custom code to allow me to specify the co_filename somewhere (marshal.loads() or some new function). My question is how important is this functionality? Do I really need to go through and add an argument to marshal.loads or some new function just to set co_filename to something that someone explicitly set in a .pyc file? Or I can let this go and have this be the one place where builtins.__import__ and importlib.__import__ differ and just not worry about it? -Brett

11 years, 7 months

3to2 0.1 alpha 1 released

by Joe Amenta

Hello all, I have released the first alpha version of 3to2 after finishing it for my Google Summer of Code 2009(tm) project. You can get the tarball for this release at http://bitbucket.org/amentajo/lib3to2/downloads/3to2_0.1-alpha1.tar.gz. This requires python 2.7, because it requires a newer version of 2to3 than what comes with 2.6. Release notes are in the RELEASE file. Development happens at http://bitbucket.org/amentajo/lib3to2/, and the source code for this release lives at http://bitbucket.org/amentajo/3to2-01-alpha-1. Report bugs at http://bitbucket.org/amentajo/lib3to2/issues/, please. Additional notes and comments can (for now) be found at http://www.startcodon.com/wordpress/?cat=4. --Joe

11 years, 7 months

Re: [Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to?

by P.J. Eby

At 02:57 PM 8/31/2009 -0700, Brett Cannon wrote: >Ignoring that 'new' is not in Python 3.x (luckily 'types' is), I want >a proper solution that doesn't require reconstructing every code >object that I happen to import. Practicality beats purity. ;-) (Especially if it allows importlib to run on older Pythons.) Also, surely you're not worried about *performance* here?

11 years, 7 months

Re: [Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to?

by P.J. Eby

At 01:39 PM 8/31/2009 -0700, Brett Cannon wrote: >On Mon, Aug 31, 2009 at 12:59, Brett Cannon<brett(a)python.org> wrote: > > On Mon, Aug 31, 2009 at 12:27, Antoine Pitrou<solipsis(a)pitrou.net> wrote: > >> Brett Cannon <brett <at> python.org> writes: > >>> > >>> I will plan to take this approach then; > >>> http://bugs.python.org/issue6811 will track all of this. Since this is > >>> a 3.2 thing I am not going to rush to implement this. > >> > >> I still don't understand what the point is of this complicated > approach (adding > >> an argument to marshal.load()) compared to the simple and obvious approach > >> (making co_filename mutable). > > > > If we add the argument to marshal.load* we can eventually drop the > > file location string from marshal data entirely by requiring people to > > specify the filename to use when the code object is created. Making > > co_filename mutable simply doesn't allow for this case unless we > > decide a default value should be used instead. > > > >I should also mention that I am +0 on the marshal.load* change. I >could be convinced to try to pursue a mutable co_filenme direction, >but considering the BDFL likes the marshal.load* approach and it opens >up the possibility of compacting the marshal format I am leaning >towards sticking with this initial direction. Why not just try the code I posted earlier, that doesn't need a mutable attribute OR an API change?

11 years, 7 months

Re: [Python-Dev] how important is setting co_filename for a module being imported to what __file__ is set to?

by P.J. Eby

At 09:33 AM 8/31/2009 -0700, Guido van Rossum wrote: >Of course, tracking down all the code objects in the return value of >marshal.load*() might be a bit tricky -- API-wise I still think that >making it an argument to marshal.load*() might be simpler. Also it >would preserve the purity of code objects. Or maybe we could just do something like this: from new import code def with_changed_filename(code_ob, filename): def remap(ob): if not isinstance(ob, code): return ob return code( ob.co_argcount, ob.co_nlocals, ob.co_stacksize, ob.co_flags, ob.co_code, map(remap, ob.co_consts), ob.co_names, ob.co_varnames, filename, ob.co_name, ob.co_firstlineno, ob.co_lnotab, ob.co_freevars, ob.co_cellvars ) return remap(code_ob) Granted, this takes a bit more memory than an in-place modification, but it's immediately usable and at least works wherever new.code is available. (I've not tested the above, so it may not work. I seem to recall the last time I wrote something like this there was something tricky about handling co_freevars and co_cellvars; I think you may need to omit them if empty, or convert them to None, or from None to an empty tuple or some such rigamarole. And a 3.x version is left as an exercise for the reader. ;-) )

11 years, 7 months

deleting setdefaultencoding iin site.py is evil

by Chris Withers

Hi All, Would anyone object if I removed the deletion of of sys.setdefaultencoding in site.py? I'm guessing "yes!" so thought I'd state my reasons now: This deletion appears to be pretty flimsy; reload(sys) and you have it back. Which is lucky, because I need it after it's been deleted... Why? Well, because you can no longer put sitecustomize.py in a project-specific location (http://bugs.python.org/issue1734860) and because for some projects the only way I can deal with encoded strings sensibly is to use setdefaultencoding, in my case at the start of a script generated by zc.buildout's zc.recipe.egg (I *know* all the encodings in this project are utf-8, but I don't want to go playing whack-a-mole with whatever modules this rather large project uses that haven't been made properly unicode aware). Yes, it needs to be used as early as possible, and the docs should say this, but deleting it seems to be petty in terms of stopping its use when sitecustomize.py is too early and too system-wide and spraying .decode('utf-8')'s all over a code base made up of a load of eggs managed by buildout simply isn't feasible... Thoughts? Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

11 years, 7 months

Excluding the current path from module search path?

by Chris Withers

Hi All, I'm being bitten by this issue: http://bugs.python.org/issue1734860 I'm not sure I agree with Daniel's closing of it so thought I'd ask here... Am I right in thinking that the general idea is that "the current working directory at the time of invoking a script or interpreter ends up on the python path" or should I be thinking "the directory that a script exists in should end up on the python path"? If the latter, then what happens in the case of just starting up an interpreter? If neither, then how come when I have two .py files in a directory, I can import one as a module from the other? In any case, as a parting comment, http://bugs.python.org/issue1232023 seems to have been committed with no tests and the only documentation being a one liner in the NEWS.txt file. Was there other discussion of this? (Incidentally, export PYTHONPATH= or its Windows equivalent circumvents whatever the patch was trying to achieve, so the change doesn't seem to make sense anyway...) cheers, Chris -- Simplistix - Content Management, Batch Processing & Python Consulting - http://www.simplistix.co.uk

11 years, 7 months

Fast Implementation for ZIP decryption

by Shashank Singh

Hi All, I have implemented the simple zip decryption in C (yes, the much loathed weak password based classical PKWARE encryption, which incidentally is the only one currently supported in python) . It performs fast, as one would expect, as compared to the current all-python implementation. Does it sound worthy enough to create a patch for and integrate into python itself? -- Regards Shashank Singh Senior Undergraduate, Department of Computer Science and Engineering Indian Institute of Technology Bombay shashank.sunny.singh(a)gmail.com http://www.cse.iitb.ac.in/~shashanksingh

11 years, 7 months

timed_command.py: consider for inclusion in std. library?

by Charles Waldman

Here's a module "timed_command" I wrote a while ago and is generally useful and might be a good addition to the standard library. It is like commands.getstatusoutput but lets you run a command with an optional timeout. Useful for systems programming where a sub-process might hang. Only works on POSIX, but could perhaps be modified to run on other platforms (I don't have the knowledge of Windows to do this). If you would like to add this to the library, I relinquish all rights to it. Here's a link to the source repository: http://repo.mwt2.org/viewvc/Python/timed_command.py?view=markup

11 years, 7 months

deprecated methods on array objects

by Jake McGuire

The python documentation says that the read() and write() methods on array objects have been deprecated since 1.5.1. I assume this is because their semantics are almost the exact opposite of read() and write() on a file-like object; array.read() reads data from a file into the array and array.write() writes data from the array to a file. This causes fatal confusion in code that checks for the existence of read() and write() to determine whether an object is file-like. Code such as httplib. What is the timeline for removing these methods from array? It has been 11 years now. -jake

11 years, 7 months

2021

2020

2019

2018

2017

2016

2015

2014

2013

2012

2011

2010

2009

2008

2007

2006

2005

2004

2003

2002

2001

2000

1999

Python-Dev August 2009