Mailman 3 January 2008 - Python-Dev

building _ctypes in trunk fails first time around
by Guido van Rossum Jan. 18, 2008

Jan. 18, 2008

If I run "make clean" and then "make", builting _ctypes fails with this message: *** WARNING: renaming "_ctypes" since importing it failed: No module named _weakref Typing "make" a second time fixes this -- it seems a simple matter of _ctypes being built before _weakref. -- --Guido van Rossum (home page: http://www.python.org/~guido/)

3 2

Rationale for NamedTemporaryFile revisited [SEC=UNCLASSIFIED]
by Ole.Nielsen＠ga.gov.au Jan. 18, 2008

Jan. 18, 2008

I found this posting, and those following it, as I too am baffled that NamedTemporaryFile doesn't let you read the generated file - even within the same script. For unit testing it is very commonplace to generate a test file on the fly and then use it as input the function being tested. NamedTemporaryFile would be the perfect candidate for this except that the file seems to disappear immediately after it has been closed. This is contrary to Dustin's comment which states that the lifetime extends until the object is garbage collected. The following script illustrates this isn't the case: from tempfile import NamedTemporaryFile fid = NamedTemporaryFile(mode='w', suffix='.tmp', dir='.') fid.write('My temp file') fid.close() # Read it (fid hasn't been garbage collected yet) print fid.name # But the file itself has gone fid2 = open(fid.name) print fid2.readlines() Can someone please help point me to the best way of creating temporary files in Python that can be read in immediately after their creation. Thanks Ole Nielsen, Geoscience Australia On Sun, Mar 18, 2007 at 11:49:55AM +1200, Greg Ewing wrote: > I've just discovered the hard way that NamedTemporaryFile > automatically deletes the file when you close it. That > doesn't seem very useful to me, since surely the reason > you're using NamedTemporaryFile instead of TemporaryFile > is that you want to do something else with it afterwards? > What's the rationale for this behaviour? For both TemporaryFile and NamedTemporaryFile, the rationale is that "temporary" extends until the object is garbage collected. TemporaryFile is deleted immediately (to prevent other applications from modifying your temporary file). NamedTemporaryFile is inteded for use when you need access to the file by filename during the lifetime of the file. In either case, when the object goes, the file should too. Dustin

2 1

Re: [Python-Dev] What to do for bytes in 2.6?
by Raymond Hettinger Jan. 18, 2008

Jan. 18, 2008

[GvR] > I believe the issue of whether and how to backport bytes > (and bytearray?) from 3.0 to 2.6 has come up before, but > I don't think we've come to any kind of conclusion. My recommendation is to leave it out of 2.6. Not every 3.0 concept has to be backported. This particular one doesn't have a straight-forward integration. It duplicates some existing functionality and in general doesn't make life better for the 2.6 coder. The only benefit I can see is that it lets you write code that is a step closer to 3.0; however, it has not been our goal to write one piece of code that runs under both major releases. I think both 2.x and 3.0 are better served if 2.x cleanly stays with the str/uncode model and 3.0 sticks with the bytes/text model. Commingling the two muddies the waters and conflates the distinctions. I think it best to make that jump all at once. Multiple-ways-to-do-it should not be the motto for 2.6. Instead, let's provide the best transition tools possible and keep both 2.6 and 3.0 as clean as possible (no duplicative or half-intergrated functionality). Raymond

2 1

ntpath r54364 (was: PEP 370, open questions)
by Paul Moore Jan. 17, 2008

Jan. 17, 2008

On 17/01/2008, Tim Golden <mail(a)timgolden.me.uk> wrote: > Ummm... see my earlier point a few posts back which refers > to r54364 which gave priority to USERPROFILE over HOMEDRIVE/HOMEPATH Sorry. I'd not realised this was a post-2.5 revision (ie, not in released code yet...) Looking at the change, it appears that the point is to support ~user, which it does by getting the current user's home, and replacing the last part, on the assumption that it is the user name. This works for USERPROFILE (in my experience, I don't know if there are corner cases where it isn't true) but does *not* work if HOME is set, and only works otherwise by preferring USERPROFILE over HOMEDRIVE/HOMEPATH (and I believe that preference is misleading, as USERPROFILE is always set if HOMEDRIVE/PATH is, so there's no point checking for HD/HP if USERDIR fails). I'd recommend that this change be reverted. To correctly get another user's home directory would involve reading (via the registry, or maybe some utility function I couldn't find at a quick glance) the value of HOMEDRIVE/HOMEPATH which is set for the other user. (In fact, given that I guess these could be set in a login script, it's entirely possible that the information simply isn't available). I don't believe that ~user is a useful concept on Windows, and I'd rather that code using it fail than give what are quite probably unreliable answers. If someone wants to support ~user where user is the name of the current user, then that could be covered by an explicit check that the name matches, and if so use the code for '~', but personally I don't think that's worth it. Paul.

3 3

SyntaxError: 'import *' not allowed with 'from .'
by George Sakkis Jan. 16, 2008

Jan. 16, 2008

I posted this on c.l.py but I didn't get a definite answer so I am asking again here: is it documented anywhere that "from .relative.module import *' is syntax error? Unless I missed it, PEP 328 doesn't mention anything about it. Also, while I understand the general caveats and warnings against "import *", at first glance it seems inconsistent to disallow it for relative imports while it is valid for absolute. Are there any particular reasons that allowing relative '*' imports is harder to implement, more fragile or generally worse in some way ? George

1 0

PEP: Post import hooks
by Christian Heimes Jan. 16, 2008

Jan. 16, 2008

I've parted the former PEP as promised. Here is the post import hook. I'll tackle the other half of the PEP later. PEP: 369 Title: Post import hooks Version: $Revision$ Last-Modified: $Date$ Author: Christian Heimes <christian(at)cheimes(dot)de> Status: Draft Type: Standards Track Content-Type: text/x-rst Created: 02-Jan-2008 Python-Version: 2.6, 3.0 Post-History: Abstract ======== This PEP proposes enhancements for the import machinery to add post import hooks. It is intended primarily to support the wider use of abstract base classes that is expected in Python 3.0. The PEP originally started as a combined PEP for lazy imports and post import hooks. After some discussion on the python-dev mailing list the PEP was parted in two separate PEPs. [1]_ Rationale ========= Python has no API to hook into the import machinery and execute code *after* a module is successfully loaded. The import hooks of PEP 302 are about finding modules and loading modules but they were not designed to as post import hooks. Use cases ========= A use case for a post import hook is mentioned in Nick Coghlan's initial posting [2]_. about callbacks on module import. It was found during the development of Python 3.0 and its ABCs. We wanted to register classes like decimal.Decimal with an ABC but the module should not be imported on every interpreter startup. Nick came up with this example:: @imp.when_imported('decimal') def register(decimal): Inexact.register(decimal.Decimal) The function ``register`` is registered as callback for the module named 'decimal'. When decimal is imported the function is called with the module object as argument. While this particular example isn't necessary in practice, (as decimal.Decimal will inherit from the appropriate abstract Number base class in 2.6 and 3.0), it still illustrates the principle. Existing implementations ======================== PJE's peak.util.imports [3]_ implements post load hooks. My implementation shares a lot with his and it's partly based on his ideas. Post import hook implementation =============================== Post import hooks are called after a module has been loaded. The hooks are callable which take one argument, the module instance. They are registered by the dotted name of the module, e.g. 'os' or 'os.path'. The callable are stored in the dict ``sys.post_import_hooks`` which is a mapping from names (as string) to a list of callables or None. States ------ No hook was registered '''''''''''''''''''''' sys.post_import_hooks contains no entry for the module A hook is registered and the module is not loaded yet ''''''''''''''''''''''''''''''''''''''''''''''''''''' The import hook registry contains an entry sys.post_import_hooks["name"] = [hook1] A module is successfully loaded ''''''''''''''''''''''''''''''' The import machinery checks if sys.post_import_hooks contains post import hooks for the newly loaded module. If hooks are found then the hooks are called in the order they were registered with the module instance as first argument. The processing of the hooks is stopped when a method raises an exception. At the end the entry for the module name is removed from sys.post_import_hooks, even when an error has occured. A module can't be loaded '''''''''''''''''''''''' The import hooks are neither called nor removed from the registry. It may be possible to load the module later. A hook is registered but the module is already loaded ''''''''''''''''''''''''''''''''''''''''''''''''''''' The hook is fired immediately. C API ----- New PyImport_* API functions '''''''''''''''''''''''''''' PyObject* PyImport_GetPostImportHooks(void) Returns the dict sys.post_import_hooks or NULL PyObject* PyImport_NotifyModuleLoaded(PyObject *module) Notify the post import system that a module was requested. Returns the module or NULL if an error has occured. PyObject* PyImport_RegisterPostImportHook(PyObject *callable, PyObject *mod_name) Register a new hook ``callable`` for the module ``mod_name`` The PyImport_PostImportNotify() method is called by PyImport_ImportModuleLevel():: PyImport_ImportModuleLevel(...) { ... result = import_module_level(name, globals, locals, fromlist, level); result = PyImport_PostImportNotify(result); ... } Python API ---------- The import hook registry and two new API methods are exposed through the ``sys`` and ``imp`` module. sys.post_import_hooks The dict contains the post import hooks: {"name" : [hook1, hook2], ...} imp.register_post_import_hook(hook: "callable", name: str) Register a new hook *hook* for the module *name* imp.notify_module_loaded(module: "module instance") -> module Notify the system that a module has been loaded. The method is provided for compatibility with existing lazy / deferred import extensions. The when_imported function decorator is also in the imp module, which is equivalent to: def when_imported(name): def register(hook): register_post_import_hook(hook, name) return register imp.when_imported(name) -> decorator function for @when_imported(name) def hook(module): pass Open issues =========== The when_imported decorator hasn't been written. The code contains several XXX comments. They are mostly about error handling in edge cases. Backwards Compatibility ======================= The new features and API don't conflict with old import system of Python and don't cause any backward compatibility issues for most software. However systems like PEAK and Zope which implement their own lazy import magic need to follow some rules. The post import hooks carefully designed to cooperate with existing deferred and lazy import systems. It's the suggestion of the PEP author to replace own on-load-hooks with the new hook API. The alternative lazy or deferred imports will still work but the implementations must call the ``imp.notify_module_loaded`` function. Reference Implementation ======================== A reference implementation is already written and is available in the *py3k-importhook* branch. [4]_ It still requires some cleanups, documentation updates and additional unit tests. Acknowledgments =============== Nick Coghlan, for proof reading and the initial discussion Phillip J. Eby, for his implementation in PEAK and help with my own implementation Copyright ========= This document has been placed in the public domain. References ========== .. [1] PEP: Lazy module imports and post import hook http://permalink.gmane.org/gmane.comp.python.devel/90949 .. [2] Interest in PEP for callbacks on module import http://permalink.gmane.org/gmane.comp.python.python-3000.devel/11126 .. [3] peak.utils.imports http://svn.eby-sarna.com/Importing/peak/util/imports.py?view=markup .. [4] py3k-importhook branch http://svn.python.org/view/python/branches/py3k-importhook/ .. Local Variables: mode: indented-text indent-tabs-mode: nil sentence-end-double-space: t fill-column: 70 coding: utf-8 End:

3 20

Extending generic functions
by Phillip J. Eby Jan. 15, 2008

Jan. 15, 2008

At 02:19 PM 1/15/2008 -0800, Guido van Rossum wrote: >While I have you, I've come across a need that I don't know how to do >with GFs. Suppose I have a GF that implements some recursive function >over container types, e.g. serialization or flattening. Now suppose >I'd like to create *another* GF that implements the same algorithm >except it does something different for one particular type; as a >concrete example, suppose we want to treat tuples atomically when >flattening. Is there a way to reuse the work of the first GF? Yes. RuleDispatch actually has a 'clone()' feature for single-dispatch generics that does exactly what you're looking for: http://peak.telecommunity.com/DevCenter/VisitorRevisited (see the heading "Extension and Reuse"). It's probably not a bad idea to put a cloning feature on my extended to-do list for PEAK-Rules. In PEAK-Rules (the system after which PEP 3124 was modelled), a generic function has a RuleSet that contains its rules, and RuleSets can be subscribed to. So, you could create a listener that automatically takes the rules added to one function and adds them to others. It's not packaged as a convenient decorator or anything, but one certainly could make one. It'd also need to have some way to ensure that the rules from the original function were treated at a lower combination precedence than anything else, but that could be handled by with a custom method type pretty easily, I think. All in all, a cloning feature might be somewhere around 20-50 lines of code to add in -- and a third party could probably roll their own without changing PEAK-Rules' source. > It doesn't work to create a new GF that calls on the first GF for types >it doesn't understand; e.g. a list could contain a tuple. Does your GF >machinery let me do this in a relatively clean way? It's relatively clean. One of my goals for the changed architecture in PEAK-Rules vs. RuleDispatch was to make it possible to do all sorts of things like this, by opening up the whole thing to extensibility. Btw, a lot of the credit for PEAK-Rules' design goes to you, in a roundabout way. Your tuple-of-types prototype made me see that it could be practical to implement generic functions using generic functions as a base -- getting rid of interfaces and adaptation altogether. I just needed to come up with a design that allowed separating the genericness of a function (e.g. the rules to be applied) from the implementation of genericness (the "engine" that turns rules into executability. In this way, a generic function can start out using just tuples of types, but then graduate to a full expression-based system, just by changing the engine.

1 0

Re: [Python-Dev] Monkeypatching idioms -- elegant or ugly?
by Phillip J. Eby Jan. 15, 2008

Jan. 15, 2008

At 01:51 PM 1/15/2008 -0800, Guido van Rossum wrote: >On Jan 15, 2008 1:27 PM, "Martin v. Löwis" <martin(a)v.loewis.de> wrote: > > > Second, a "metaclass" to add a number of methods (or other attributes) > > > to an existing class, using a convenient class notation: > > > > I think this is similar to my "partial" classes: > > > > http://pypi.python.org/pypi/partial > >Indeed it is. I guess my only innovation is realizing that you don't >have to create a real metaclass -- you can set __metaclass__ to a >function that does the magic. I like your feature of refusing >overrides unless flagged with @replace. > >I think that despite the objection that monkeypatching shoudn't be >made too easy, it's worth at looking into a unification of the API, >features, and implementation. I'm curious: has this affected your thoughts re: overloading existing functions? Note that overloading-in-place would provide the next_method idiom for calling the original function. (I'm assuming you still don't like the idea of changing a function's code to do it, just wondering about the non-implementation aspect. :) )

2 1

r59947 - in python/trunk:Lib/test/test_structseq.py Misc/NEWS
by Raymond Hettinger Jan. 14, 2008

Jan. 14, 2008

[Christian Heimes] > Log: > Added new an better structseq representation. E.g. repr(time.gmtime(0)) now returns 'time.struct_time(tm_year=1970, tm_mon=1, > tm_mday=1, tm_hour=0, tm_min=0, tm_sec=0, tm_wday=3, tm_yday=1, tm_isdst=0)' instead of '(1970, 1, 1, 0, 0, 0, 3, 1, 0)'. The > feature is part of #1816: sys.flags FWIW, I was looking into something similar but didn't proceed because it would break eval(repr(s)) == s as the constructor signature wants all the args in a tuple and won't accept keywords. Still, I think what you did is a nice improvement. Here's a couple more if you want to proceed down that path: 1. Have structseq subclass from PyTupleObject so that isinstance(s, tuple) returns True. This makes the object usable whenever tuples are needed. 2. Add _fields, _asdict, and _replace to match the API in collections.namedtuple(). The _fields tuple should only include the visible positional fields while _asdict() and _replace() should include all of the fields whether visible or accessible only by attribute access. Thanks for the nice work. Raymond

4 12

test module availability on different Python implementations
by Giampaolo Rodola' Jan. 14, 2008

Jan. 14, 2008

Hi, Today I received a report from a guy who tried to run the test suite of a module of mine. The test suite uses the test module for running tests, for temporary files support and for doing a bunch of other things. He tested it on CentOS 5 and looks like python 2.5.1 includes the test module but python 2.4.3 does not: [root@dell test]# python2.4 test_ftpd.py Traceback (most recent call last): File "test_ftpd.py", line 37, in ? from test import test_support ImportError: No module named test Another guy reported the same thing by having tried to run the test suite against a Windows mobile OS equipped with PythonCE 2.4 where the test module seems to be not available too. I was wondering if there was someone aware of this problem.

2 2