Mailman 3 December 2002 - Python-Dev

[Python-checkins] python/dist/src/Objects abstract.c,2.93.6.7,2.93.6.8
by David Goodger Dec. 18, 2002

Dec. 18, 2002

This backport checkin (Oct 5 2002) changed the exception text between 2.2.1: >>> int(None) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: object can't be converted to int and 2.2.2 (last line): >>> int(None) Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: int() argument must be a string or a number Should this change have been backported to 2.2.2 from the 2.3 codebase? Patrick O'Brien brought this to my attention. He found a test in the Docutils test suite that broke under 2.2.2. I've parameterized the broken test to work on old & new Python versions, which I would have had to do for Python 2.3 anyway. While minor, I think this was an enhancement and not a bug fix. Reference: discussion in http://www.python.org/sf/563740 -- David Goodger <goodger(a)python.org> Open-source projects: - Python Docutils: http://docutils.sourceforge.net/ (includes reStructuredText: http://docutils.sf.net/rst.html) - The Go Tools Project: http://gotools.sourceforge.net/

6 10

Re: [Python-Dev] Zipping Zope3
by Wiktor Sadowski Dec. 17, 2002

Dec. 17, 2002

[Just van Rossum] >Ugh, you should really consider getting a decent diff tool :-/ How can >*I* test if if I have no clue what exactly you've done? Well, as you already know my implementation of custom imports (import hooks) is a bit different and I am using modified Python22 ,so it's not easy to make diffs. Zipimport.c in attached zipimport.tar.gz is to provide you with more details. Don't try to compile it ;-) >I can see that keeping the file object open could speed things up, but >is it a polite & decent thing to do? Yes, but it would be nice to not fopen/fclose for each __importer__.get_data. Best Wiktor http://www.wiktorsadowski.com

2 2

Zip import and sys.path manipulation (was Re: directory hierarchy proposal)
by Phillip J. Eby Dec. 17, 2002

Dec. 17, 2002

(CC'd to python-dev because of possible impact on zipimport implementation/API) At 08:00 AM 12/16/02 -0500, Jim Fulton wrote: >Each zope package __init__ file would have a bit of boilerplate code >that would include all zope directories in sys.path: > > # Boilerplate code that combines all zope directories in sys.path > import sys, os > __path__ = [os.path.join(p, 'zope') for p in sys.path] > >Jim That should probably read: __path__ = [os.path.join(p, 'zope') for p in sys.path if isinstance(p,StringTypes)] or else it could break in 2.3 if somebody's putting non-strings on sys.path. I don't remember exactly how the python-dev discussion turned out, but I kind of got the impression that it was going to be allowable in 2.3. Also, I'm pretty sure it will not give the expected/desired results in conjunction with zipfile imports. For these and related reasons, I would suggest that a slightly different snippet be used instead (the names are lousy, improvements welcome): from namespace_packages import makeNamespacePath __path__ = makeNamespacePath('zope') And then make whatever carries "namespace_packages" be something that's distributed separately from Zope, preferably part of the standard library or distutils (and backported where possible). If we can make it so that the __init__ code never *has* to change, that will minimize the likelihood of incompatible __init__.py's floating around. Meanwhile, a module that actually *knows* something about what can be on sys.path (and how to interpret it) can do so. In order to support something like this, importer objects would need to be able to return something that's the moral equivalent of "os.path.join" on an existing sys.path entry; either another importer object or a string that can be handled by one. At this point, checking for a '.zip' extension at the end of a string isn't going to cut it any more.

10 24

Zipping Zope3
by Wiktor Sadowski Dec. 17, 2002

Dec. 17, 2002

[Guido van Rossum] >The problem is that Zope3 needs *some* way of storing data and config >files related to specific packages, and there isn't really any other >convenient and conventional way except storing it in a package >directory and using the package's __file__ to find it. __importer__ , the zip specific alternative to __file__ could be provided to solve the problem. in an __init__.py: if __file__: ............................ elif __importer__: f=___importer__.open()#to speedup and to get direct access to the binary #without ___importer__.open() should work as well data1=___importer__.get_data("ModuleName\\index.html") data2=___importer__.get_data("ModuleName\\help.html") if data1: .......... f.close() It would require some minor changes to Just's zipimport.c + new function in import.c PyImport_ExecCodeModuleEx2(char *name, PyObject *co, char *pathname,PyObject *importer) where instead of setting __file__: if (PyDict_SetItemString(d, "__importer__", importer) != 0) ........................................................ As for zipimport.c: new methods: open() for speedup : ........ char* fname=PyString_AsString(self->archive); self->arcfile=PyFile_FromString(fname,"rb"); ....... return self->arcfile; #PyObject*arcfile new field(zipimporter struct) .................................................................. get_data(data_path)#modified load_module .................................................................. in load_module: mod = PyImport_ExecCodeModuleEx2(fullname, code, modpath,self); instead of mod = PyImport_ExecCodeModuleEx(fullname, code, modpath); Tested;) Regards Wiktor http://www.wiktorsadowski.com

2 1

known obvious thing or bug (rexec)?
by Samuele Pedroni Dec. 17, 2002

Dec. 17, 2002

latest CVS Python 2.3a0 (#1, Dec 14 2002, 07:19:32) [GCC 2.95.2 20000220 (Debian GNU/Linux)] on linux2 Type "help", "copyright", "credits" or "license" for more information. consider >>> import rexec >>> r=rexec.RExec() >>> r.r_exec("class C: pass\nC.a=0") Traceback (most recent call last): File "<stdin>", line 1, in ? File "/home/users/p/pe/pedronis/python/dist/src/Lib/rexec.py", line 309, in r_ exec exec code in m.__dict__ File "<string>", line 2, in ? RuntimeError: classes are read-only in restricted mode vs. >>> r.r_exec("class C(object): pass\nC.a=0") >>> regards

7 17

Re: [Python-Dev] summary for 2002-12-01 through 2002-12-15
by Michael Hudson Dec. 17, 2002

Dec. 17, 2002

Skip Montanaro <skip(a)pobox.com> writes: > Actually, no, I haven't yet resolved this. Someone else said 'make > distclean' seemed to work for them. For the time being I'm just using two > CVS checkouts and building in the source tree. I need to see if a bug > report needs filing... I reopened the bug (#599248) whose fix caused the problem. Last news was that AMK was "thinking about it". I guess we could just back it out until such time as someone comes up with a better fix. Cheers, M. -- US elections For those of you fearing that the rest of the world might be making fun of the US because of this: Rest assured, we are. -- http://www.advogato.org/person/jameson/diary.html?start=12

1 0

RE: [Python-Dev] Re: [Zope3-dev] Zip import and sys.path manipulation (was Re: directory hierarchy proposal)
by Moore, Paul Dec. 16, 2002

Dec. 16, 2002

From: Guido van Rossum [mailto:guido@python.org] > I find string subclasses an ugly hack that should not be used. Agreed. > The meta hook (or whatever it's called now) should do this. Why bother? > > - strings which don't have a path structure wil break > > We can fix that by skipping anything for which > os.path.isdir(dir) [for the original value of dir] isn't true. Agreed. This makes extend_path "do the right thing" for anything that can currently be on a path, and punt to an extension hook for anything else. That sounds reasonable (but see below). > > (and of course, objects other than string subclasses don't > > get "extended"). > > We can use some kind of extension hook to deal with those. It depends where you envisage that hook. I'd be unhappy with requiring something in the import hook implementation - two reasons, first because the current hook mechanism is nice and simple, and hence easy to document and use, and second because people simply won't implement the path-extension hook, and nearly everything will work fine - the breakage only occurs when the hook is used in conjunction with pkgutil. At that point, saying "I told you so" isn't much help... I've seen this sort of thing happen too often before to be comfortable with it. A simple protocol, with an obscure little requirement which is to support a relatively rare usage. People *never* implement that correctly, and the result is annoying breakages. > The important point is that the idiom placed in __init__.py > can remain the same (the two lines quoted above) while the > implementation of pkgutil.extend_path() can evolve or be > modified (even monkey-patched if you're desperate) as the > import hooks evolve. I'd say the extension mechanism should be external to both the import hook and pkgutil. It then becomes the user's responsibility, when they combine a particular hook with the use of pkgutil, to write the glue. pkgutil.register_hook(whatever) Hook authors can, of course, implement a convenience method, to be used as pkgutil.register_hook(ihook.pkgutil_hook) but if they fail to, at least the user isn't left up the creek without the proverbial paddle :-) (Hope that made sense - I was hand-waving a bit too much...) > > I don't think that either of these is even remotely a > > showstopper, but they do need to be documented - specifically > > because as a module, this code can be imported by anybody, > > and so the constraints on the user should be made clear. > > Of course. Though with the current state of documentation, > I give less priority to documenting every little detail of > this code than to getting it in the code base (with *some* > documentation, for sure). At the moment, I'm playing with the hook mechanism, with the intent of being able to provide some of the documentation. At the very least, I'd like to be able to do a tutorial-style "how to write a simple import hook" document. But yes, a lot of this area is only minimally documented right now, so we don't need to aim for complete perfection just yet :-) > OTOH I expect that most things you'd want to place on sys.path > *do* have a natural string representation. After all, if > something doesn't have a name, it's hard to talk about. And if > it has a name, it can be placed in PYTHONPATH (with the minor > quibble that URLs use ':' which -- at least on Unix -- is also > used as the separator in *PATH variables. :-( ) Hmm. Arguable. I take your point, but I'm looking at it from a different point of view. A "place to store modules" could be something as complex as a database (TCL Starkits use MetaKit). I *could* define a string representation encoding the table name, columns to use, etc etc, but why bother? Just use a database importer object directly. You can't put it on PYTHONPATH, but you can easily use something like this in a packaged application context. Let's wait and see what real use brings on this one. > Maybe importer objects should have a standard API that lets you > ask for an extended object. This way lies a VFS implementation. I'd say we should write a general import hook based on a VFS implementation, rather than the other way around. And let's leave a full VFS alone for now :-) Paul.

2 1

Re: [Python-Dev] PEP-298: buffer interface widening too much?
by Todd Miller Dec. 16, 2002

Dec. 16, 2002

Martin v. Löwis wrote: >>I've used this aspect of PyArg_ParseTuple, but forgot about it. I >>was thinking that you're right, the original object is "gone", we're >>stuck. But... it's still (in all probability) lying around in the arg >>tuple, so some sort of PyArg_ReleaseBuffers is possible. >> >> > >That looks clumsy, > Well, yes, especially in light of PEP-286. :) >and also too easy to forget unless done systematically, > I really liked the idea of doing the buffer release behind the scenes, as suggested by PEP-286 and below. >in which case it will add a considerable performance cost. I think the > But won't *any* systematic solution have a performance cost? Argtuples don't sound "free". I'm really playing devil's advocate here, because I like the concept of PEP-286. >locked buffer interface is much simpler in comparison. > I disagree here. I didn't see PEP-298 address PyArg_ParseTuple at all, so there's no real basis for comparison on this point. I'll grant you that PyArg_ReleaseBuffers is probably a lame solution to a problem better solved by PEP-286; that said, because use of PyArg_ReleaseBuffers is explicit, there is no problem distributing the performance penalty to places where there is a corresponding benefit. I was not suggesting dropping the existing release call; merely adding a layer which could make sense out of PyArg_ParseTuple specs. > >If Jack ever manages to design an alternative parse tuple interface, I >assume it will arrange to automatically release the buffer when the function >returns (or, more precisely, when the buffer is deallocated. > >In fact, my PEP 286 could address this issue: the locked buffers would be >associated with the enhanced tuple, so that releasing the tuple would unlock >the buffers. > That sounds pretty cool. >This *still* has the potential for breaking code, since now >the buffers are locked while the function runs (i.e. callbacks couldn't >expand the array anymore); it is likely that such breakage would uncover >a bug in the locking assumptions of the existing code. > > I agree. But the means exist to work around such problem (release and re-aquire), and I would guess they are even rarer than the existing "buffer safety" issues. >Your attempt to fix your proposal indicates one of the strengths of the PEP >process: most initial proposals have flaws, and, unless completely spelled >out, readers will typically miss the flaws on initial reading. The weakness, >of course, is that a PEP might be sitting around for a long time, and nobody >would be reviewing it. > > > >>Incidentally, does PEP-298 solve this now? >> >> > >Depends on what "this" is. Existing code will continue to work as before: >No locking in ParseTuple happens, as a result, the arguments are unlocked >when the function returns; this is desirable. Since the buffers aren't >locked, relocation may happen. If such relocation is ever observed, people >would have to replace the "s" parser with an "O" parser, and perform the >locking themselves. > I'm just saying that we don't need 2 extra "locking calls" to achieve this. If the default behavior for the existing Python objects (array, mmap, ...) is a onetime warning when a lock is abused, where's the issue? And with the warning framework, the one time warning could be mutated into an exception or supressed by a variety of means. My thinking is that *all* buffer pointers should be "logically" locked. Objects that don't need locks don't need to change. All extensions should be written and/or updated to handle objects which require locking. Extensions which aren't updated will trigger warnings which could be supressed or converted into exceptions. For me, that's simpler than saying "correct extensions" lock the buffer, but "obsolete ones" may still segfault w/o warning. Perhaps saying that "a warning is enough" is too limiting, but this is a fringe issue, so I think maybe it is. > >So, yes, PEP-298 addresses this - although modification of existing code is >necessary to make use of this feature (as is in your fixed proposal). > > > >>Well, this does sound bad, but I was assuming the core and library >>would be "fixed" at the first introduction of locking, and the only >>problem would be with non-conforming extensions. I now think that >>"lock violations" might be better handled using the warning framework, >>defaulting to a one time warning. >> >> > >I think we should strive for a solution that keeps the function parameters >locked while the function runs, and unlocks them automatically at function >return time. > I like this idea, but wonder if it won't inflict a performance penalty on all function calls. > >This shows another flaw in the wording of PEP 298: It implies without saying >that the expectation is that you should have as many release calls as you >have lock calls. > I was certainly assuming this was the case. > >Regards, >Martin > > > Regards, Todd

3 5

summary for 2002-12-01 through 2002-12-15
by Brett Cannon Dec. 16, 2002

Dec. 16, 2002

Yes, it's that time again! You guys (are there any ladies on python-dev?) have about a day to let me know about my mistakes before I send this out to the rest of the world. Oh, I am leaving off the FixedPoint discussion on this summary on purpose; I feel that it will be all clarified soon and so I am going to wait to summarize it after that has occurred. And Martin, your name is correctly spelled (complete with emulaut!) in the original and should hopefully show up when I send it out to c.l.p. Enjoy. +++++++++++++++++++++++++++++++++++++++++++++++++++++ python-dev Summary for 2002-12-01 through 2002-12-15 +++++++++++++++++++++++++++++++++++++++++++++++++++++ ====================== Summary Announcements ====================== Aahz suggested that I try writing the summary from a third-person perspective so as to remove ambiguity if anyone ever quotes something from a summary which was written in first-person. I am giving it a go in this summary although there is not much here that is affected. There was an error in the last summary where I gave Guido's position on something incorrectly (summary of dict evaluation order). It has been fixed. I am going to fix errors like that in old summaries with an errata note in the text version. It won't show up in the HTML version, though. And go to PyCon_. =) .. _PyCon: http://www.python.org/pycon/ =================================== `New universal import mechanism`__ =================================== __ http://mail.python.org/pipermail/python-dev/2002-December/030602.html Splinter threads: - `New and Improved Import Hooks <http://mail.python.org/pipermail/python-dev/2002-December/030638.html>`__ - `Re: New and Improved Import Hooks <http://mail.python.org/pipermail/python-dev/2002-December/030708.html>`__ - `Re[2]: [Python-Dev] New and Improved Import Hooks <http://mail.python.org/pipermail/python-dev/2002-December/030776.html>`__ - `New Import Hook & New Zip Importer <http://mail.python.org/pipermail/python-dev/2002-December/030755.html>`__ - `(no subject) <http://mail.python.org/pipermail/python-dev/2002-December/030812.html>`__ - `Another approach for the import mechanism <http://mail.python.org/pipermail/python-dev/2002-December/030828.html>`__ - `Re: Another approach for the import mechanism <http://mail.python.org/pipermail/python-dev/2002-December/030985.html>`__ - `New Import Hook & New Zip Importer <http://mail.python.org/pipermail/python-dev/2002-December/030755.html>`__ - `Import: where's the PEP? <http://mail.python.org/pipermail/python-dev/2002-December/030868.html>`__ - `zipimport & import hooks <http://mail.python.org/pipermail/python-dev/2002-December/030886.html>`__ - `VFS <http://mail.python.org/pipermail/python-dev/2002-December/030952.html>`__ - `Zip imports, PEP 273, and the .zip extension <http://mail.python.org/pipermail/python-dev/2002-December/030953.html>`__ - `Import compromise <http://mail.python.org/pipermail/python-dev/2002-December/030956.html>`__ - `zipimport, round 3 (or would that be 37?) <http://mail.python.org/pipermail/python-dev/2002-December/030968.html>`__ - `Complexity of import.c <http://mail.python.org/pipermail/python-dev/2002-December/031000.html>`__ - `zipimport, round 4 <http://mail.python.org/pipermail/python-dev/2002-December/031009.html>`__ - `new import hooks & zip import <http://mail.python.org/pipermail/python-dev/2002-December/030755.html>`__ In case you can't figure it out from the title of all of the splnter threads, there was an immense discussion about `PEP 273`_ and getting an import mechanism for zipped modules. To give a little back-story, PEP 273 proposes allowing modules to be put into a zip file for easy distribution. You could then just add the zip file to ```sys.path`_`` , PYTHONPATH_ , etc. and importing from the zip file would be handled properly. It was scheduled to be included in Python 2.3 thanks to Paul Moore keeping James Ahlstrom's patch synced with CVS. All was right with the world, more or less. But then Wiktor Sadowski commented on a checkin for `import.c`_ and how he would like a better way to write custom imports. This led to Just van Rossum to write a new import mechanism. Basically Just was unhappy with how much fiddling the PEP 273 patch was doing to import.c. So, just like any self-respecting programmer would do, he came up with his own solution. =) Most of the discussion revolved around a few central points. One was whether this was even worth the effort. Since the PEP 273 patch worked, why should Just bother reinventing the wheel? The whole reason Just decided to come up with another implementation was a major point. The other was that he felt he could come up with a more general way of allowing custom importers *and* have zip imports a part of the core. There was also a discussion of whether Gordon McMillan's `iu.py`_ wasn't a good solution; it was already well-tested and coded. But Just wanted a certain functionality that iu.py didn't offer... Just wanted to be able to put arbitrary objects in ``sys.path``. This would allow the object to be called to figure out whether it had the module being requested or not. Taking the zip importer as an example, there would be a zipimporter instance in ``sys.path`` for each zip file in the path (once it was discovered it was a zip file; bootstrapping has been carefully handled). The issue of backward-compatibility was instantly brought up. It has been assumed that everything in ``sys.path`` was a string and starting to include zipimporter objects would break that. The idea of making zipimporters (and any subsequent importers in the stdlib) subclasses of strings was brought up and basically accepted, although Guido viewed it as an ugly hack. But in the end, Just's version acts like iu.py. The basic algorithm for handling zip files is:: def find_module(name, path): if isbuiltin(name): return builtin_filedescr if isfrozen(name): return frozen_filedescr if path is None: path = sys.path for p in sys.path: # I think Just meant to say ``path`` here try: v = zipimporter(p) except ZipImportError: pass else: w = v.find_module(name) if w is not None: return w ...handle builtin file system import... It is easily generalized to work for any registered importers. There is also the idea of adding something called a "metapath" that iu.py has. It would allow one to override any import by catching it before trying ``sys.path``. One other discussion was over this was going to need a PEP. Just has written up an explanation, but as to whether a PEP is required has not been decided. Just's implementation is now a patch at http://www.python.org/sf/652586 . .. _PEP 273: http://www.python.org/peps/pep-0273.html .. _sys.path: http://www.python.org/doc/current/lib/module-sys.html .. _PYTHONPATH: http://www.python.org/doc/current/tut/node8.html .. _import.c: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Pytho… .. _iu.py: http://www.mcmillan-inc.com/iu.html =================== `Mac OSX issues`__ =================== __ http://mail.python.org/pipermail/python-dev/2002-December/030604.html Continuation of http://mail.python.org/pipermail/python-dev/2002-November/030525.html Guido got a hold of a OS X box and ran the test suite on it and found four tests that didn't pass. One was for ``locale`` (which Guido didn't care about), another was the ``test_largefile`` test, another was for ``re``, and the last one was for ``socket``. The ``test_largefile`` suite seeks on a new file to a huge position. On OS X and Windows 2000 this actually adds to the file up to the seek point so as to make it a complete file. Windows 9x, on the other hand, just writes to that offset regardless of whether it is within the bounds of the file. Because it creates such a huge file it takes forever to complete and thus has been made an optional test on OS X. The ``re`` tests were failing for a lack of space on the stack. to allow the tests to pass you need to execute the statement ``ulimit -s 2000`` which ups the stack size to approximately 2 Mb. This has sense been added to the regression testing framework to be done for you if needed. ``socket`` was failing on ``.getsockbyaddr()`` when trying it for the machine's own name. Barry Warsaw discovered that the test passed if you set hostnames by using NetInfo. ============================================================== `RE: how to kill process on Windows started with os.spawn?`__ ============================================================== __ http://mail.python.org/pipermail/python-dev/2002-December/030595.html Skip Montanaro wanted to be able to kill a Windows process externally. He suggested implemented a ``os.kill()`` function that would kill a process in a platform-independent way. Problem is that Windows wants the process hook which is different from what POSIX wants. Another issue is that the Windows docs say that you should not kill processes externally. It seems the idea has been dropped. ========================================================= `extension module .o files wind up in the wrong place`__ ========================================================= __ http://mail.python.org/pipermail/python-dev/2002-December/030644.html Skip Montanaro discovered that if you built Python in a subdirectory of the source directory (e.g., source in ``source/`` and you are building in ``source/build/``) the .o files are put in the wrong place (they use ``source/`` as the root directory to place things instead of ``source/build/``). Skip resolved his issue by doing a ``make distclean`` and then running ``../configure``. ========================================= `Make universal newlines non-optional`__ ========================================= __ http://mail.python.org/pipermail/python-dev/2002-December/030692.html Martin v. Lwis asked if anyone mind not making universal newline support a compile option. He volunteered to remove the ``#ifdef`` statements and such to get this done. Jack Jansen (who wrote the universal newline code) pointed out that if it was required there could be issues on certain platforms. Martin had no issue with this since this would force the platform maintainers to fix the code for it to work. In the end Martin suggested leaving it an option in 2.3 and removing it as an option and requiring it in 2.4. ============================================ `Re: [Patches] Patch for xmlrpc encoding`__ ============================================ __ http://mail.python.org/pipermail/python-dev/2002-December/030984.html A discussion over a patch for ``xmlrpclib`` ended up on python-dev. The relevancy of it was that the statement ``sys.{get,set}defaultencoding`` is only useful for coercion of Unicode to byte strings. It was also stated that ASCII should always be assumed to be the default encoding. Also, never put any non-ASCII text in a string; that is what Unicode objects are for. ========================== `PEP 242 Numeric kinds`__ ========================== __ http://mail.python.org/pipermail/python-dev/2002-December/030859.html Paul DuBois emailed the list mentioning how a group of people were supposed to get together to decide the fate of `PEP 242`_ in June 2002; that didn't happen. So Paul suggested making the suggested "making it a separate deliverable on the Numeric_ site and withdrawing the PEP". Everyone who responded pretty much said to add the proposed module. A final outcome has not been decided as to whether Paul will rescind the PEP or push it forward. If you have an opinion on the matter, let it be known. .. _PEP 242: http://www.python.org/peps/pep-0242.html .. _Numeric: http://www.pfdubois.com/numpy/ ================== `We got leaks!`__ ================== __ http://mail.python.org/pipermail/python-dev/2002-December/030933.html Tim Peters discovered that the new datetime_ type was leaking memory. The C version was slowly leaking about 10 references per loop through the testing suite. But the Python version was leaking 37 reference per loop through. Tim did his reference checking using this chunk of code:: def test_main(): import gc r = unittest.TextTestRunner(stream=sys.stdout, verbosity=2) s = test_suite() while True: r.run(s) gc.collect() print gc.garbage # this is always an empty list print '*' * 10, 'total refs:', sys.gettotalrefcount() This substituted for the ``test_main()`` for the testing suite printed out the reference numbers rather nicely. Well, it didn't take the weekend to solve. Kevin Jacobs stepped up to help. The first thing he found was that it was not occurring in a debug build (which is why when you are testing new code you should **always** run it in a debug build). Then Kevin discovered the issue came up when executing a ``__hash__()`` call on anything. Tim then realized a missing decrement in ``slot_tp_hash()`` was the culprit. Then Kevin found another culprit in ``cPickle_``. That was also patched (and everything has been flagged to be backported to the 2.2 branch). The reason this thread is being mentioned is because it 1) was all solved in 5 hours and 40 minutes which is pretty cool and 2) using the chunk of code that Tim provided to stare at the reference count is a good way to look for memory leaks in your code. .. _datetime: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/nondist/sandbo… .. _cPickle: http://www.python.org/doc/current/lib/module-cPickle.html ========================= `os.path.commonprefix`__ ========================= __ http://mail.python.org/pipermail/python-dev/2002-December/030947.html Armin Rigo noticed how ``os.path.commonprefix()`` works character-by-character and not by path separators as one might expect (although the documentation does warn about its char-by-char nature). Skip Montanaro suggested ``os.path.commonpathprefix()`` be added. He mentioned that `Tools/scripts/trace.py`_ had such functionality. As of this writing no one has taken the initiative to add the function. .. _Tools/scripts/trace.py: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Tools… ======================================================== `__getstate__() not inherited when __slots__ present`__ ======================================================== __ http://mail.python.org/pipermail/python-dev/2002-December/030980.html Greg Ward noticed that "If a class and its superclass both define __slots__, it appears that __getstate__() is not inherited from the superclass." He wasn't sure if this was a bug or a feature. Well, it's a feature; since there is no guarantee that the superclass's ``__getstate__()`` will know about the subclass's ``__slots__``, it is not automatically inherited. Kevin Jacobs provided the following snippet of code to deal with this "feature":: def __getstate__(self): state = getattr(self, '__dict__', {}).copy() for obj in type(self).mro(): for name in getattr(obj,'__slots__',()): if hasattr(self, name): state[name] = getattr(self, name) return state def __setstate__(self, state): for key,value in state.items(): setattr(self, key, value) ==================== `Tarfile support`__ ==================== __ http://mail.python.org/pipermail/python-dev/2002-December/031086.html Gustavo Niemeyer asked for some help in reviewing the patch ( http://www.python.org/sf/651082 ) that adds Lars Gustabel's tarfile module to the stdlib. The question of the license came up. Lars said he would be willing to change the license over to the `Python Software Foundation`_ . An initial license hand-off agreement is available at http://www.python.org/psf/psf-contributor-agreement.html . .. _Python Software Foundation: http://www.python.org/psf/ ============================== `Tcl, Tkinter, and threads`__ ============================== __ http://mail.python.org/pipermail/python-dev/2002-December/031107.html Martin v. Lwis has now modified ```_tkinter`_`` as so to be able to be used when Tcl has been compiled with thread support. Special thanks goes out to Martin for taking over maintenance of the module. .. __tkinter: http://cvs.sourceforge.net/cgi-bin/viewcvs.cgi/python/python/dist/src/Modul… ============ `PEP 298`__ ============ __ http://mail.python.org/pipermail/python-dev/2002-December/031109.html Splinter threads: - `PEP-298: buffer interface widening too much? <http://mail.python.org/pipermail/python-dev/2002-December/031163.html>`__ - `PEP-298: buffer interface widening too much? <http://mail.python.org/pipermail/python-dev/2002-December/031184.html>`__ Thomas Heller wanted to resolve whether to move forward with an implementation for `PEP 298`_ . This led to a discussion over the specifics of the PEP. Martin v. Lwis wnated some more specific wording from the PEP. Todd Miller then stepped in to say he didn't like how much of an interface was being tacked on. Martin commented that his `PEP 286`_ would help in this whole situation, which Todd ended up agreeing with. .. _PEP 298: http://www.python.org/peps/pep-0298.html .. _PEP 286: http://www.python.org/peps/pep-0286.html

2 1

Re: [Python-Dev] Re: [Zope3-dev] Zip import and sys.path manipulation (was Re: directory hierarchy proposal)
by Moore, Paul Dec. 16, 2002

Dec. 16, 2002

Guido said: > Time's running short (we're close to releasing Python 2.3 alpha 1); > we should decide on a name and a specific policy ASAP. I propose the > following: > > from pkgutil import extended_path > __path__ = extended_path(__path__, __name__) > > Where pkgutil.py could be something like this: > > import sys.path > > def extended_path(path, name): > """Extend a package's path. (XXX more.)""" > path = path[:] > for dir in sys.path: > if isinstance(dir, (str, unicode)): > dir = os.path.join(dir, name) > if dir not in path and os.path.isdir(dir): > path.append(dir) > return path Are you proposing this for the standard library? It certainly sounds reasonable, but the documentation should make clear that it still fails for certain types of sys.path entry. Specifically, - string subclasses will get converted into basic strings - strings which don't have a path structure wil break (and of course, objects other than string subclasses don't get "extended"). I don't think that either of these is even remotely a showstopper, but they do need to be documented - specifically because as a module, this code can be imported by anybody, and so the constraints on the user should be made clear. In the long term, I think the two most likely things to appear on sys.path, apart from directory names, are - "Filesystem namespace extensions" like the zipfile support. That is, things which act like virtual directories. - raw importer objects Frankly, just dumping a raw imported onto sys.path (or a package path) is so much easier than implementing a string naming scheme, that unless there's an overwhelmingly natural string representation (virtual directory or URL are the only two that come to mind) it's simpler not to bother. So saying that module paths should only contain pathlike strings, or importer objects, is probably reasonable. I wouldn't recommend enforcing this, just documenting it as the normal convention. Modules can document that they don't work with applications that violate this convention, and relentless experimentalists still have the freedom to break the recommendation (at their own risk!) if they want to see what happens... Paul.

4 4