Mailman 3 June 1999 - Python-Dev

24 Jun '99

marangoz wrote: > > > How about introducing internal macros for explicit ob_refcnt accesses > in the core? Actually, there are a number of places where one can see > "op->ob_refcnt" logic, which could be replaced with _Py_GETREF(op), > _Py_SETREF(op, n) thus decoupling completely the low level refcount > management defined in object.h: > > #define _Py_GETREF(op) (((PyObject *)op)->ob_refcnt) > #define _Py_SETREF(op, n) (((PyObject *)op)->ob_refcnt = (n)) > > Comments? Of course, the above should be (PyObject *)(op)->ob_refcnt. Also, I forgot to mention that if this detail doesn't hurt code aesthetics, one (I) could experiment more easily all sort of weird things with refcounting... I formulated the same wish for malloc & friends some time ago, that is, use everywhere in the core PyMem_MALLOC, PyMem_FREE etc, which would be defined for now as malloc, free, but nobody seems to be very excited about a smooth transition to other kinds of malloc. Hence, I reiterate this wish, 'cause switching to macros means preparing the code for the future, even if in the future it remains intact ;-). Defining these basic interfaces is clearly Guido's job :-) as he points out in his summary of the last Open Source summit, but nevertheless, I'm raising the issue to let him see what other people think about this and allow him to make decisions easier :-) -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252 ----- End of forwarded message ----- -- Vladimir MARANGOZOV | Vladimir.Marangozov(a)inrialpes.fr http://sirac.inrialpes.fr/~marangoz | tel:(+33-4)76615277 fax:76615252

1 0

automatic wildcard expansion on Win32
by David Ascher 21 Jun '99

21 Jun '99

A python-help poster finally convinced me that there was a way to enable automatic wildcard expansion on win32. This is done by linking in "setargv.obj" along with all of the other MS libs. Quick testing shows that it works. Is this a feature we want to add? I can see both sides of that coin. --david PS: I saw a RISKS digest posting last week which had a horror story about wildcard expansion on some flavor of Windows. The person had two files with long filenames: verylongfile1.txt and verylongfile2.txt But Win32 stored them in 8.3 format, so they were stored as verylo~2.txt and verylo~1.txt (Yes, the 1 and 2 were swapped!). So when he did del *1.txt he removed the wrong file. Neat, eh? (This is actually relevant -- it's possible that setargv.obj and glob.glob could give different answers). --david

4 6

Relative package imports
by M.-A. Lemburg 21 Jun '99

21 Jun '99

Although David has already copy-posted a message regarding this issue to the list, I would like to restate the problem to get a discussion going (and then maybe take it to c.l.p for general flaming ;). The problem we have run into on starship is that some well-known packages have introduced naming conflicts leading to the unfortunate situation that they can't be all installed on the same default path: 1. Zope has a module named DateTime which also is the base name of the package mxDateTime. 2. Both Zope and PIL have a top-level module named ImageFile.py (different ones of course). Now the problem is how to resolve these issues. One possibility is turning Zope and PIL into proper packages altogether. To ease this transition, one would need a way to specify relative intra-package imports and a way to tell pickle where to look for modules/packages. The next problem we'd probably run into sooner or later is that there are quite a few useful top-level modules with generic names that will conflict with package names and other modules with the same name. I guess we'd need at least three things to overcome this situation once and for all ;-): 1. Provide a way to do relative imports, e.g. a single dot could be interpreted as "parent package": modA.py modD.py [A] modA.py modB.py [B] modC.py modD.py In modC.py: from modD import * (works as usual: import A.B.modD) from .modA import * (imports A.modA) from ..modA import * (import the top-level modA) 2. Establish a general vendor based naming scheme much like the one used in the Java world: from org.python.core import time,os,string from org.zope.core import * from com.lemburg import DateTime from com.pythonware import PIL 3. Add a way to prevent double imports of the same file. This is the mayor gripe I have with pickle currently, because intra- package imports often lead to package modules being imported twice leading to many strange problems (e.g. splitting class hierarchies, problems with isinstance() and issubclass(), etc.), e.g. from org.python.core import UserDict u = UserDict.UserDict() import UserDict v = UserDict.UserDict() Now u and v will point to two different classes: >>> u.__class__ <class org.python.core.UserDict.UserDict at 80d3b48> >>> v.__class__ <class UserDict.UserDict at 80aed18> 4. Add some kind of redirection or lookup hook to pickle et al. so that imports done during unpickling can be redirected to the correct (possibly renamed) package. -- Marc-Andre Lemburg ______________________________________________________________________ Y2000: 196 days left Business: http://www.lemburg.com/ Python Pages: http://www.lemburg.com/python/

3 5

stackable ints [stupid idea (ignore) :v]
by Aaron Watters 19 Jun '99

19 Jun '99

While we're talking about stacks... I've always considered it a major shame that Python ints and floats and chars and stuff have anything to do with dynamic allocation, and I always suspected it might be a major speed performance boost if there was some way they could be manipulated without the need for dynamic memory management. One conceivable alternative approach would change the basic manipulation of objects so that instead of representing objects via pyobject pointers everywhere represent them using two "slots" in a structure for each object, one of which is a type descriptor pointer and the other being a (void *) which could contain the data directly for small objects such as ints, floats, chars. In this case, for example, integer addition would never require any memory management, as it shouldn't, I think, in a perfect world. IE instead of C-stack or static: Heap: (pyobject *) ------------> (refcount, typedescr, data ...) in general you get (typedescr repr* ----------------------> (refcount, data, ...) ) or for small objects like ints and floats and chars simply (typedescr, value) with no dereferencing or memory management required. My feeling is that common things like arithmetic and indexing lists of integers and stuff could be much faster under this approach since it reduces memory management overhead and fragmentation, dereferencing, etc... One bad thing, of course, is that this might be a drastic assault on the way existing code works... Unless I'm just not being creative enough with my thinking. Is this a good idea? If so, is there any way to add it to the interpreter without breaking extension modules and everything else? If Python 2.0 will break stuff anyway, would this be an good change to the internals? Curious... -- Aaron Watters ps: I suppose another gotcha is "when do you do increfs/decrefs?" because they no longer make sense for ints in this case... maybe add a flag to the type descriptor "increfable" and assume that the typedescriptors are always in the CPU cache (?). This would slow down increfs by a couple cycles... Would it be worth it? Only the benchmark knows... Another fix would be to put the refcount in the static side with no speed penalty (typedescr repr* ----------------------> data refcount ) but would that be wasteful of space?

4 9

Flat Python in Linux Weekly
by Christian Tismer 18 Jun '99

18 Jun '99

Howdy, Who would have thought this... Linux Weekly took notice. http://lwn.net/bigpage.phtml derangedly yours - chris -- Christian Tismer :^) <mailto:tismer@appliedbiometrics.com> Applied Biometrics GmbH : Have a break! Take a ride on Python's Kaiserin-Augusta-Allee 101 : *Starship* http://starship.python.net 10553 Berlin : PGP key -> http://wwwkeys.pgp.net PGP Fingerprint E182 71C7 1A9D 66E9 9D15 D3CC D4D7 93E2 1FAE F6DF we're tired of banana software - shipped green, ripens at home

2 1

org.python.org
by David Ascher 17 Jun '99

17 Jun '99

Not all that revolutionary, but an interesting migration path. FWIW, I think the underlying issue is a real one. We're starting to have more and more conflicts, even among package names. (Of course the symlink solution doesn't work on Win32, but that's a detail =). --david ---------- Forwarded message ---------- Date: Thu, 17 Jun 1999 13:44:33 -0400 (EDT) From: Andy Dustman <adustman(a)comstar.net> To: Gordon McMillan <gmcm(a)hypernet.com> Cc: M.-A. Lemburg <mal(a)lemburg.com>, Crew List <crew(a)server.python.net> Subject: Re: [Crew] Wizards' Resolution to Zope/PIL/mxDateTime conflict? On Thu, 17 Jun 1999, Gordon McMillan wrote: > M.A.L. wrote: > > > Or maybe we should start the com.domain.mypackage thing ASAP. > > I know many are against this proposal (makes Python look Feudal? > Reminds people of the J language?), but I think it's the only thing > that makes sense. It does mean you have to do some ugly things to get > Pickle working properly. Actually, it can be done very easily. I just tried this, in fact: cd /usr/lib/python1.5 mkdir -p org/python (cd org/python; ln -s ../.. core) touch __init__.py org/__init__.py org/python/__init__.py >>> from org.python.core import rfc822 >>> import profile So this seems to make things nice and backwards compatible. My only concern was having __init__.py in /usr/lib/python1.5, but this doesn't seem to break anything. Of course, if you are using some trendy new atrocity like Windoze, this might not work. -- andy dustman | programmer/analyst | comstar communications corporation telephone: 770.485.6025 / 706.549.7689 | icq: 32922760 | pgp: 0xc72f3f1d _______________________________________________ Crew maillist - Crew(a)starship.python.net http://starship.python.net/mailman/listinfo/crew

2 1

RE: [Python-Dev] stackable ints [stupid idea (ignore) :v]
by Aaron Watters 17 Jun '99

17 Jun '99

> no-positive-suggestions-just-grousing-ly y'rs - tim On the contrary. I think this is definitively a bad idea. Retracted. A double negative is a positive. -- Aaron Watters === "Criticism serves the same purpose as pain. It's not pleasant but it suggests that something is wrong." -- Churchill (paraphrased from memory)

1 0

mmap
by David Ascher 16 Jun '99

16 Jun '99

Another topic: what are the chances of adding the mmap module to the core distribution? It's restricted to a smallish set of platforms (modern Unices and Win32, I think), but it's quite small, and would be a nice thing to have available in the core, IMHO. (btw, the buffer object needs more documentation) --david

7 14

String methods... finally
by Barry A. Warsaw 16 Jun '99

16 Jun '99

I've finally checked my string methods changes into the source tree, albeit on a CVS branch (see below). These changes are outgrowths of discussions we've had on the string-sig, with I think Greg Stein giving lots of very useful early feedback. I'll call these changes controversial (hence the branch) because Guido hasn't had much opportunity to play with them. Now that he -- and you -- can check them out, I'm sure I'll get lots more feedback! First, to check them out you need to switch to the string_methods CVS branch. On Un*x: cvs update -r string_methods You might want to do this in a separate tree because this will sticky tag your tree to this branch. If so, try cvs checkout -r string_methods python Here's a brief summary of the changes (as best I can restore the state -- its been a while since I actually made all these changes ;) Strings now have as methods most of the functions that were previously only in the string module. If you've played with JPython, you've already had this feature for a while. So you can do: Python 1.5.2+ (#1, Jun 10 1999, 18:22:14) [GCC 2.8.1] on sunos5 Copyright 1991-1995 Stichting Mathematisch Centrum, Amsterdam >>> s = 'Hello There Devheads' >>> s.lower() 'hello there devheads' >>> s.upper() 'HELLO THERE DEVHEADS' >>> s.split() ['Hello', 'There', 'Devheads'] >>> 'hello'.upper() 'HELLO' that sort of thing. Some of the string module functions don't make sense as string methods, like join, and others never had a C implementation so weren't added, like center. Two new methods startswith and endswith act like their Java cousins. The string module has been rewritten to be completely (I hope) backwards compatible. No code should break, though they could be slower. Guido and I decided that was acceptable. What else? Some cleaning up of the internals based on Greg's suggestions. A couple of new C API additions. Builtin int(), long(), and float() have grown a few new features. I believe they are essentially interchangable with string.atoi(), string.atol(), and string.float() now. After you guys get to toast me (in either sense of the word) for a while and these changes settle down, I'll make a wider announcement. Enjoy, -Barry

11 80

Re: [Python-Dev] Re: [Python-Dev] Re: [Python-Dev] String methods... finally
by Jack Jansen 15 Jun '99

15 Jun '99

> The same should happen as for L"foo" + " " + L"bar". This is probably the most reasonable solution. Unfortunately it breaks Marks truly novel suggestion that 0.join(1, 2) becomes 102, but I guess we'll have to live with that:-) -- Jack Jansen | ++++ stop the execution of Mumia Abu-Jamal ++++ Jack.Jansen(a)oratrix.com | ++++ if you agree copy these lines to your sig ++++ www.oratrix.nl/~jack | see http://www.xs4all.nl/~tank/spg-l/sigaction.htm

3 2