Mailman 3 June 2002 - Python-Dev

Slicing
by David Abrahams 18 Jun '02

18 Jun '02

I did a little experiment to see if I could use a uniform interface for slicing (from C++): >>> range(10)[slice(3,5)] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: sequence index must be integer >>> class Y(object): ... def __getslice__(self, a, b): ... print "getslice",a,b ... >>> y = Y() >>> y[slice(3,5)] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: unsubscriptable object >>> y[3:5] getslice 3 5 This seems to indicate that I can't, in general, pass a slice object to PyObject_GetItem in order to do slicing.** Correct? So I went looking around for alternatives to PyObject_GetItem. I found PySequence_GetSlice, but that takes int parameters, and AFAIK there's no rule saying you can't slice on strings, for example. Further experiments revealed: >>> y['hi':'there'] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: unsubscriptable object >>> class X(object): ... def __getitem__(self, x): ... print 'getitem',x ... >>> X()['hi':'there'] getitem slice('hi', 'there', None) So I /can/ slice on strings, but only through __getitem__(). And... >>> class Z(Y): ... def __getitem__(self, x): ... print 'getitem',x ... >>> Z()[3:5] getslice 3 5 >>> Z()['3':5] getitem slice('3', 5, None) So Python is doing some dispatching internally based on the types of the slice elements, but: >>> class subint(int): pass ... >>> subint() 0 >>> Z[subint():5] Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: unsubscriptable object So it's looking at the concrete type of the slice elements. I'm not sure I actually understand how this one fails. I want to make a generalized getslice function in C which can operate on a triple of arbitrary objects. Here's the python version I came up with: def getslice(x,start,finish): if (type(start) is type(finish) is int and hasattr(type(x), '__getslice__')): return x.__getslice__(start, finish) else: return x.__getitem__(slice(start,finish)) Have I got the logic right here? Thanks, Dave **it seems like a good idea to make it work in the Python core, by recognizing slice objects and dispatching the elements to __getslice__ if they are ints and if one is defined. Have I overlooked something? +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

1 0

MultiDict / Table, suggestion for a new module
by Edward C. Jones 18 Jun '02

18 Jun '02

I have written a module called MultiDict.py which can be found at http://members.tripod.com/~edcjones/MultiDict.py . It contains two classes MultiDict and Table. MultiDict is like a dictionary except that each key can occur more than once. It is like the multimap in the C++ Standard Template Library except that MultiDicts are hashed rather than sorted. Table views a 2 dimensional nested list as a "relation" (a set of n-tuples). I use it for simple one table databases where the full panoply of SQL is not needed. If there is interest in this, I will write a PEP and some documentation. Thanks, Edward C. Jones

1 0

large file support
by Jeremy Hylton 18 Jun '02

18 Jun '02

I've run into a problem with large files using Python 2.1.2 and a Linux 2.4.9 box. We've got a large file -- almost 6GB -- that Python chokes on even though regular shell tools seem to be fine. In particular, os.stat() of the file fails with EOVERFLOW and open() of the file fails with EFBIG. The stat() failure is really bad because it means os.path.exists() returns false. strace tells me that other tools open the file passing O_LARGEFILE, but Python does not. (They pass it even for small files.) I can't find any succient explanation of O_LARGEFILE, but Google turns up all sorts of pages that mention it. It looks like the right way to open large files, but it only seems to be defined in <asm/fcntl.h> on the Linux box in question. I haven't had any luck searching for a decent way to invoke stat() and have it be prepared for a very large file. I think Python is definitely broken here. Can anyone offer any clues or pointers to documentation? Better yet, a fix. I'm happy to help integrate and test it. Jeremy

8 23

Quota on sf.net
by loewis＠informatik.hu-berlin.de 18 Jun '02

18 Jun '02

It appears SF is rearranging servers, and asks projects to honor their disk quota, see https://sourceforge.net/forum/forum.php?forum_id=183601 There is a per-project disk quota of 100MB; /home/groups/p/py/python currently consumes 880MB. Most of this (830MB) is in htdocs/snapshots. Should we move those onto python.org? Regards, Martin

8 12

addressing distutils inability to track file dependencies
by Skip Montanaro 18 Jun '02

18 Jun '02

I wonder if it would be better to have distutils generate the appropriate type of makefile and execute that instead of directly building objects and shared libraries. This would finesse some of the dependency tracking problems that pop up frequently. Skip

28 109

RE: [Python-Dev] Playstation 2 and GameCube ports
by Gustavo Cordova 18 Jun '02

18 Jun '02

> > > That would certainly get my son's attention and might even > > get him started in programming. I wouldn't mind seeing your > > efforts written up in a PEP. > > What exactly can you accomplish with Python on one of these boxes? > > Don't you need a (costly) developers license in order to use this? > > --Guido van Rossum (home page: http://www.python.org/~guido/) > Nope, just hack away at it in PS2-Linux, quite nice; you can distribute your own games. Now, if there's a PyGame and PyGL for PS2, then I think that some very cool hacks and demos are gonna start appearing for the piss-two in a little time. :-) -gus

1 0

unicode() and its error argument
by Skip Montanaro 18 Jun '02

18 Jun '02

The unicode() builtin accepts an optional third argument, errors, which defaults to "strict". According to the docs if errors is set to "ignore", decoding errors are silently ignored. I seem to still get the occasional UnicodeError exception, however. I'm still trying to track down an actual example (it doesn't happen often, and I hadn't wrapped unicode() in a try/except statement, so all I saw was the error raised, not the input string value). This reminds me, it occurred to me the other day that a plain text version of cgitb would be useful to use for non-web scripts. You'd get a lot more context about the environment in which the exception was raised. Skip

6 11

Re: [Python-checkins] python/dist/src/Objects bufferobject.c,2.17,2.18 classobject.c,2.156,2.157 descrobject.c,2.26,2.27 funcobject.c,2.54,2.55 sliceobject.c,2.14,2.15
by Michael Hudson 18 Jun '02

18 Jun '02

gvanrossum(a)users.sourceforge.net writes: > Index: classobject.c [...] > + static PyObject * > + new_class(PyObject* unused, PyObject* args) > + { > + PyObject *name; > + PyObject *classes; > + PyObject *dict; > + > + if (!PyArg_ParseTuple(args, "SO!O!:class", > + &name, > + &PyTuple_Type, &classes, > + &PyDict_Type, &dict)) > + return NULL; > + return PyClass_New(classes, dict, name); > + } > + What's this for? It's not referred to anywhere, so I'm getting warnings about it. I'd just hack it out, but it only just got added... Cheers, M. -- ARTHUR: Don't ask me how it works or I'll start to whimper. -- The Hitch-Hikers Guide to the Galaxy, Episode 11

2 1

Re: [Python-Dev] test_socket failures
by Michael Gilfix 18 Jun '02

18 Jun '02

Guido, please let me know if you want me to do anything more regarding the test_socket.py stuff and perhaps some timeout stuff before I take off (this wednsday). A note for the interested: I'll be gone for a month on vacation, so response won't be timely. Regards, -- Mike -- Michael Gilfix mgilfix(a)eecs.tufts.edu For my gpg public key: http://www.eecs.tufts.edu/~mgilfix/contact.html

1 0

behavior of inplace operations
by David Abrahams 17 Jun '02

17 Jun '02

My initial post asking about the implementation of += sparked a small thread over on python-list, from which I've come to the conclusion that my little optimization suggestion (don't try to set the attribute or item if the inplace op returns its first argument) is actually more semantically correct. For better or worse, these ideas aren't all mine, as http://aspn.activestate.com/ASPN/Mail/Message/python-list/1222524 attests. Consider: >>> t = ([1],[2],[3]) >>> t[0].append(2) # OK, the elements of the tuple are mutable >>> t ([1, 2], [2], [3]) >>> >>> t[1] += [3] # ?? Just did the equivalent operation above Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: object doesn't support item assignment >>> t # Despite the exception, the operation succeeded! ([1, 2], [2, 3], [3]) So, the exception happens because the user is ostensibly trying to modify this immutable tuple... but of course she's not. She's just trying to modify the element of the tuple, which is itself mutable, and that makes the exception surprising. Even more surprising in light of the exception is the fact that everything seems to have worked. In order to get this all to make sense, she needs to twist her brain into remembering that inplace operations don't really just modify their targets "in place", but also try to "replace" them. However, if we just set up the inplace operations so that when they return the original object there's no "replace", all of these problems go away. We don't lose any safety; trying to do += on an immutable tuple element will still fail. Also it makes tuples a generic replacement for lists in more places. There are other, more-perverse cases which the proposed change in semantics would also fix. For example: >>> class X(object): ... def __init__(self, l): ... self.container = l # will form a cycle ... self.stuff = [] ... def __iadd__(self, other): ... self.stuff += other # add to X's internal list ... del self.container[0] ... return self ... >>> l = [ 1, 2, 3] >>> l.append(X(l)) # the X element refers to l >>> l [1, 2, 3, <__main__.X object at 0x00876668>] >>> l[3] += 'a' # the element is gone by write-back time. Traceback (most recent call last): File "<stdin>", line 1, in ? IndexError: list assignment index out of range >>> l # But everything succeeded [2, 3, <__main__.X object at 0x00876668>] >>> l[2].stuff ['a'] >>> l.append('tail') # this one's even wierder >>> l[2] += 'a' >>> l [3, <__main__.X object at 0x00876668>, <__main__.X object at 0x00876668>] These are too esoteric to be compelling on their own, but my proposal would also make them work as expected. Thoughts? -Dave ------------ P.S. This error message is kind of wierd: >>> t = ([1],[2],[3]) >>> t[1] += 3 Traceback (most recent call last): File "<stdin>", line 1, in ? TypeError: argument to += must be iterable ^^^^^^^^^^^^^^^^ ?? +---------------------------------------------------------------+ David Abrahams C++ Booster (http://www.boost.org) O__ == Pythonista (http://www.python.org) c/ /'_ == resume: http://users.rcn.com/abrahams/resume.html (*) \(*) == email: david.abrahams(a)rcn.com +---------------------------------------------------------------+

4 11