adam
Send Python-Dev mailing list submissions to python-dev@python.org
To subscribe or unsubscribe via the World Wide Web, visit http://mail.python.org/mailman/listinfo/python-dev or, via email, send a message with subject or body 'help' to python-dev-request@python.org
You can reach the person managing the list at python-dev-admin@python.org
When replying, please edit your Subject line so it is more specific than "Re: Contents of Python-Dev digest..."
Today's Topics:
1. ArtifactFile: I can't change my patch files (James C. Ahlstrom) 2. Re: problems with DBM nonuniformity (Jason R. Mastaler) 3. Re: -DINET6 in Makefile (Martin v. Loewis) 4. Re: -DINET6 in Makefile (M.-A. Lemburg) 5. RE: gc.garbage (Tim Peters) 6. RE: gc.garbage (Tim Peters) 7. RE: Subclassing varying length types (What's a PyStructSequence ?) (Tim Peters) 8. Re: -DINET6 in Makefile (Martin v. Loewis) 9. Re: gc.garbage (Martin v. Loewis) 10. test_builtin failing? or just 64-bit platforms (Mark Favas) 11. RE: test_builtin failing? or just 64-bit platforms (Tim Peters) 12. Re: test_builtin failing? or just 64-bit platforms (Guido van Rossum) 13. RE: test_builtin failing? or just 64-bit platforms (Tim Peters) 14. RE: test_builtin failing? or just 64-bit platforms (Barry A. Warsaw) 15. RE: test_builtin failing? or just 64-bit platforms (Tim Peters)
--__--__--
Message: 1 Date: Fri, 30 Nov 2001 12:07:03 -0500 From: "James C. Ahlstrom" <jim@interet.com> To: python-dev@python.org Subject: [Python-Dev] ArtifactFile: I can't change my patch files
I am trying to replace files in my patch #483466, but I am getting
File delete: ArtifactFile: Permission Denied
on SourceForge. If I need to be a member of the Python project to do this, could someone please add me? Or tell me what's wrong?
The new files implement the changes to sys.path[0] that we have been discussing. I still need to make further changes to import.c so that "" and relative paths are handled. So don't install this patch yet, but look it over for problems if you want.
Jim
--__--__--
Message: 2 To: python-list@python.org, python-dev@python.org, Date: Fri, 30 Nov 2001 10:17:24 -0700 From: "Jason R. Mastaler" <jason-dated-1007831845.6c6a51@mastaler.com> Subject: [Python-Dev] Re: problems with DBM nonuniformity
Skip Montanaro <skip@pobox.com> writes:
Seems to me the natural thing to do would be to add "get_data_filename" and "get_index_filename" methods (or something similar) to the underlying modules (dbhash, bsddb, dbm, etc) and expose them through anydbm.
I see. So you agree that with the current implementation, there isn't a reliable way to do what I'm trying to do with DBM?
It's too late for 2.2, but I suspect if you implemented something and method name(s) could be settled on it would make it into CVS early in the 2.3 cycle. This seems like a small enough change that you just file a bug report on SourceForge with the proposal and add an implementation when you have something workable.
I'm not sure when I'll be able to get to this, but I'll put it on my TODO list. In the meantime, I think I'll just support the auto regeneration feature I mentioned with CDB[1] instead of DBM since its file interface is consistent across platforms.
Footnotes: 1. python-cdb extension module (http://pilcrow.madison.wi.us/)
-- (TMDA - http://tmda.sourceforge.net) (Python-based SPAM reduction system)
--__--__--
Message: 3 Date: Fri, 30 Nov 2001 21:22:24 +0100 From: "Martin v. Loewis" <martin@v.loewis.de> To: mal@lemburg.com CC: python-dev@python.org Subject: Re: [Python-Dev] -DINET6 in Makefile
What's the reasoning behind putting -DINET6 into the default compiler options of the generic Makefile ?
I believe the sole reason is that the author of the patch didn't know how to get it into pyconfig.h. itojun recently confirmed that all uses of the INET6 can be replaced with ENABLE_IPV6, and that the define may go away.
I hesitate to change that, though, since some of the IPv6 implementations *may* require that INET6 is defined when processing the "system" headers (not all IPv6 implementations we support actually come with the operating system).
I'm just asking because such a define will be inherited by all extensions being compiled with distutils and the Makefile.pre.in setup process... sounds like trouble if you ask me.
What kind of trouble?
Shouldn't the define be placed into the pyconfig.h file or only in those extensions which need it ?
Wouldn't it cause the same trouble there?
Regards, Martin
--__--__--
Message: 4 Date: Fri, 30 Nov 2001 21:46:14 +0100 From: "M.-A. Lemburg" <mal@lemburg.com> Organization: eGenix.com Software GmbH To: "Martin v. Loewis" <martin@v.loewis.de> CC: python-dev@python.org Subject: Re: [Python-Dev] -DINET6 in Makefile
"Martin v. Loewis" wrote:
What's the reasoning behind putting -DINET6 into the default compiler options of the generic Makefile ?
I believe the sole reason is that the author of the patch didn't know how to get it into pyconfig.h. itojun recently confirmed that all uses of the INET6 can be replaced with ENABLE_IPV6, and that the define may go away.
I hesitate to change that, though, since some of the IPv6 implementations *may* require that INET6 is defined when processing the "system" headers (not all IPv6 implementations we support actually come with the operating system).
For Python's own use, it should suffice defining the symbol in pyconfig.h.
I'm just asking because such a define will be inherited by all extensions being compiled with distutils and the Makefile.pre.in setup process... sounds like trouble if you ask me.
What kind of trouble?
The symbol could enable some logic which may not be desired by the application, e.g. cause system includes to change, socket semantics of wrapped libs could also be affected etc.
Shouldn't the define be placed into the pyconfig.h file or only in those extensions which need it ?
Wouldn't it cause the same trouble there?
No, because the pyconfig.h import is under extension control (e.g. you can first include the system or lib header files and only then import pyconfig.h).
-- Marc-Andre Lemburg CEO eGenix.com Software GmbH ______________________________________________________________________ Consulting & Company: http://www.egenix.com/ Python Software: http://www.lemburg.com/python/
--__--__--
Message: 5 From: "Tim Peters" <tim.one@home.com> To: "Neil Schemenauer" <nas@python.ca>, "PythonDev" <python-dev@python.org> Subject: RE: [Python-Dev] gc.garbage Date: Fri, 30 Nov 2001 18:30:43 -0500
[Neil Schemenauer]
Is there some way to prevent people from assigning to certain module variables?
[Tim]
Not that I know of. If you're terribly concerned, gc could look up "garbage" in its dict on each access. That's what, e.g., PRINT_ITEM does with sys.stdout. ...
[Neil]
What would happen if it's not a list? PRINT_ITEM raises RuntimeError. I suppose the collector could do the same.
Sure, that would be fine.
But I'd be keener to see new words spelling out which parts of the gc interface are and aren't intended "to work" across releases ...
All of them? :-) Seriously, there could come a time when GC can no longer be disabled. The debugging and threshold stuff is fairly implementation dependent. get_referrers() and get_objects() are highly implementation dependent. I suppose gc.collect() should always be available. Anything else is fair game, IMHO.
I meant "new words" in the docs, not on Python-Dev <wink>.
Incidentally, I can't say I'm happy with GC as it stands.
Well, you're young and hopeful -- you'll get over both. I have, and am indeed happy with GC as it stands.
It uses too much memory now that so many objects are tracked.
There I disagree, but subtly: it always used "too much" memory. The marginal memory cost in adding a gazillion new tracked types was minor, as very few programs have a gazillion frame objects or traceback objects or generator-iterator objects (etc) sitting around. The vast bulk of the damage was done the instant lists, tuples, dicts and instances got
So it goes.
I had worked on the idea of a separate heap for GC objects for a while but couldn't figure out how to make generational collection to work.
Generational gimmicks are rare in non-copying collectors for this very reason, right?
As Don Beaudry's sig says: "so much code, so little time". :-)
Time for Don to change his sig -- his young and hopeful days should be long gone by now too <wink>.
--__--__--
Message: 6 From: "Tim Peters" <tim.one@home.com> To: <python-dev@python.org> Subject: RE: [Python-Dev] gc.garbage Date: Fri, 30 Nov 2001 18:30:44 -0500
[Martin v. Loewis]
I wish the C API hadn't been changed for 2.2, rendering useless all code that had been created to support GC in 2.0 and 2.1.
Would we really need more than one hand to count all that code <0.9 wink>?
not-aware-of-any-myself-ly y'rs - tim
--__--__--
Message: 7 From: "Tim Peters" <tim.one@home.com> To: <python-dev@python.org> Subject: RE: [Python-Dev] Subclassing varying length types (What's a PyStructSequence ?) Date: Fri, 30 Nov 2001 18:35:23 -0500
[M.-A. Lemburg]
... Hmm, this makes me wonder: perhaps we should start thinking about phasing out varying length PyObjects in the interpreter...
No chance, IMO: the memory savings is too great.
esp. the inability to subclass strings looks like a bummer for future extensions of this particular type. Unicode doesn't have this problem, BTW.
OTOH, I know someone at Zope Corp who could testify with force about the memory burden of switching to Unicode strings -- if you've got gobs of 'em, it's much worse than a factor of 2 blowup. Moving to obmalloc.c should help that a lot (two malloc overheads per Unicode string, and obmalloc overheads are much lower).
Or we need to come up with a fairly nice way of making subclassing varying length types a lot easier, e.g. by adding a special pointer ob_ext to PyObject_VAR_HEAD which then allows declaring type extensions in an malloced buffer.
Thoughts ?
Not a one <wink>.
--__--__--
Message: 8 Date: Sat, 1 Dec 2001 02:07:19 +0100 From: "Martin v. Loewis" <martin@v.loewis.de> To: mal@lemburg.com CC: python-dev@python.org Subject: Re: [Python-Dev] -DINET6 in Makefile
Shouldn't the define be placed into the pyconfig.h file or only in those extensions which need it ?
Wouldn't it cause the same trouble there?
No, because the pyconfig.h import is under extension control (e.g. you can first include the system or lib header files and only then import pyconfig.h).
Of course, doing so would be really stupid. Python.h *must* be the first include, or things may break.
Regards, Martin
--__--__--
Message: 9 Date: Sat, 1 Dec 2001 02:11:34 +0100 From: "Martin v. Loewis" <martin@v.loewis.de> To: tim.one@home.com CC: python-dev@python.org Subject: Re: [Python-Dev] gc.garbage
[Martin v. Loewis]
I wish the C API hadn't been changed for 2.2, rendering useless all code that had been created to support GC in 2.0 and 2.1.
Would we really need more than one hand to count all that code <0.9 wink>?
Mine was among it (in pyexpat), and I really hate looking at the code now. Not only need I support two versions of Unicode (i.e. having or not having it), but also three versions of GC, all in a single file.
Regards, Martin
--__--__--
Message: 10 Date: Sat, 01 Dec 2001 09:55:49 +0800 From: Mark Favas <mark.favas@csiro.au> Organization: CSIRO To: python-dev@python.org Subject: [Python-Dev] test_builtin failing? or just 64-bit platforms
Anyone else getting test_builtin failures with current CVS, or does it only show on 64-bit platforms? Changes in the past week seem to have caused the failure. Isolated to following (will post bug on SF):
***** older CVS works ***** 201 mark@gonzo.per.dem.csiro.au python Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4 Type "help", "copyright", "credits" or "license" for more information.
import sys s=-1-sys.maxint d=`s` d '-9223372036854775808' s -9223372036854775808 ^D
***** current CVS fails ***** 202 mark@gonzo.per.dem.csiro.au cd dist/src 203 mark@gonzo.per.dem.csiro.au ./python Python 2.2b2+ (#541, Dec 1 2001, 08:04:58) [C] on osf1V4 Type "help", "copyright", "credits" or "license" for more information.
import sys s=-1-sys.maxint d=`s` d '8t\x10@\x01' s -9223372036854775808 ^D
-- Mark Favas - m.favas@per.dem.csiro.au CSIRO, Private Bag No 5, Wembley, Western Australia 6913, AUSTRALIA
--__--__--
Message: 11 From: "Tim Peters" <tim.one@home.com> To: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms Date: Fri, 30 Nov 2001 21:41:56 -0500
[Mark Favas]
Anyone else getting test_builtin failures with current CVS,
No.
or does it only show on 64-bit platforms? Changes in the past week seem to have caused the failure. Isolated to following (will post bug on SF):
Thanks!
***** older CVS works ***** 201 mark@gonzo.per.dem.csiro.au python Python 2.2b2+ (#539, Nov 26 2001, 09:52:25) [C] on osf1V4 Type "help", "copyright", "credits" or "license" for more information.
import sys s=-1-sys.maxint d=`s` d '-9223372036854775808' s -9223372036854775808 ^D
***** current CVS fails ***** 202 mark@gonzo.per.dem.csiro.au cd dist/src 203 mark@gonzo.per.dem.csiro.au ./python Python 2.2b2+ (#541, Dec 1 2001, 08:04:58) [C] on osf1V4 Type "help", "copyright", "credits" or "license" for more information.
import sys s=-1-sys.maxint d=`s` d '8t\x10@\x01' s -9223372036854775808 ^D
It doesn't make sense, but it *smells* like a sprintf -> PyOS_snprintf screwup ... OK, our int repr code has always been wrong(!):
static PyObject * int_repr(PyIntObject *v) { char buf[20]; PyOS_snprintf(buf, sizeof(buf), "%ld", v->ob_ival); return PyString_FromString(buf); }
20 bytes isn't enough to hold the result on a 64-bit box (insert rant about the idiot practice of trying to make stack buffers as small as possible). You have 20 characters in your result, but need 21 to hold the trailing 0 byte too. I don't know what snprintf does when there's not enough room, but I think you just showed us what it does on Tru64 <wink>.
Doing repr impllicitly instead works because that goes thru int_print instead of int_repr. Doing repr explicitly "worked" before by accident (the trailing null overwrite some random byte on the stack). Please change 20 to 200(*) (or 64 -- your choice) and see whether that fixes it?
--__--__--
Message: 12 To: "Tim Peters" <tim.one@home.com> cc: "Mark Favas" <mark.favas@csiro.au>, python-dev@python.org Subject: Re: [Python-Dev] test_builtin failing? or just 64-bit platforms From: Guido van Rossum <guido@python.org> Date: Fri, 30 Nov 2001 22:01:24 -0500
I don't know what snprintf does when there's not enough room, but I think you just showed us what it does on Tru64 <wink>.
Hm, snprintf is *supposed* to truncate the result, but it seems that here it refused to do anything. Maybe PyOS_snprintf should be a wrapper that checks the return value of snprintf? I noticed that *none* of the recently checked-in PyOS_snprintf calls have their return value checked, and I think it's better to leave it that way (since in many cases we really don't know what to do instead) -- maybe PyOS_snprintf should even return void (or does it already?).
--Guido van Rossum (home page: http://www.python.org/~guido/)
--__--__--
Message: 13 From: "Tim Peters" <tim.one@home.com> To: "Guido van Rossum" <guido@python.org> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms Date: Fri, 30 Nov 2001 22:42:58 -0500
Hm, snprintf is *supposed* to truncate the result,
According to C99, yes, but MS has its own pre-C99 snprintf semantics, and glibc has had at least two different versions.
but it seems that here it refused to do anything. Maybe PyOS_snprintf should be a wrapper that checks the return value of snprintf?
Check it for what? We can't (at least not yet) count on uniform behavior across platform snprintf implementations.
I noticed that *none* of the recently checked-in PyOS_snprintf calls have their return value checked,
They were all derived (and most mindlessly) from existing sprintf calls
didn't check either. A goal of this transformation was to change as
----- Original Message ----- From: <python-dev-request@python.org> To: <python-dev@python.org> Sent: Saturday, December 01, 2001 6:33 AM Subject: Python-Dev digest, Vol 1 #1748 - 15 msgs tracked. that little
as possible.
and I think it's better to leave it that way (since in many cases we really don't know what to do instead) -- maybe PyOS_snprintf should even return void (or does it already?).
The comments suggest it wants to return the C99-defined value (an int, < 0 for an encoding error, else the number of characters (excluding \0) written if the size is big enough, else the number of characters that would have been written (excluding \0) had size been big enough. So the output was converted in full iff the return value is non-negative and strictly less than the size passed in. That's fine by me, and I'll note that MS snprintf meets that much too (if size isn't big enough, it returns a negative result). So perhaps in a debug build PyOS_snprintf could assert that the returned value is non-negative and less than the passed-in size ...
--__--__--
Message: 14 Date: Fri, 30 Nov 2001 23:17:31 -0500 To: "Tim Peters" <tim.one@home.com> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms From: barry@zope.com (Barry A. Warsaw)
"TP" == Tim Peters <tim.one@home.com> writes:
TP> It doesn't make sense, but it *smells* like a sprintf -> TP> PyOS_snprintf screwup ... OK, our int repr code has always TP> been wrong(!):
| static PyObject * | int_repr(PyIntObject *v) | { | char buf[20]; | PyOS_snprintf(buf, sizeof(buf), "%ld", v->ob_ival); | return PyString_FromString(buf); | }
TP> 20 bytes isn't enough to hold the result on a 64-bit box TP> (insert rant about the idiot practice of trying to make stack TP> buffers as small as possible). You have 20 characters in your TP> result, but need 21 to hold the trailing 0 byte too. I don't TP> know what snprintf does when there's not enough room, but I TP> think you just showed us what it does on Tru64 <wink>.
Heh, I was going to suggest that this might be a good place to substitute a call to PyString_FromFormat*() but then I read this little nugget:
case 'd': case 'i': case 'x': (void) va_arg(count, int); /* 20 bytes should be enough to hold a 64-bit integer */ n += 20; break;
love-ly y'rs, -Barry
--__--__--
Message: 15 From: "Tim Peters" <tim.one@home.com> To: "Barry A. Warsaw" <barry@zope.com> Cc: "Mark Favas" <mark.favas@csiro.au>, <python-dev@python.org> Subject: RE: [Python-Dev] test_builtin failing? or just 64-bit platforms Date: Fri, 30 Nov 2001 23:31:51 -0500
[Barry]
... Heh, I was going to suggest that this might be a good place to substitute a call to PyString_FromFormat*() but then I read this little nugget:
case 'd': case 'i': case 'x': (void) va_arg(count, int); /* 20 bytes should be enough to hold a 64-bit integer */ n += 20; break;
This use of 20 is fine (I checked all occurrences of "20" in the codebase, BTW); int_repr's use of 20 was wrong because it failed to allow for the trailing \0 byte too; the loop in PyString_FromFormatV doesn't have to worry about that (PyString_FromStringAndSize() later adds 1 of its own for the trailing \0 -- of course it doesn't actually add anything, but it adds 1 "in effect" <wink -- jeez, C string handling is obscure!>).
--__--__--
_______________________________________________ Python-Dev mailing list Python-Dev@python.org http://mail.python.org/mailman/listinfo/python-dev
End of Python-Dev Digest
tags
participants (1)
-
adam